WO2023065222A1 - Microscope synchronization control system and method - Google Patents

Abstract

A microscope synchronization control system and method. The system comprises a microscope high-speed synchronization control device (2), a camera (3), a laser (4), an LED light source (5), and a spatial light modulator (6); an upper computer (1) is connected to the microscope high-speed synchronization control device (2) via a serial port, and is used for sending a specific setting sequence and a data sequence to set the microscope high-speed synchronization control device (2) or sending a trigger sequence to control the microscope high-speed synchronization control device (2) to generate a digital signal. Synchronous operation of the camera (3), the laser (4), the LED light source (5), the spatial light modulator (6), and an analog signal control piezoelectric ceramic Z stage (7) is controlled according to the digital signal by means of the microscope high-speed synchronization control device (2) to achieve accurate photographing of the camera in a complex microscope system.

Description

A microscope synchronization control system and method technical field

The application belongs to the technical field of microscope imaging control, and in particular relates to a microscope synchronous control system and method.

Background technique

Microscopic imaging is widely used in biological research and is an important research tool to reveal life processes. The core of the microscope imaging control system is multi-dimensional image acquisition (dimensions include time t, XY position, Z-axis slice, and multiple fluorescence channels). In order to enable the microscope to perform shooting tasks according to the predetermined process, the traditional method is to send Commands to change the parameters of each device in turn or issue action instructions, inevitably need to communicate with each device through software, this communication often takes a few milliseconds, resulting in additional delays (several seconds) when shooting each frame of image ten milliseconds). In addition, in the traditional method, the time sequence required for camera shooting is usually issued by the software in the computer, and the clock sequence controlled by the software controls the time point of image acquisition. Due to the non-real-time nature of the operating system, this time sequence may have a few milliseconds fluctuations of tens of milliseconds. For high-speed microscope imaging, such as the study of bacterial swimming requires an acquisition speed of 30Hz or higher, precise time control cannot be achieved through computer software, and TTL needs to be used as an external trigger signal to control the shooting of the microscope.

Improving the data collection efficiency of microscope images is of great significance to scientific research. It can 1) increase the capacity of data sets required for scientific research and improve the statistical reliability of results; 2) shorten the time required to collect the same data, Improve the timeliness of measurement. Reducing unnecessary delay time is the basic method to improve acquisition efficiency. It requires the microscope system to achieve high-speed and high-precision control of cameras, electric translation stages, and light sources in time and space. Turn on the light source corresponding to the wavelength, and start the next stage movement immediately after the camera exposure is completed, so as to realize the low delay of the camera collecting images in time (t) and space (x, y, z) in this cycle. For fluorescence imaging, within the exposure time of the camera, if the excitation light source is turned on behind or turned off early, the fluorescence excitation will be weakened, and the early turn on or delayed turn off of the excitation light source will increase the toxicity and photobleaching of biological samples. The camera imaging time and turn on the excitation A high degree of time synchronization of light sources is a prerequisite for long-term and high-frequency fluorescence image acquisition. However, traditional software control methods cannot meet such high-precision timing requirements, so there is an urgent need for a high-precision and high-speed image acquisition hardware control device, which is of great significance for high-speed and high-throughput image acquisition of fluorescence microscopes.

Contents of the invention

The embodiment of the present application provides a microscope synchronous control system and method. The system has multiple digital signal outputs to realize accurate shooting of complex processes in the system.

According to an embodiment of the present application, a microscope synchronization control system is provided, including: a host computer, a microscope high-speed synchronization control device, a camera, a laser, an LED light source, and a spatial light modulator;

The host computer is connected to the microscope high-speed synchronization control device through the serial port, and is used to send specific setting sequences and data sequences to set the microscope high-speed synchronization control device or send trigger sequences to control the microscope high-speed synchronization control device to generate digital signals;

The microscope high-speed synchronous control device controls the synchronous operation of the camera, laser, LED light source and spatial light modulator according to the digital signal. The digital signal includes the camera exposure trigger signal, the LED light source trigger signal, the laser trigger signal and the spatial light modulator trigger signal;

The microscope high-speed synchronization control device sends the camera shooting trigger signal to the camera to control the camera to shoot; the microscope high-speed synchronization control device sends a digital signal to the LED power supply to control the LED power supply to turn on;

The microscope high-speed synchronization control device sends the laser trigger signal to the laser, and the microscope high-speed synchronization control device sends the modulator trigger signal to the spatial light modulator. When the camera is shooting, the spatial light modulator is triggered, and the spatial light modulator outputs a TTL signal to the microscope high-speed Synchronization control device, microscope high-speed synchronization control device sends the laser trigger signal and the spatial light modulator trigger signal to the laser and the spatial modulator after "ANDing" to synchronize the input signals of the spatial light modulator and the laser to realize the LED light source And the timing of the camera is precisely controlled to make the camera shoot.

The technical solution adopted in the embodiment of the present application also includes: the system also includes a motorized displacement platform, and the digital signal also includes a trigger signal of the motorized displacement platform. The received trigger signal of the electric stage moves in the X-Y two-dimensional plane;

After the camera shoots, the high-speed synchronous control device of the microscope will send a trigger signal to the motorized displacement platform to move the motorized displacement platform to the next preset position;

After the motorized stage moves to the preset position, an external trigger signal will be sent to the high-speed synchronization control device of the microscope to start the next shooting cycle.

The technical solution adopted in the embodiment of the present application also includes: the system also includes a piezoelectric ceramic Z stage, and the digital signal also includes an analog signal of the Z stage, and the high-speed synchronization control device of the microscope sends the Z stage analog signal to the piezoelectric ceramic Z stage, and the piezoelectric ceramic Z stage The station moves up and down according to the received analog signal of station Z;

After the camera shooting cycle ends, the piezoelectric ceramic Z stage will move up and down.

The technical solution adopted in the embodiment of the present application also includes: the microscope high-speed synchronous control device includes an integrated circuit board and an external power adapter, the integrated circuit board is provided with a main control chip, the main control chip is connected to two digital-analog chips, Each chip is connected with an op-amp filter, and the power adapter supplies power to the system; the power adapter provides the voltage converted by the transformer chip for the main control chip, and the power adapter provides the digital-analog chip voltage for the two digital-analog chips, and the power supply for the two op-amps. The filter provides a filtered voltage;

The two operational amplifier filters output analog signals respectively, and the main control chip provides sixteen digital signal outputs.

The technical solution adopted in the embodiment of the present application also includes: the integrated circuit board is also provided with a host computer communication transcoding circuit, an electronic erasable programmable read-only storage circuit, a power supply isolation and voltage stabilization circuit, a digital-to-analog signal conversion circuit, filter and amplifier circuit;

The upper computer communication transcoding circuit is used to realize the two-way communication between the upper computer and the microscope high-speed synchronous control device;

The electronic erasable programmable read-only storage circuit has the function of supporting multiple rewriting and power-off storage at the same time, and can realize the pre-stored data sequence to the storage module of the control box, and provide the function of power-off storage;

Power supply isolation and voltage stabilization circuit, used to isolate input voltage and analog voltage, and filter the voltage;

A digital-to-analog signal conversion circuit is used to generate an analog signal with 14-bit resolution to achieve high synchronization during the microscope shooting process;

The filtering and amplifying circuit is used for amplifying the voltage and performing filtering processing on the amplified voltage.

Another technical solution adopted in the embodiment of the present application is: a microscope synchronization control method, comprising the following steps:

The microscope high-speed synchronization control device receives the data sequence, and the microscope high-speed synchronization control device checks whether the data sequence meets the specification;

If it meets the specification, store the data sequence;

Judging whether the data sequence is in the storage mode, if not, entering the waiting mode, if so, storing the data sequence to the storage circuit in the microscope high-speed synchronous control device, and entering the waiting mode;

The microscope high-speed synchronization control device receives the trigger signal, the trigger carries at least a data sequence, and the microscope high-speed synchronization control device checks the data sequence;

Call the data sequence, read and cache the data sequence in the microscope high-speed synchronous control device;

Call the cached data sequence, output digital signals based on the data sequence, and realize precise timing control of the LED light source and camera based on the digital signal, so that the camera can shoot.

The technical solution adopted in the embodiment of the present application also includes: before the microscope high-speed synchronization control device receives the trigger signal, it also includes:

Set the camera's exposure time and other parameters.

The technical solution adopted in the embodiment of the present application also includes: after realizing the precise timing control of the LED light source and the camera based on the digital signal, after the camera takes pictures, it also includes:

The microscope high-speed synchronous control device judges whether the shooting is completed;

If so, end the shooting, otherwise control the camera to move on the X-Y two-dimensional plane and shoot until the shooting on the X-Y two-dimensional plane is completed.

The technical solution adopted in the embodiment of the present application also includes: after the shooting of the X-Y two-dimensional plane is completed, it also includes:

The high-speed synchronous control device of the microscope receives and controls the camera to move up or down to realize shooting at another viewing distance until the shooting is completed.

The technical solution adopted in the embodiment of the present application also includes: before the microscope high-speed synchronization control device judges whether the shooting is completed, it also includes:

Preset the number of shots taken on each X-Y two-dimensional plane of the camera;

Preset the number of times the camera moves up or down, and the distance of each movement.

Compared with the prior art, the beneficial effect produced by the embodiment of the present application lies in: the microscope synchronization control system and method in the embodiment of the present application, the system includes: a microscope high-speed synchronization control device, a camera, a laser, an LED light source and a spatial light modulator; The host computer is connected to the microscope high-speed synchronization control device through the serial port, and is used to send specific setting sequences and data sequences to set the microscope high-speed synchronization control device or send trigger sequences to control the microscope high-speed synchronization control device to generate digital signals. The high-speed synchronous control device of the microscope controls the synchronous operation of the camera, laser, LED light source and spatial light modulator according to the digital signal to realize precise shooting of the camera in the complex microscope system.

Description of drawings

The drawings described here are used to provide a further understanding of the application and constitute a part of the application. The schematic embodiments and descriptions of the application are used to explain the application and do not constitute an improper limitation to the application. In the attached picture:

Fig. 1 is the schematic diagram of the microscope synchronous control system of the present application;

Fig. 2 is the signal output diagram of the microscope synchronous control system of the present application;

Fig. 3 is the digital level change in a fast X-Y scanning of the signal output of the microscope synchronous control system of the present application;

The marked part of Figure 4 is an excerpt from the first 220 milliseconds of Figure 3 of this application;

Figure 5 is a schematic diagram of external interrupt microscope control of the high-speed synchronous control system of the microscope of the present application;

Fig. 6 is the flow chart of the microscope synchronous control system method of the present application;

Fig. 7 is the input process during the use of the high-speed synchronous control system of the microscope of the present application;

Figure 8 is the output flow during the use of the high-speed synchronous control system of the microscope of the present application;

Reference signs: 1-host computer, 2-microscope high-speed synchronous control device, 3-camera, 4 laser, 5-LED light source, 6-spatial light modulator, 7-electric translation stage, 8-voltage ceramic Z stage, 9 - Fluorescence filter cubes.

Detailed ways

In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is an embodiment of a part of the application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

It should be noted that the terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

Example 1

According to an embodiment of the present application, a liver segmentation method based on deep learning for abdominal body images is provided, as shown in FIG. light modulator 6;

The upper computer 1 is connected to the microscope high-speed synchronization control device through a serial port, and is used to send a specific setting sequence and data sequence to set the microscope high-speed synchronization control device 2 or send a trigger sequence to control the microscope high-speed synchronization control device 2 to generate digital signals;

The microscope high-speed synchronous control device 2 controls the synchronous operation of the camera 3, laser 4, LED light source 5 and spatial light modulator 6 according to the digital signal. The digital signal includes the exposure trigger signal of the camera 3, the trigger signal of the LED light source 5, the trigger signal of the laser 4 and the space Optical modulator 6 trigger signal;

The microscope high-speed synchronization control device 2 sends the camera 3 shooting trigger signal to the camera 3, and controls the camera 3 to shoot; the microscope high-speed synchronization control device 2 sends a digital signal to the LED power supply, and controls the LED power supply to turn on;

The microscope high-speed synchronization control device 2 sends the trigger signal of the laser 4 to the laser 4, and the microscope high-speed synchronization control device 2 sends the modulator trigger signal to the spatial light modulator 6. When the camera 3 is shooting, the spatial light modulator 6 is triggered, and the spatial light modulation The high-speed synchronization control device 2 of the microscope sends a TTL signal to the microscope high-speed synchronization control device 2, and the microscope high-speed synchronization control device 2 sends the trigger signal of the laser 4 and the trigger signal of the spatial light modulator 6 to do "AND" to the laser 4 and the spatial modulator to synchronize the space The input signals of the light modulator 6 and the laser 4 realize precise timing control of the LED light source 5 and the camera 3 so that the camera 3 can take pictures.

The microscope synchronization control system and method in the embodiment of the present application, the system includes: a microscope high-speed synchronization control device 2, a camera 3, a laser 4, an LED light source 5 and a spatial light modulator 6; the upper computer 1 communicates with the microscope high-speed synchronization control device through a serial port The connection is used to send a specific setting sequence and data sequence to set the microscope high-speed synchronization control device 2 or send a trigger sequence to control the microscope high-speed synchronization control device 2 to generate digital signals. The high-speed synchronous control device 2 of the microscope controls the synchronous operation of the camera 3, the laser 4, the LED light source 5 and the spatial light modulator 6 according to the digital signal to realize the precise shooting of the camera 3 in the complex microscope system.

Taking the timing of camera 3 as the main signal to trigger synchronous control shooting of other equipment is the main feature of existing microscopes in realizing high-speed synchronous shooting technology. The support and scalability of the system are insufficient. To address this shortcoming, this application designs and builds a high-speed synchronous control device with multi-channel digital output and analog output signals based on the high-precision timing control method triggered externally by the equipment, to realize complex processes in more complex microscope systems Accurate shooting; In addition, by setting the external trigger of the synchronization control device itself, the microscope can complete the closed-loop self-driven high-speed shooting process.

According to an aspect of the embodiment of the present application, a microscope high-speed synchronous control system is provided, including a microscope high-speed synchronous control device 2, a host computer 1, and microscope-related equipment. The microscope-related equipment includes a camera 3, a piezoelectric ceramic Z stage 8, an electric XY translation stage (electric translation stage 7), laser 4, LED light source 5, spatial light modulator 6, etc. Among them, the electric XY translation stage means that the electric translation stage 7 moves on the X-Y two-dimensional plane, and the piezoelectric ceramic Z stage 8 means that it moves up and down in the Z-axis direction.

The microscope high-speed synchronous control device 2 includes an integrated circuit board and an external power adapter. Among them, the integrated circuit board contains the following parts: main control chip, peripheral circuit, upper computer 1 communication transcoding circuit, electronic erasable programmable read-only storage circuit, power supply isolation and voltage stabilization circuit, digital-to-analog signal conversion circuit, operation Amplifier-related filtering and amplifying circuits.

The upper computer 1 is used for operator interaction, and sends specific setting sequences and data sequences through the serial port to set the microscope high-speed synchronization control device 2 or send a trigger sequence to control the microscope high-speed synchronization control device 2 to generate signals, which can be, but not limited to, notebook computers, A terminal that can communicate, such as a desktop computer or a single-chip microcomputer.

Microscope-related equipment includes a camera 3, a piezoelectric ceramic Z stage 8, a motorized translation stage 7, a laser 4, an LED light source 5, a spatial light modulator 6, and the like. The exposure mode of the camera 3 is generally set by each manufacturer, and the external trigger is the basic mode, that is, the exposure of the camera 3 is triggered by the pulse level; the piezoelectric ceramic Z stage 8 can be controlled by an analog voltage, and the range of the voltage is determined according to the stroke and precision and step length; the electric stage 7 has two kinds of signals, one as the input signal of the stage, the pulse level in the signal will trigger the movement of the stage, and one as the output signal of the stage, when the stage is moving Send the pulse level from the signal after it is in place; the switch of the laser 4 and the LED light source 5 is triggered by the pulse high level; at the same time, because the input signal of the spatial light modulator 6 has high requirements on timing, it is generally controlled by a high-precision The TTL level is controlled as an input signal, and the user can choose input and output with different modes. The microscope high-speed synchronization control system of the present application can be used as a signal control center to synchronize the input signals of the spatial light modulator 6 and the laser 4, so as to realize the light source and the laser. Camera 3 achieves high-precision timing control.

Referring to Fig. 1 and Fig. 2, the communication between the host computer 1 and the microscope high-speed synchronization control device 2 in this application, and the signal transmission between the microscope high-speed synchronization control device 2 and the related equipment of the microscope. Specifically:

The communication between the host computer 1 and the microscope high-speed synchronization control device 2 includes the input sequence sent by the host computer 1 to the microscope high-speed synchronization control device 2 and the palindrome sequence fed back by the microscope high-speed synchronization control device 2 . The input sequence includes the setting sequence, which is used to set the working mode of the microscope high-speed synchronization control device 2, such as switching the external interrupt function and storage function, etc.; and the data sequence and signal trigger sequence of different modes, used to describe the microscope high-speed synchronization control device 2 The signal sequence to send. The palindrome sequence includes the data or systematic error report of the microscope high-speed synchronization control device 2 and the palindrome of completing the current task.

The signal transmission between the microscope high-speed synchronization control device 2 and microscope-related equipment can be summarized as no less than sixteen channels of digital signal output and no less than two channels of analog output provided by the microscope high-speed synchronization control device 2 to other microscope-related equipment and not less than two interrupt inputs; digital signal 1 to digital signal 16 in FIG. 2 correspond to the output of the first digital signal to the sixteenth digital signal. The digital signal output is used to control the camera 3, the motorized stage 7, the laser 4, the LED light source 5, the spatial light modulator 6 and other TTL triggering devices. The analog output is used in this example to control the piezoelectric ceramic Z stage 8, with an effective range of 0-10V and a step size of 1mV. The interrupt input is used for external TTL pulse signal triggering or affecting the signal transmission of the device, so as to realize precise synchronous control with microscope-related equipment. The external interrupt can also support other complex signal input modes, such as the output signal of the 68-bit image of the spatial light modulator, and the specified digital signal is calculated and output, and the spatial light modulator 6 can display 8 bits Precise synchronization of image time and multi-channel light source signals.

The microscope high-speed synchronous control device 2 includes an integrated circuit board and an external power adapter. The integrated circuit board is provided with a main control chip. The main control chip is connected with two digital-analog chips, and each of the two digital-analog chips is connected with an operational amplifier filter. The power adapter supplies power to the system; the power adapter provides the voltage converted by the transformer chip for the main control chip, the power adapter provides the digital-analog chip voltage for the two digital-analog chips, and the filter voltage for the two op-amp filters; The filters output analog signals respectively, and the main control chip provides sixteen digital signal outputs.

Specifically, the integrated circuit board in the microscope high-speed synchronous control device 2 mentioned in this application includes a main control chip, peripheral circuits, a communication transcoding circuit of the host computer 1, an electronic erasable programmable read-only memory circuit, a power supply isolation And voltage stabilizing circuit, digital-to-analog signal conversion circuit, filter and amplification circuit related to operational amplifier. The upper computer 1 communication transcoding circuit is used to realize the two-way communication between the upper computer 1 and the microscope high-speed synchronous control device 2; the electronic erasable programmable read-only storage circuit has the function of simultaneously supporting multiple copying and power-off storage, and can Realize the pre-stored data sequence in the storage module of the control box, and provide the function of power-off storage; the power supply isolation and voltage stabilization circuit is used to isolate the input voltage and the analog voltage, and to filter the voltage; the digital-to-analog signal conversion circuit is used for The analog signal with 14-bit resolution is generated to realize high synchronization during the microscope shooting process; the filtering and amplifying circuit is used to amplify the voltage and filter the amplified voltage.

In this example, the main control chip and peripheral circuits are STM32H7 series chips whose main frequency can reach 480MHz. The peripheral circuits include crystal oscillator circuit, programming circuit and manual reset circuit. Its high main frequency feature can satisfy high-precision timing control, realize high synchronization during the microscope shooting process, and maximize the performance supported by the equipment.

The communication method of host computer 1 is selected as the communication method of USB to serial port in this example. The communication line is simple, and only two transmission lines need to be sent and received to realize two-way communication. In this example, the transcoding circuit uses the commonly used serial port conversion chip CH340, which can meet the high baud rate serial port communication.

The electronic erasable programmable read-only storage circuit in this example uses the AT24 series chip, which has a storage space of 524,288 bits, and supports functions such as multiple rewriting and power-off storage. It can realize the pre-stored data sequence to the storage module of the control box, and provide the function of power-off storage.

Power supply isolation and voltage stabilization circuit In this example, TPH1515S-3W chip is used to isolate the input voltage and analog voltage, and LM1117S, TPS5430 and TPS7A330 are used to obtain the digital 5V, digital 3.3V, analog 5V, and analog 12V required in the circuit And analog -12V voltage, and use the corresponding method to filter the voltage.

In this example, the digital-to-analog signal conversion circuit uses the DAC904 high-speed digital-to-analog conversion chip to generate 14-bit resolution analog signals. Its conversion rate up to 165MSPS can meet high-precision timing control and achieve high synchronization during the microscope shooting process. Play to the maximum performance supported by the device. And choose ADA4898 to convert the differential signal output by DAC904 into single-ended signal.

In this example, AD620 instrumentation amplifier chip is used as the amplification module for the filter and amplification circuit related to the operational amplifier, AD810 is used as the active low-pass filter module, and LM358 is used as the range adjustment module of the analog output. The analog output range of the microscope high-speed synchronous control device 2 is -11V to +11V, and the output range is adjusted to 0-10V in this example.

The input voltage is provided by a separate 15V regulated adapter, and the input voltage will be converted after passing through the DCDC transformer chip: power supply for the chip and USB serial port power supply voltage (digital 5V), analog signal digital-analog chip voltage after isolation and transfer (analog 5V) and filter voltage (analog 10V). The main control chip communicates with terminal 1 of the user's host computer through the USB serial port, and the user command is sent to the main control chip in the form of hexadecimal code and reads the returned information from the main control chip. The system can be set to the external interrupt mode, and the external TTL signal will trigger the pre-stored signal output of the chip after entering. In this example, the main control chip directly provides 16 channels of digital signal output, and the analog signal output is output by the main control chip after digital-to-analog conversion and filtering.

In this example, by connecting the signal to the external trigger pin of each device, the correspondence between each digital signal channel of the microscope high-speed synchronization control device 2 and the device function of the trigger pass is shown in Table 1, and Table 1 is the digital signal and trigger Correspondence between device functions.

signal number	name	The trigger method of the corresponding device
digital signal 1	Camera (camera 3 exposure trigger signal)	Rising edge trigger, start shooting
digital signal 2	SLM SPI0 (spatial light modulator 6 trigger signal)	Pulse trigger, SLM displays current RO
digital signal 3	SLM SPI1 (spatial light modulator 6 trigger signal)	Pulse trigger, image switching in RO
digital signal 4	Laser 405 (405nm laser trigger signal)	Active high, 405nm laser on
digital signal 5	Laser 445 (445nm laser trigger signal)	Active high, 445nm laser on
digital signal 6	Laser 488 (488nm laser trigger signal)	Active high level, 488nm laser on
digital signal 7	Laser 515 (515nm laser trigger signal)	Active high, 515nm laser on
digital signal 8	Laser 561 (561nm laser trigger signal)	Active high, 561nm laser on
digital signal 9	Laser 640 (640nm laser trigger signal)	Active high level, 640nm laser on
digital signal 10	LED_A (LED trigger signal)	Active high, A LED on
Digital Signal 11	LED_B (LED trigger signal)	Active high, B LED on
digital signal 12	LED_C (LED trigger signal)	Active high, C LED on
digital signal 13	LED_D (LED trigger signal)	Active high, D LED on
digital signal 14	ASI_IN (trigger signal of electric stage 7)	Pulse Triggered, ASI Mobile
digital signal 15	LC (trigger signal for ferroelectric liquid crystal polarization rotator)	Pulse trigger, D5020 switching voltage
digital signal 16	motor (motor trigger signal)	Active high, motor vibration

In the embodiment, the system includes a motorized displacement platform 7, and the digital signal includes a trigger signal of the motorized displacement platform 7. The microscope high-speed synchronization control device 2 sends the trigger signal of the motorized displacement platform 7 to the electric XY displacement platform, and the motorized displacement platform 7 is based on the received motorized displacement. The trigger signal of stage 7 moves in the X-Y two-dimensional plane;

After the camera 3 shoots, the microscope high-speed synchronization control device 2 will send a trigger signal to the electric displacement platform 7 that has completed the shooting, so that the electric displacement platform 7 moves to the next preset position;

After the electric stage 7 moves to the preset position, an external trigger signal will be sent to the microscope high-speed synchronous control device 2 to make it start the next shooting cycle.

In the embodiment, the system also includes a piezoelectric ceramic Z stage 8, the digital signal includes a Z stage analog signal, and the microscope high-speed synchronization control device 2 sends the Z stage analog signal to the piezoelectric ceramic Z stage 8, and the piezoelectric ceramic Z stage 8 receives the The Z analog signal moves up and down;

After the shooting cycle of the camera 3 ends, the piezoelectric ceramic Z stage 8 will move up and down.

The motions of the electric displacement stage 7 and the piezoelectric ceramic Z stage 8 of the present application are described in detail below with specific embodiments:

Referring to Figure 1 to Figure 4, the minimum time unit provided in this example is 1 microsecond, and the minimum adjustable range is 0.25 microsecond. This application can be used to encode any number of data sequences of any time length and any level on all pins. Using the basic method needs to clarify the required signal time sequence, that is, the user needs to clearly specify the change time before each digital or analog signal changes. Level state, the time point when the change occurs and the level state after the change. Figure 3 and Figure 4 show the sequence of several digital signals in a fast XY scan shooting process, the fast XY scan includes the synchronization of camera 3 exposure trigger signal and LED light source 5 and laser 4 trigger signal, and setting the micromirror high-speed synchronous control The signal of the device and the feedback signal of the spatial light modulator 6 are "ANDed" to output. 3 and 4, the Y axis represents the level change of the digital signal, 1 represents the high level of the digital signal, and 0 represents the low level of the digital signal. The X-axis represents the running time of the system, and the unit is 0.1 milliseconds. The part marked in FIG. 4 is an excerpt of the first 220 milliseconds in FIG. 3 .

The fast XY scan shooting process in this example follows the following operations: First, the exposure of camera 3 is triggered by the rising edge of the trigger signal of camera 3 (0.5 milliseconds in the time axis), and the camera 3 starts from receiving the rising edge signal to the hardware start It takes about 24 milliseconds to finish and start shooting. After shooting starts (23.5 milliseconds on the time axis), change the digital signal corresponding to the LED light source 5 to a high level, wherein the trigger mode of the LED light source 5 is completely synchronized with the digital signal, that is, the LED light source 5 is turned on when the digital signal is high level, When the digital signal is at low level, the LED light source 5 is turned off. In this example, the LED light source 5 needs to be turned on for 50 milliseconds, that is, at 73.5 milliseconds on the time axis, the digital signal becomes low level, the LED light source 5 stops generating signals, and the single shooting ends. In this example, the interval between multiple shots is selected as 20 milliseconds, that is, the next shot starts at 93.5 milliseconds. The operation is the same as the previous shooting, first use the rising edge of the trigger signal of camera 3 to trigger the exposure of camera 3 (95 milliseconds on the time axis), and trigger the spatial light modulator 6 (in this example) after shooting starts (119 milliseconds on the time axis) In is the second digital signal) and the digital signal corresponding to the laser 4 (in this example, the fourth, fifth, sixth and seventh digital signals). At the same time, when the spatial light modulator 6 is triggered, the output of the spatial light modulator 6 is a specially modulated TTL signal fed back to the microscope high-speed synchronization control device 2 as an external interrupt signal, and the microscope high-speed synchronization control device 2 sends the digital signal to the laser 4. The signal is an output after 'ANDing' the set laser 4 signal and the spatial light modulator 6 signal as an external interrupt. The subsequent shots related to the laser 4 all follow the same operation, and the purpose of this operation is to completely synchronize the turn-on and turn-off timing of the laser with the flipping timing of the liquid crystal of the spatial light modulator 6 . After a shooting cycle including the LED light source 5 and 4 lasers 4 signals ends, the microscope high-speed synchronization control device 2 will send a trigger signal for the electric displacement stage 7 that has completed the shooting, so that the electric displacement stage 7 moves to the next field of view, and the electric displacement After the stage 7 moves, an external trigger signal will be sent to the microscope high-speed synchronous control device 2 to start the next shooting cycle.

In addition, in the Z-scanning mode, after each shooting cycle, the piezoelectric Z stage will move in the Z-axis direction once. In this example, a total of 17 photos were taken, starting from -1.6nm at the relative center position of the piezoelectric Z stage (the total stroke is 150nm, and 75nm is selected as the center position in this process), and the Z axis is stepped after each shooting cycle 0.2nm. Table 2 shows the changes of digital signals and analog signals and the data sent in one XY scanning mode.

Table 2

According to the above operation, the level of the digital signal and the analog signal will change at the following time points: 0 millisecond, 0.5 millisecond, 10.5 millisecond, 23.5 millisecond, 74.5 millisecond and so on. At each time point when a level change occurs, a set of data is required to describe the changed digital signal, analog signal and the duration of this change. As shown in the above table at 0ms, the level state is described as follows: the digital signals are all low level, the analog signal 1 voltage is 4.8933V corresponding to the piezoelectric Z stage height of 73.4nm, and the analog signal 2 is a constant 5V. At 0.5ms, the trigger signal of camera 3 (the first digital signal) becomes high level, and its digital signal is converted to 0x0001 in hexadecimal, the voltage of analog signal 1 remains at 4.8933V, and the voltage of analog signal 2 remains at a constant 5V . After completing one shooting cycle, that is, entering the second shooting cycle at 93.5ms, the digital signal level at this time is consistent with 0.5V, but the analog signal level has stepped 0.2nm, and its level is 4.9067V. Analog signal two remains constant at 5V.

The above basic control method can describe all output sequences, but for some situations with long duration and relatively repetitive sequence content, some general complex methods can reduce code redundancy, improve the efficiency of single-chip processing, and reduce the time for single-chip processing data sequences. For example, for a completely repeated sequence, a description of the number of cycles can be added to the sequence, and a specific output signal can be triggered at a specific frequency. In addition, specific data sequences can be customized to improve the efficiency of sending sequences as much as possible according to the needs of users. Microscope high-speed synchronous control device 2 currently contains the classification of data sequences in Table 3:

table 3

The control method of the microscope high-speed synchronous control system is shown in Figure 5. In this example, during the shooting process of the microscope imaging camera 3, firstly, the exposure time of the camera 3, the intensity of the light source and other parameters are set on the PC side, and the camera 3 and the light source are set to be in a waiting state according to the trigger mode of the related equipment. In the example of this application, in the XY scanning mode, the microscope uses the MS2000 electric translation stage 7 of ASI Company as the XY electric translation stage 7, and loads the fast XY firmware with ARRAY scanning. The laser 4 uses Coherent’s OBIS series laser, and the LED uses Thorlabs The LED or CoolLED company's PE4000 series, the spatial light modulator 6 uses ForthDD's QXGA-R10 series chip. Use the TTL level of the microscope high-speed synchronous control device 2 as a pulse signal to trigger the movement of the XY electric translation stage 7, and the electric translation stage 7 will send a TTL pulse signal as an external interrupt of the control system after the electric translation stage 7 moves in place, and the trigger control system has been written into the stored Time-sequential transmission realizes the exposure of the camera 3, light source and other equipment, and completes the shooting of the microscope image. Use the signal that the electric stage 7 is in place as the trigger for the next signal, and use the pre-stored sequence to improve the operating efficiency of the equipment. In the Z scan mode, the piezoelectric ceramic Z stage 8 is controlled by an analog voltage of 0-10V, and a shooting sequence is sent in each analog signal step to complete the microscope image shooting of each channel. In summary, multi-dimensional microscope shooting of time, XY and Z scans and channels is realized.

Referring to Fig. 7 and Fig. 8, it describes the two main routine processes of the interaction of the microscope high-speed synchronous control device 2 in the system. Fig. 7 describes the input process during the use of the microscope high-speed synchronous control system, which mainly includes the host computer 1 and the microscope Interaction between devices 2 is controlled synchronously at high speed. The user edits data instructions on the host computer 1 and can use specific software to generate a visualized sequence for inspection. After sending data instructions to the microscope high-speed synchronous control device 2, the device itself will check the instructions and provide feedback. In the case of some high-speed continuous output, the inspection and feedback stages can be shielded to achieve the purpose of increasing the operating speed. If the check sequence is correct, the data command will be stored in the cache, and the main control chip will judge whether to store it in the electronic erasable programmable read-only memory circuit, and enter the waiting mode.

Figure 8 describes the routine output flow during the use of the microscope high-speed synchronization control system, which mainly includes the interaction between the host computer 1, the microscope high-speed synchronization control device 2 and the microscope peripherals. Usually, the user needs to set the peripherals of the microscope first, as shown in Figure 7 and Figure 8, the user needs to set the exposure time and other parameters of the camera 3 on the host computer 1. After setting, the trigger stage can be started. There are currently two ways to trigger the output signal. The first is to use the host computer 1 to directly send the trigger command. The second is to send the external interrupt signal after the peripheral device is ready. The second way Relatively speaking, it can provide a faster trigger speed. After receiving the instruction, the main control chip judges whether to read the data in the electronic erasable programmable read-only memory circuit. As in the operation described in Figure 8, after a shooting cycle ends, the microscope high-speed synchronous control device 2 will judge whether all operations are completed, and if it is completed, then send a trigger signal for the electric translation platform 7 that has completed the shooting, so that the electric translation platform 7 can move To the next field of view, after the motorized stage 7 moves, it will send an external trigger signal to the microscope high-speed synchronous control device 2 to make it start the next shooting cycle. If all operations are complete, the loop will end and the next instruction will be awaited.

As shown in Figure 3, it is the signal output of multiple signals in a fast XY scan, and the data command in the basic mode is:

80010027108000003200820000C99080000052D0800100271080000032008008000320800A002A30800200012C8000002C248008000320800A8002A308000002 A002A3080000029048008000320800A002A3080000029048008000320800A002A308000004E848001002710800000320080200003208022002A30800200020C00120 3208022002A30800000290480200003208022002A30800000290480200003208022002A30800000290480200003208022002A308000004E8480010002003080 82002A30800200012C8000002C2480800003208082002A30800000290480800003208082002A30800000290480800003208082002A3080000029048080200803208080200803 E848001002710800000320081000003208102002A30800200012C8000002C2481000003208102002A30800000290481000003208102002A3081030000029003020202 0000290481000003208102002A308000000190A00000025880000001F420002000200020002000200020002000200020002000200020002000200020002002020 02000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020 2000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020 20 002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000.

Among them, every ten digits of data is the data required when a group of level changes, the first four digits are the hexadecimal numbers represented by the digital signal after this level change, and the last six digits represent the distance from the next level change time of occurrence.

Example 2

According to an embodiment of the present application, a microscope synchronization control method is provided, as shown in FIG. 6 , including the following steps:

S101: The microscope high-speed synchronization control device receives the data sequence, and the microscope high-speed synchronization control device checks whether the data sequence meets the specification;

S102: If the specification is met, store the data sequence;

S103: Determine whether the data sequence is in the storage mode, if not, enter the waiting mode, if so, store the data sequence to the storage circuit in the microscope high-speed synchronization control device, and enter the waiting mode;

S104: The microscope high-speed synchronization control device receives the trigger signal, the trigger carries at least a data sequence, and the microscope high-speed synchronization control device checks the data sequence;

S105: call the data sequence, read and cache the data sequence in the high-speed synchronization control device of the microscope;

S106: call the cached data sequence, output digital signals based on the data sequence, realize precise timing control of the LED light source and the camera based on the digital signal, and enable the camera to shoot.

This application uses the microscope high-speed synchronization control device to output digital signals based on the data sequence to control the precise shooting of the camera in the complex microscope system.

The microscope synchronous control method of the present application will be described in detail below with specific examples:

Step 1: The host computer 1 interacts with the microscope high-speed synchronous control device 2. The user edits the data command carrying the data sequence at the host computer 1 end and sends it to the microscope high-speed synchronous control device 2; the user edits the data command at the host computer 1 end and Specific software can be used to generate visualized sequences for inspection; in addition, through the microscope high-speed synchronous control device 2 itself, the sequence can also be inspected to see if the data sequence conforms to the specification and can be correctly identified, and feedback is provided. In some cases of high-speed continuous output The inspection and feedback stages can be blocked to achieve the purpose of improving the running speed.

Step 2: If the verification data sequence conforms to the specification or the verification sequence is correct, the data sequence will be stored in the cache.

Step 3: Determine whether the data sequence is to be stored in the electronic erasable programmable read-only storage circuit by the main control chip, if not, enter the waiting mode, otherwise store the data sequence to the storage circuit in the microscope high-speed synchronization control device 2, and enter standby mode.

The above steps 1 to 3 are the input process during the use of the high-speed synchronous control system of the microscope, as shown in FIG. 7 .

Step 4: The microscope high-speed synchronization control device 2 receives the trigger signal, the trigger carries at least a data sequence, and the microscope high-speed synchronization control device 2 checks the data sequence;

Before the microscope high-speed synchronous control device 2 receives the trigger signal, it also includes:

Set parameters such as the exposure time of camera 3.

Specifically, the microscope high-speed synchronization controls the interaction between the device 2 and the microscope peripherals. Usually, the user needs to set the peripherals of the microscope first, then set the exposure time and other parameters of the camera 3 on the host computer 1, and then start the trigger stage after setting.

At present, there are two ways to trigger the output signal. The first way is to use the host computer 1 to directly send the trigger command, and the second way is to send the external interrupt signal after the peripheral device is ready. The second way can provide relatively faster trigger speed.

Step 5: After receiving the trigger signal, the main control chip of the microscope high-speed synchronization control device 2 judges whether to read the data in the electronic erasable programmable read-only memory circuit. If the data sequence is called, the data sequence in the microscope high-speed synchronous control device 2 is read and cached.

Step 6: call the cached data sequence, based on the data sequence, output the digital signal to the camera 3, the piezoelectric ceramic Z stage 8, the electric XY translation stage, the LED light source 5 and the spatial light modulator 6, so that the LED light source 5 and the camera 3 Precise timing control enables the camera 3 to shoot.

The above steps 4 to 6 are the output process during the use of the high-speed synchronous control system of the microscope, as shown in FIG. 8 .

In the embodiment, after realizing the precise timing control of the LED light source 5 and the camera 3 based on the digital signal, and making the camera 3 take pictures, it also includes:

The microscope high-speed synchronous control device 2 judges whether the shooting is completed;

If then end shooting, otherwise control camera 3 to move on the X-Y two-dimensional plane and shoot until finishing the shooting on the X-Y two-dimensional plane.

Specifically, after a shooting cycle ends, the microscope high-speed synchronous control device 2 will judge whether the shooting is completed, if not, the microscope high-speed synchronous control device 2 sends a trigger signal for the electric translation platform 7 that has completed the shooting, and the electric XY translation platform closes After receiving the trigger signal, the motorized stage 7 moves to the next field of view. After the motorized XY stage moves, an external trigger signal will be sent to the microscope high-speed synchronization control device 2 to start the next shooting cycle. If all operations are completed, the loop will be terminated and subsequent instructions will be awaited.

In the embodiment, after the shooting of the X-Y two-dimensional plane is completed, it also includes:

The microscope high-speed synchronous control device 2 receives and controls the camera 3 to move up or down to realize shooting at another viewing distance until the shooting is completed.

Before the microscope high-speed synchronous control device 2 judges whether the shooting is completed, it also includes:

Preset the number of shots taken on each X-Y two-dimensional plane of the camera 3;

The number of times the camera 3 moves up or down, and the distance of each movement are preset.

In the microscope high-speed synchronous control system of this application, the TTL level of the microscope high-speed synchronous control device 2 is used as a pulse signal to trigger the movement of the XY electric translation platform 7, and the electric translation platform 7 sends a TTL pulse signal as an external interruption of the control system after moving in place , the trigger control system has been written into the stored time sequence to realize the exposure of the camera 3, the light source and other equipment, and complete the shooting of the microscope image. Use the signal that the electric stage 7 is in place as the trigger for the next signal, and use the pre-stored sequence to improve the operating efficiency of the equipment.

Compared with the prior art, the present application can solve the problem of high synchronization in the shooting timing control of the current microscope, and realize high-speed synchronous shooting. The overall structure is simple, the cost is low, the actual operation is convenient, and the compatibility is wide, which is of great significance for improving the efficiency of microscope shooting and reducing photobleaching in fluorescence imaging. As shown in Figure 1, the objective lens 10 observes through the fluorescence filter block 9, and the camera 3 takes pictures through the fluorescence filter block 9.

This application has been verified by the actual machine test of fluorescent microscope image shooting, and can fully realize the shooting of multi-field and multi-channel fluorescence. After editing the timing of the microscope shooting protocol, control the exposure of the camera 3, turn on the light source, and the synchronization of the spatial light modulator 6. The operation is simple, the applicability is wide, and the overall cost is low, so the application has great development prospects and commercial value.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A microscope synchronization control system, characterized in that it includes: a host computer, a microscope high-speed synchronization control device, a camera, a laser, an LED light source, and a spatial light modulator;

   The host computer is connected to the microscope high-speed synchronization control device through a serial port, and is used to send a specific setting sequence and data sequence to set the microscope high-speed synchronization control device or send a trigger sequence to control the microscope high-speed synchronization control device to generate digital Signal;

   The microscope high-speed synchronization control device controls the camera, laser, LED light source and spatial light modulator and analog signals to control the synchronous operation of the piezoelectric ceramic Z stage according to the digital signal, the digital signal includes a camera exposure trigger signal , LED light source trigger signal, laser trigger signal and spatial light modulator trigger signal, the analog signal controls the horizontal height of piezoelectric ceramic Z stage;

   The microscope high-speed synchronization control device sends the camera shooting trigger signal to the camera to control the camera to shoot; the microscope high-speed synchronization control device sends a digital signal to the LED power supply to control the LED power supply to turn on;

   The microscope high-speed synchronization control device sends a laser trigger signal to the laser, the microscope high-speed synchronization control device sends a modulator trigger signal to the spatial light modulator, and triggers the spatial light modulation when the camera is shooting The spatial light modulator outputs a TTL signal to the microscope high-speed synchronization control device, and the microscope high-speed synchronization control device sends the signal after the "AND" of the laser trigger signal and the spatial light modulator trigger signal to the The laser and the spatial modulator are used to synchronize the input signals of the spatial light modulator and the laser to realize precise timing control of the LED light source and the camera, so that the camera can take pictures.
2. The microscope synchronous control system according to claim 1, wherein the system also includes a motorized stage, the digital signal also includes a trigger signal of the motorized stage, and the microscope high-speed synchronization control device sends the motorized stage A trigger signal is sent to the electric XY stage, and the electric stage moves in the X-Y two-dimensional plane according to the received trigger signal of the electric stage;

   After the camera shoots, the high-speed synchronous control device of the microscope will send a trigger signal to the electric displacement stage to move the electric displacement stage to the next preset position;

   After the electric stage moves to the preset position, an external trigger signal will be sent to the high-speed synchronous control device of the microscope to start the next shooting cycle.
3. The microscope synchronization control system according to claim 2, wherein the system also includes a piezoelectric ceramic Z stage, the digital signal also includes an analog signal of the Z stage, and the microscope high-speed synchronization control device sends the Z stage An analog signal is sent to the piezoelectric ceramic Z stage, and the piezoelectric ceramic Z stage moves up and down according to the received analog signal of the Z stage;

   After the camera shooting cycle ends, the piezoelectric ceramic Z stage will move up and down.
4. The microscope synchronization control system according to claim 2, wherein the microscope high-speed synchronization control device includes an integrated circuit board and an external power adapter, the integrated circuit board is provided with a main control chip, and the main control chip Two digital-analog chips are connected, each of the two digital-analog chips is connected with an operational amplifier filter, and the power adapter supplies power to the system; Voltage, the power adapter provides the digital-analog chip voltage for the two digital-analog chips, and provides the filter voltage for the two op-amp filters;

   The two op-amp filters output analog signals respectively, and the main control chip provides sixteen digital signal outputs.
5. The microscope synchronous control system according to claim 4, wherein the integrated circuit board is also provided with a host computer communication transcoding circuit, an electronic erasable programmable read-only storage circuit, a power supply isolation and voltage stabilization circuit , digital-to-analog signal conversion circuit, filter and amplifier circuit;

   The upper computer communication transcoding circuit is used to realize the two-way communication between the upper computer and the microscope high-speed synchronization control device;

   The electronic erasable programmable read-only storage circuit has the function of supporting multiple rewriting and power-off storage at the same time, and can realize the pre-stored data sequence to the storage module of the control box, and provide the function of power-off storage;

   Power supply isolation and voltage stabilization circuit, used to isolate input voltage and analog voltage, and filter the voltage;

   The digital-to-analog signal conversion circuit is used to generate multiple independent analog signals with 14-bit resolution, so as to realize high synchronization during the microscope shooting process;

   The filtering and amplifying circuit is used for amplifying the voltage and performing filtering processing on the amplified voltage.
6. A microscope synchronous control method, characterized in that, comprising the following steps:

   The microscope high-speed synchronization control device receives the data sequence, and the microscope high-speed synchronization control device checks whether the data sequence meets the specification;

   If the specification is met, the data sequence is stored;

   Judging whether the data sequence is in the storage mode, if not, entering the waiting mode, if so, storing the data sequence to the storage circuit in the microscope high-speed synchronous control device, and entering the waiting mode;

   The microscope high-speed synchronization control device receives a trigger signal, the trigger carries at least the data sequence, and the microscope high-speed synchronization control device checks the data sequence;

   calling the data sequence, reading and caching the data sequence in the microscope high-speed synchronous control device;

   The buffered data sequence is called, and based on the data sequence, a digital signal is output, and based on the digital signal, the precise timing control of the LED light source and the camera is realized, so that the camera can shoot.
7. Microscope synchronous control method according to claim 6, is characterized in that, before described microscope high-speed synchronous control device receives trigger signal, also comprises:

   Parameters such as exposure time of the camera are set.
8. The microscope synchronous control method according to claim 6, characterized in that, after realizing the precise timing control of the LED light source and the camera based on the digital signal, and making the camera shoot, it also includes:

   The microscope high-speed synchronous control device judges whether the shooting is completed;

   If so, end the shooting, otherwise control the camera to move on the X-Y two-dimensional plane and shoot until the shooting on the X-Y two-dimensional plane is completed.
9. The microscope synchronous control method according to claim 8, further comprising:

   The high-speed synchronous control device of the microscope receives and controls the camera to move up or down to realize shooting at another viewing distance until the shooting is completed.
10. The microscope synchronization control method according to claim 9, characterized in that before the microscope high-speed synchronization control device judges whether the shooting is completed, it also includes:

    preset the number of times of shooting on each of the X-Y two-dimensional planes of the camera;

    The number of times the camera moves up or down, and the distance of each movement are preset.

Images