JP7476067B2 - Imaging device and imaging method - Google Patents

Description

This invention relates to a technology for capturing images by receiving fluorescence emitted from an object to which excitation light is incident, and in particular to a technology for capturing images while scanning the object.

For example, for the purpose of analyzing cultured cells or pathological tissues, image data obtained by imaging these is analyzed. When the area to be imaged is sufficiently large relative to the imaging field of view, the entire object is imaged while performing a so-called scanning movement, in which the imaging element such as a CCD camera and the object are moved relative to each other.

For example, the imaging device described in Patent Document 1 captures images of multiple wells in a well plate that holds samples while scanning the imaging unit, thereby acquiring images of each well in the well plate at once. By capturing images intermittently while scanning the imaging element and the object to be imaged in this way, it is possible to capture images of a wide range at high speed. More specifically, while performing scanning movement as described above, illumination light is intermittently turned on and imaging is performed in synchronization with this. For example, in the case of bright-field imaging, the illumination light irradiation time for one shot can be about several tens of microseconds.

On the other hand, in fluorescence imaging, in which an object to be imaged is irradiated with excitation light such as ultraviolet light and the fluorescence excited from the object is received, the irradiation time of the excitation light is longer, for example on the order of milliseconds, in order to provide a sufficient amount of exposure to excite the fluorescence while minimizing damage to the object to be imaged.

When the irradiation time or exposure time of the illumination light or excitation light is long, the amount of scanning movement of the object to be imaged during irradiation increases, causing blurring in the captured image. Even if the irradiation time or exposure time is short, a similar phenomenon occurs if the scanning movement speed increases. To address this problem, the technology described in Patent Document 2 reduces image blurring by performing image processing that takes into account the amount of scanning movement during exposure on images captured with multiple exposures.

JP 2020-052119 A US Patent Application Publication No. 2017/0330310

Image processing methods such as those in the above-mentioned conventional technology take a long time to process, and the effective imaging speed decreases, especially when the imaging range is wide. One possible method for solving image blur using hardware is to use a TDI (Time Delay Integration) sensor as the imaging element. However, TDI sensors are expensive, and there are no TDI sensors suitable for use as area sensors, so they are not very versatile. In addition, exposure and focus adjustment take time, so they are not practical for the purpose of high-speed imaging.

This invention was developed in consideration of the above problems, and aims to provide a technology for performing fluorescent imaging while scanning an object to be imaged, which has a relatively simple configuration and is capable of imaging a wide imaging range at high speed.

To achieve the above object, one aspect of the imaging device according to the present invention includes an imaging section that irradiates excitation light onto an object to be imaged, receives fluorescence emitted from the object, and images the object, and a scanning movement mechanism that scans and moves the imaging section at a constant speed relative to the object in a direction intersecting the direction of incidence of the excitation light. Here, the imaging section includes an excitation light source that emits the excitation light, a light receiving element that intermittently receives the fluorescence emitted from the object to be imaged, and a tracking movement mechanism that is capable of moving the light receiving element within a predetermined range in a tracking direction and in a reverse direction, the tracking direction being the direction in which an optical path of the fluorescence from a point on the object to the light receiving element moves with the scanning movement. The light receiving element is a two-dimensional image sensor, and the tracking movement mechanism moves the light receiving element from a predetermined initial position along the tracking direction at a speed corresponding to the scanning movement speed during a period in which the light receiving element receives the fluorescence, thereby causing the light receiving element to track the movement of an imaging position at which the image of the object to be imaged is formed on the imaging unit , and during a period in which the light receiving element does not receive the fluorescence, moves the light receiving element in the opposite direction to return it to its initial position.

Moreover, one aspect of the imaging method according to the present invention is an imaging method for imaging an object by irradiating excitation light onto the object and receiving fluorescence emitted from the object, and in order to achieve the above object, an imaging section having an excitation light source and a light receiving element which is a two-dimensional image sensor , and the object are scanned and moved at a constant speed relative to the object in a direction intersecting with a direction of incidence of the excitation light, and a direction in which an optical path of the fluorescence from a point on the object to be imaged to the light receiving element moves in association with the scanning movement is defined as a tracking direction, while the light receiving element is moved back and forth within a predetermined range in the tracking direction and in the opposite direction, and receives fluorescence from the object to be imaged during a period in which the light receiving element is moving in the tracking direction , and during the movement of the light receiving element in the tracking direction, the light receiving element is moved along the tracking direction at a speed corresponding to the scanning movement speed, thereby causing the light receiving element to track the movement of an imaging position where an image of the object to be imaged is formed on the imaging section .

In the invention configured in this way, the light receiving element moves in response to the scanning movement of the imaged object while imaging is being performed. When the imaged object and the light receiving element are scanned and moved relative to each other, if the exposure time is long, the incident position of the light emitted from the imaged object on the light receiving element moves, causing the image to blur. In the case of fluorescent imaging, a long exposure time is unavoidable due to the principle of fluorescence generation. Therefore, in this invention, the light receiving element is moved so as to follow the movement of the incident position of the fluorescent light emitted from a point on the imaged object. In this way, it is possible to eliminate the relative movement between the light receiving element and the imaged object during the light receiving period. This makes it possible to prevent the occurrence of image blur. Note that the relative movement speed between the light receiving element and the imaged object does not have to be completely zero, as long as the image blur is kept within an acceptable range.

The light receiving element is returned to its initial position during periods when light is not being received during intermittent imaging. This allows light reception to begin in the next light receiving period from the same position as in the previous light receiving period. In other words, the light receiving element moves in response to the relative movement of the imaged object during successive periods of light reception, and as a whole, the relative position of the light receiving element to the imaged object is sequentially changed.

The amount of movement required for the tracking movement of the light receiving element is very small, being approximately the same as the relative movement of the imaged object during the light receiving period. For example, if the length of the light receiving period is 1 millisecond and the scanning movement speed is 10 millimeters per second, which are typical values for fluorescence imaging, the amount of movement required is approximately 10 micrometers. Therefore, it is quite possible to achieve high-speed movement using a typical actuator.

Note that the "period during which the light receiving element receives fluorescence and the period during which it does not receive light" refers to the period during which the light receiving element effectively receives fluorescence as signal light and the period during which it does not receive it. For example, a method for the light receiving element to actively select whether to receive light or not is to make the incidence of fluorescence on the light receiving element intermittent by opening and closing an appropriate shutter mechanism while the excitation light source is continuously turned on. It is also possible to provide a period during which the light receiving element does not actually receive light by temporarily stopping the operation of the light receiving element, for example by cutting off the power supply, or by not reading out the signal from the light receiving element for a certain period of time or not using the read signal for subsequent processing.

On the other hand, there is a method in which the excitation light source irradiates excitation light intermittently to create periods when no fluorescence is emitted from the live sample, resulting in periods when fluorescence is received by the light receiving element and periods when it is not. For example, the light source itself may be turned on intermittently, or a continuously lit light source may be combined with a shutter mechanism that temporarily blocks the emitted light. In these cases, even if the light receiving element itself is in a state where it can receive light at all times, the fluorescence that should be received does not enter the light receiving element, and light is not received. Considering the phototoxicity that causes damage to live samples such as cells when irradiated with light, a method in which excitation light is intermittently incident on the live sample is more preferable in order to reduce damage.

As described above, according to the present invention, while the light receiving element receives light, the light receiving element is moved to follow the movement of the incident position of the light incident from the object to be imaged. This makes it possible to capture images of a wide imaging range at high speed while suppressing image blurring.

1 is a diagram showing an embodiment of an imaging device according to the present invention; FIG. 1 is a diagram illustrating the principle of intermittent imaging. 11A and 11B are diagrams illustrating image blur caused by a lighting pulse width. 4 is a timing chart showing an imaging operation in the present embodiment. 10 is a timing chart showing an operation when bright field imaging and fluorescent imaging are simultaneously processed.

Figure 1 shows one embodiment of an imaging device according to the present invention. Specifically, Figure 1(a) is a diagram showing a schematic diagram of the main configuration of the imaging device 1 of this embodiment, and Figure 1(b) is a block diagram showing the electrical configuration of the imaging device 1. In the coordinate axes shown in Figure 1(a), the XY plane is the horizontal plane, and the Z direction represents the up-down direction.

The imaging device 1 is a device for imaging a live sample S held in a sample container C. As the sample container C, various types of containers can be used, such as a shallow flat-bottomed dish called a dish, a well plate with an array of multiple wells for holding culture media, a glass slide, a flask, etc. It is desirable for the bottom surface of the sample container C to be flat and transparent. As a material for the container, for example, glass, transparent resin, etc. can be used.

Examples of the live sample S include tissue specimens collected from living organisms, artificially cultured cells, and their clumps. The live sample S to be subjected to the fluorescent imaging described below is stained (labeled) in advance using a fluorescent reagent or the like. If necessary, the live sample S is supported in a sample container C together with an appropriate culture medium M. The entire live sample S distributed in the culture medium M in the sample container C becomes the subject of imaging in this embodiment.

The imaging device 1 mainly comprises a holder 10, an imaging unit 20, a scanning movement mechanism 30, and a control unit 50. The holder 10 holds a sample container C in a substantially horizontal position with its bottom open. The imaging unit 20 images the sample S in the sample container C held by the holder 10. The scanning movement mechanism 30 moves the holder 10 and the imaging unit 20 relatively in the horizontal direction (Y direction in this example), thereby realizing the scanning movement of the imaging unit 20 relative to the sample container C held by the holder 10. The control unit 50 controls the operation of each part of the device to execute the imaging process described below.

The imaging unit 20 includes light sources 21 and 22, an imaging optical system 23, a focus control mechanism 24, a filter replacement mechanism 25, an imaging element 26, and a tracking movement mechanism 27. The light source 21 irradiates the live sample S with visible illumination light for bright-field imaging, and may be, for example, a white LED (Light Emitting Diode) light source. The light source 21 is disposed above the holder 10, and irradiates the live sample S with illumination light from the top side of the sample container C.

On the other hand, the light source 22 irradiates the live sample S with short-wavelength light, such as ultraviolet light, in order to excite fluorescence from the live sample S. For example, an ultraviolet laser can be used as the light source 22. When the live sample S is administered with multiple types of fluorescent reagents, excitation light of multiple different wavelengths is emitted from the light source 22. As described below, the excitation light emitted from the light source 22 is incident on the live sample S via the imaging optical system 23.

The imaging optical system 23 includes an objective lens 231, a filter section 232, a reflecting mirror 233, and an imaging lens 234. The objective lens 231 focuses the light emitted from the bottom surface of the sample container C. The objective lens 21 is coupled to a focus adjustment mechanism 24, which adjusts the focal position of the objective lens 231 by moving it vertically.

The filter section 232 includes a dichroic mirror 232a arranged on the optical path of the light focused by the objective lens 231, an excitation filter 232b arranged between the light source 22 and the dichroic mirror 232a, and a fluorescence filter 232c arranged between the dichroic mirror 232a and the reflecting mirror 233. The dichroic mirror 232a reflects the excitation light emitted from the light source 22 to enter the sample container C, while passing the light emitted from the raw sample S. The excitation filter 232b selectively passes a specific wavelength component of the light emitted from the light source 22 that acts as excitation light. Note that if the light emitted by the light source 22 itself contains only a specific wavelength component, the excitation filter can be omitted.

The fluorescent filter 232c passes the fluorescence of the detection target among the light emitted from the live sample S during fluorescent imaging, while absorbing other components. There are multiple types of fluorescent filters 232c with different passing wavelengths corresponding to multiple types of fluorescent reagents with different emission wavelengths, and the filter replacement mechanism 25 has the function of switching between these and selectively placing them in the optical path.

Light emitted from the light source 22 and passing through the excitation filter 232b becomes excitation light having only a predetermined wavelength component, and the excitation light whose optical path is folded back by the dichroic mirror 232a is incident on the live sample S via the objective lens 231. Fluorescence excited from the live sample S by the excitation light is collected by the objective lens 231 and enters the imaging lens 234 via the fluorescence filter 232c and the reflection mirror 233 for folding back the optical path. The imaging lens 234 collects the incident light and forms an image of the live sample S on the light receiving surface 261 of the image sensor 26.

The imaging element 26 is a two-dimensional image sensor (area sensor), and may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging element 26 captures a two-dimensional image of the raw sample S formed on the light receiving surface 261 by the imaging lens 234. More specifically, it captures a two-dimensional image of the culture medium M within a predetermined imaging field including the raw sample S as viewed from the bottom surface of the sample container C, and outputs an image signal according to the content.

When white illumination light is irradiated from the light source 21, the image sensor 26 can capture a bright-field image of the live sample S. When excitation light is irradiated from the light source 22, the image sensor 26 can capture a fluorescent image of the live sample S.

The imaging element 26 is supported by a tracking movement mechanism 27 so that it can move up and down. More specifically, the tracking movement mechanism 27 has a fixed part 271 fixed to an appropriate frame and a movable part 272 attached so as to be movable up and down relative to the fixed part 271, and the imaging element 26 is attached to the movable part 272. The movable part 272 moves up and down due to the operation of an appropriate actuator, thereby moving up and down the imaging element 26. In other words, the imaging element 26 moves back and forth within a predetermined movable range by moving alternately in the directions of the arrows Dr and Dt.

The required range of motion for the imaging element 26 is at most several hundred micrometers. The required movement speed is the same as the scanning movement speed of the scanning movement mechanism 30, which is, for example, about 10 millimeters per second, as described below. It is more preferable to be able to achieve movement at a higher speed than this. As an actuator for achieving such movement, various types of actuators can be used, such as piezoelectric elements, linear motors, solenoids, etc.

The scanning movement mechanism 30 realizes the scanning movement of the imaging unit 20 relative to the sample container C. Specifically, it moves at least one of the holder 10 that holds the sample container C and the imaging unit 20 in the horizontal direction. The relative movement between the holder 10 and the imaging unit 20 thus realized corresponds to the scanning movement of the imaging unit 20 relative to the sample container C.

Moving either the holder 10 or the imaging unit 20 is technically equivalent. From the viewpoint of not applying stimuli such as vibrations to the live sample S in the sample container C, it is practical to fix the holder 10 and move the imaging unit 20. In this case, at least the light source 21, the imaging optical system 23, and the imaging element 26 of the imaging unit 20 must be moved as a unit. Note that if the light source 21 uniformly illuminates the entire surface of the sample container 21, there is no need to move the light source 21.

However, for the sake of convenience, the following description will be given assuming that the holder 10 moves relative to the stationary imaging unit 20. The direction of movement of the holder 10 at this time, that is, the scanning movement direction of the sample container C as seen from the imaging unit 20, will be represented by the symbol Ds. In this example, the scanning movement direction Ds is the (+Y) direction.

The required scanning movement speed is, for example, about 10 millimeters per second. As a driving mechanism for achieving such movement, for example, a ball screw mechanism, a linear motor, a belt drive mechanism, or any other suitable mechanism can be used.

The control unit 50 controls each part of the device described above. For this purpose, the control unit 50 includes a CPU (Central Processing Unit) 51, a memory 52, a storage 53, and an interface (IF) 54.

The CPU 51 executes a control program stored in advance in the storage 53 and causes each part of the device to perform a specific operation, thereby achieving various processes. The memory 52 temporarily stores data generated during processing by the CPU 51. The storage 53 stores and saves the control program to be executed by the CPU 51, the original image data obtained by capturing an image, and the processed image data obtained by processing the original image data.

By the CPU 51 executing the control program, various functional blocks are realized in software. The functional blocks realized in this way include a light source control unit that controls the lighting of the light sources 21, 22, an imaging control unit that controls the imaging process by the imaging element 26, an image processing unit that performs various image processing on the original image data to create the desired image, and a mechanical control unit that controls the movable mechanisms provided in the focus adjustment mechanism 24, the scanning movement mechanism 30, etc. Note that some of these may be realized in hardware.

The interface unit 54 has a user interface function of accepting operation input from the user and presenting information such as processing results to the user, as well as a function of exchanging data with an external device connected via a communication line. To realize the user interface function, the interface unit 54 is connected to an input acceptance unit 541 that accepts operation input from the user, and a display unit 542 that displays and outputs messages to the user, processing results, etc.

The control unit 50 has a hardware configuration equivalent to that of a general-purpose processing device such as a personal computer or a workstation. Therefore, the control unit 50 may be a general-purpose processing device incorporating a control program for implementing the processing functions described below. In other words, a general-purpose computer can be used as the control unit 50 of this imaging device 1. When using a general-purpose processing device, it is sufficient for the imaging device 1 to have the minimum control functions required to operate each part of the imaging unit 20, etc.

The imaging process in the imaging device 1 configured as described above will be described below. In order to image an area larger than the area that can fit in the imaging field of view in one imaging session, the imaging device 1 performs imaging while moving the imaging unit 20 in a scanning manner relative to the sample container C. Because the imaging element 26 is a two-dimensional image sensor, imaging can be performed intermittently while moving the imaging unit 20 in a scanning manner, thereby sequentially capturing images of the imaging target area.

Figure 2 is a diagram showing the principle of intermittent imaging. As shown in Figure 2(a), after one imaging is performed with a partial region R1 of the imaging target region Rt contained within the imaging field of view of the imaging unit 20, the imaging field of view is moved to another partial region R2 and imaging is performed again. By repeating this process, it is possible to capture an entire area larger than the imaging field of view in one of the images. By stitching these images together, it is possible to include the entire imaging target region Rt in a single image. For smooth stitching, it is desirable for images captured at adjacent positions to partially overlap each other.

For example, such images can be obtained by scanning the imaging field of view over the imaging target region Rt at a constant speed while illuminating light is incident at constant time intervals and imaging each time. For example, the light source may be pulsed at a constant period Tc. In bright-field imaging, phase-contrast imaging, dark-field imaging, etc., which use visible light as illumination light and directly receive a portion of it, the illumination time Tv required for pulsed illumination is relatively short, for example, on the order of several tens of microseconds, as shown in Figure 2(a).

On the other hand, in fluorescence imaging, the excitation light is not received directly, but rather the fluorescence emitted by the fluorescent material excited by the excitation light is received. For this reason, it is necessary to ensure a sufficient amount of exposure to excite the subject, and as shown in Figure 2 (b), the illumination time Te required for pulse illumination is, for example, about 1 millisecond. If the illumination time (pulse width) becomes longer in this way, the obtained image will become blurred.

Figure 3 is a diagram explaining image blurring caused by the lighting pulse width. In the diagram, the top row shows the lighting pulse width of the illumination light, the middle row shows a light path diagram showing the path of the light during the lighting period, and the bottom row shows an example of the image obtained. Note that in the middle row, to make the diagram easier to see, it is assumed that there is no folding of the light path by the reflecting mirror 233. This diagram shows the shape of the image of a point P on the imaging target plane St when illumination light is incident with the pulse width shown in the top row.

First, consider the case where illumination light with a relatively short pulse width, i.e., a short lighting time, is irradiated, as shown in (a). At the start of irradiation, an image of point P appears at position Q on the light receiving surface 261 of the image sensor 26. Point P moves with the scanning movement in the direction of the arrow Ds, and at the end of irradiation, it has moved to the position shown as point P' in the figure. A corresponding image appears at position Q' on the light receiving surface 261. During this time, illumination light is irradiated continuously, so in the captured image, the image that should be a single point extends long in the direction corresponding to the scanning movement direction, which causes the image to blur. As shown in (b) of the figure, the wider the pulse width, the greater the image blur.

This is not limited to fluorescence imaging, but can also occur in imaging under visible light illumination. For example, a similar problem can occur when the illumination light is turned on for a long time to ensure a sufficient amount of light in phase contrast imaging or dark field imaging. The following method can be applied to solve this problem.

As shown in (c), the image sensor 26 is displaced along the moving direction Dt at the same speed as the moving speed of the image position Q associated with the movement of the point P due to the scanning movement. Then, even if the point P moves, the position on the light receiving surface 261 where the image is formed does not change. In other words, the image of the point P is recorded in the image as a clear point without image blur. In this way, by moving the image sensor 26 so as to follow the movement of the image formation position associated with the scanning movement, it is possible to obtain an image without image blur. Such movement of the image sensor 26 is referred to as "following movement" in this specification. Also, the direction along the moving direction of the image formation position associated with the scanning movement is referred to as the "following direction Dt", and the opposite direction is referred to as the "reverse direction Dr".

In this embodiment, in order to achieve the above-described tracking movement, the image sensor 26 is supported by a tracking movement mechanism 27. Below, the imaging operation of this embodiment, which combines the scanning movement of the sample container C, the intermittent irradiation of the excitation light, and the tracking movement of the image sensor 26, will be described. For the following explanation, of the positions of the image sensor 26 shown in FIG. 3(c), the position at the start of illumination, indicated by a solid line, is represented by the symbol P1, and the position at the end of illumination, indicated by a dotted line, is represented by the symbol P2.

Figure 4 is a timing chart showing the imaging operation in this embodiment. As described above, the sample container C is scanned relative to the imaging unit 20 at a constant speed, while excitation light is intermittently incident on the sample container C at a constant period Tc. At this time, the tracking movement mechanism 27 moves the imaging element 26 back and forth at the period Tc. That is, the tracking movement mechanism 27 moves the imaging element 26 back and forth between a predetermined movement start position Ps and movement end position Pe within the movable range of the imaging element 26. The direction from the movement start position Ps to the movement end position Pe is the tracking direction Dt, and the opposite direction is the reverse direction Dr.

The illumination of the excitation light and the tracking movement are synchronized as follows. That is, the reciprocating period of the tracking movement is the same as the illumination period Tc of the excitation light. In addition, during the period when the excitation light is illuminated, the image sensor 26 moves in the tracking direction Dt. The position of the image sensor 26 at the time when the excitation light is turned on is position P1 shown in FIG. 3(c), and the position of the image sensor 26 at the time when the excitation light is turned off is position P2 shown in FIG. 3(c). At least the speed at which the image sensor 26 moves from position P1 to position P2 corresponds to the scanning movement speed.

Light incident on the image sensor 26 during the period when the excitation light is on is received as valid signal light. During this time, the image sensor 26 moves from position P1 to position P2 at a constant speed, following the movement of the imaging position associated with the scanning movement. As a result, light emitted from one point on the raw sample S is incident on the same point on the light receiving surface 261 of the image sensor 26, regardless of displacement due to the scanning movement. Therefore, no blurring of the image occurs.

The image signal can be read out from the image sensor 26 at any timing within the period from when the excitation light is turned off to when it is turned on again. During this period, the tracking movement mechanism 27 moves the image sensor 26 in the reverse direction Dr back to the movement start position Ps. Then, before the excitation light is next turned on, it resumes movement in the tracking direction Dt.

By repeating such operations while performing imaging, macroscopically, the image sensor 26 scans and moves relative to the live sample S to be imaged, while microscopically, the relative position between the image sensor 26 and the live sample S remains substantially unchanged during the short period when the excitation light is on. This makes it possible to prevent blurring of the image caused by the positional relationship between the live sample S and the image sensor 26 changing due to scanning movement during irradiation with the excitation light.

In addition, the image sensor 26 in FIG. 4 moves at a constant speed from the movement start position Ps to the movement end position Pe, and then returns to the movement start position Ps at a constant speed. However, the requirements for preventing the image from becoming blurred are that at least the movement from position P1 to position P2 (following movement) is performed in the following direction Dt at a constant speed corresponding to the scanning movement speed, and that the image sensor 26 returns to the original position before the next excitation light is turned on. Therefore, the manner of movement in other sections is not limited to this. For example, the image sensor 26 may stop for a certain period of time at least at the movement end position Pe and the movement start position Ps. In addition, the movement in the reverse direction Dr does not have to be at a constant speed. For this reason, as the following movement mechanism 27, a mechanism in which the movement speed in one direction is the same as the movement speed in the opposite direction, or a mechanism in which the movement speeds are not the same, can be used.

In the above explanation, the speed at which the imaging element 26 moves in a tracking manner from position P1 to position P2 is "a speed corresponding to the scanning movement speed." In other words, the speed at this time is not necessarily the same as the scanning movement speed. The moving speed of the imaging position on the light receiving surface 261 of the imaging element 26 when a point in the sample container C moves at the scanning movement speed due to the scanning movement should be the tracking movement speed. For example, if the imaging optical system 23 is a magnifying optical system or a reducing optical system, the moving speed of the imaging position also changes depending on the magnification. Therefore, the speed of the tracking movement needs to be determined according to the moving speed of the actual imaging speed.

The moving speed of the imaging position and the speed of the tracking movement do not have to be exactly the same. When these speeds are the same, the imaging position on the image sensor 26 remains unchanged, so in principle no image blurring occurs. On the other hand, if a certain degree of image blurring is acceptable, some speed difference may be acceptable. For example, if the deviation in the imaging position is smaller than the pixel size of the image sensor 26, the deviation can be essentially ignored.

The tracking direction Dt is also not necessarily the same as the scanning movement direction. As mentioned above, the tracking direction Dt changes depending on whether the optical path is turned back or whether the imaging optical system 23 is an upright or inverted system. The tracking direction Dt is simply the direction in which the imaging position on the imaging element 26 changes due to scanning movement.

In the above explanation, in order to make it easier to understand the principle, an example was given in which only one fluorescence image is taken during one period Tc. However, in medical and research settings, it is common to compare an image suitable for visual observation, such as a bright-field image, with a fluorescence image. For this purpose, it is desirable to acquire both a bright-field image and a fluorescence image of the same sample. Although the scanning for bright-field imaging and the scanning for fluorescence imaging may be performed separately, it is convenient to acquire these images simultaneously in a series of scans, as this shortens the time required for imaging. This can be achieved as follows.

Figure 5 is a timing chart showing the operation when bright-field imaging and fluorescent imaging are simultaneously processed. First, we focus on bright-field imaging using a white light source. With scanning movement at a constant speed, the white illumination light for bright-field imaging is turned on only for a fixed period of the cycle Tc. By receiving light incident on the image sensor 26 each time it is turned on, bright-field images Iv1, Iv2, Iv3, ..., for each imaging field are acquired in sequence. The lighting cycle Tc is determined in relation to the scanning movement speed so that the imaging fields of view in these bright-field images overlap slightly.

After the white illumination light is turned off, excitation light (abbreviated here as "excitation light B") for exciting blue (B) fluorescence is turned on for a certain period of time. At this time, a fluorescent filter 232c corresponding to blue is placed in the optical path of the signal light. Blue fluorescent images (abbreviated here as "fluorescent images B") Ib1, Ib2, Ib3, ... are acquired from the signal light incident on the image sensor 26. Note that the entire filter section 232 may be replaced as necessary, that is, the dichroic mirror 232a and excitation filter 232b may be switched together with the fluorescent filter 232c.

Similarly, after the excitation light B is turned off, the filter replacement mechanism 25 switches the fluorescent filter 232c to one corresponding to green (G), and the excitation light for exciting green fluorescence (abbreviated here as "excitation light G") is turned on for a certain period of time. After the light is turned off, the fluorescent filter 232c is switched to one corresponding to red (R), and the excitation light for exciting red fluorescence (abbreviated here as "excitation light G") is turned on for a certain period of time. During these turn-on periods, green fluorescent images (abbreviated here as "fluorescence images G") Ig1, Ig2, Ig3, ..., and red fluorescent images (abbreviated here as "fluorescence images R") Ir1, Ir2, Ir3, ..., are acquired.

The imaging fields of view of these bright-field images and fluorescent images are slightly shifted due to differences in the lighting timing. However, because a continuous image can be obtained by combining overlapping images, the shift in imaging position is not a real problem. In addition, because the amount of positional shift between images is known, it is easy to align the images of each color based on that information.

During this time, based on the above principle, the imaging element 26 moves back and forth in the following direction Dt and the reverse direction Dr. More specifically, each time fluorescence imaging of each color is performed, the imaging element 26 moves in the following direction Dt, and after imaging is completed, moves in the reverse direction Dr before imaging of the next color begins. When imaging any color, the imaging element 26 is at position P1 when the excitation light starts to be turned on, and at position P2 when it ends. This prevents blurring of the fluorescence images of each color. Furthermore, the positional relationship between the imaging optical system 23 and the imaging element 26 during imaging is the same for all colors, allowing the fields of view of the imaging optical system 23 and the imaging element 26 to be used effectively.

The tracking movement of the image sensor 26 is not essential in bright-field imaging. For this reason, the image sensor 26 can be left stationary. If the position at this time is set to position P1, the positional relationship between the imaging optical system 23 and the image sensor 26 can be made the same for bright-field images and fluorescent images. In principle, it is effective to have the image sensor 26 track the movement of the imaging position even in the case of bright-field imaging. This is because, even though the illumination time of the light source 21 is sufficiently short, scanning movement is still performed during that time.

The above is the operation when capturing the fluorescent images of each color of RGB individually. In this method of capturing fluorescent images of each color while switching the fluorescent filter used, it is possible to use, for example, an image sensor 26 for monochrome imaging. On the other hand, when an image sensor 26 for color imaging is used, RGB can be separated after capturing images. In this way, if the individual fluorescent colors to be observed can be separated, the images may be captured collectively as color images without distinguishing between colors. In this case, there is no need to switch fluorescent filters. Also, if multi-band compatible filters are used as the dichroic filter 232a, excitation filter 232b, and fluorescent filter 232, there is no need to replace the filter section even when capturing images of each color individually.

As described above, in the imaging device 1 of the above embodiment, by moving the imaging element 26 so as to follow the displacement of the imaging position accompanying the scanning movement, it is possible to prevent image blurring even in fluorescence imaging in which illumination light (excitation light) is irradiated for a relatively long period of time. The amount of movement required of the imaging element 26 is very small, and simple reciprocating movement is sufficient, so that the tracking operation can be achieved with relatively small and simple mechanical components.

As described above, in the above embodiment, the imaging unit 20 functions as the "imaging section" of the present invention, and the light source 22, the image sensor 26, and the tracking movement mechanism 27 function as the "excitation light source," "light receiving element," and "tracking movement mechanism" of the present invention, respectively. Also, the light source 21 functions as the "illumination light source" of the present invention.

The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, as described above, the scanning movement between the sample container C and the imaging unit 20 may involve moving either one or both.

In addition, in the above embodiment, the imaging element 26 is moved up and down by the tracking movement mechanism 27 to achieve tracking movement, but the imaging element may be moved relative to the optical system by driving optical elements such as lenses and mirrors included in the imaging optical system 23.

In addition, in the tracking operation in the above embodiment, when the image sensor 26 is moved from position P1 to position P2 during the illumination period of the excitation light, the image sensor 26 is moved back and forth between the outer movement start position Ps and movement end position Pe. This is to ensure stable movement from position P1 to position P2 at a constant speed. However, if a similarly stable movement speed is achievable or a certain degree of speed fluctuation is acceptable, the movement may be started from position P1 and may be ended at position P2.

For example, in the above embodiment, intermittent imaging is performed by pulsating the light source, but the same effect can be obtained by intermittently receiving light using the image sensor. For example, it is possible to perform intermittent imaging by using an imaging unit equipped with a global shutter and opening the shutter for only a portion of a certain period. However, since irradiating the live sample S with strong light for a long period of time can damage the live sample S (phototoxicity), it is desirable not to turn on the light source except when imaging, as described above. In addition, by using an imaging unit equipped with a rolling shutter and synchronizing the rolling shutter and the lighting of the light source, it is possible to shorten the irradiation time of the live sample S and improve the tracking accuracy of the image sensor.

In addition, in the above embodiment, bright-field imaging and fluorescent imaging are combined and performed simultaneously, but as explained in the principle explanation, the present invention also functions effectively in fluorescent imaging without bright-field imaging. In addition, imaging using other imaging methods, such as phase contrast imaging or dark-field imaging, may be combined instead of bright-field imaging.

As described above by way of example of specific embodiments, in the present invention, for example, the excitation light source may be one that intermittently irradiates excitation light onto the object to be imaged. With such a configuration, the intermittent illumination of the excitation light also results in intermittent generation of fluorescence, and the fluorescence can be intermittently received by the light receiving element.

In this case, for example, the tracking movement mechanism may be configured to move the light receiving element in synchronization with the incidence of excitation light from the excitation light source to the imaged object. With such a configuration, the light receiving element moves in synchronization with the intermittent irradiation of excitation light, making it possible to prevent image blurring in each image captured for each irradiation.

For example, the light receiving element may be a two-dimensional image sensor. With this configuration, a wide two-dimensional area can be imaged in one capture, so a wide target area can be imaged at high speed by sequentially changing the imaging position. In this case, continuous imaging causes image blurring due to scanning movement, but this problem can be suppressed by performing intermittent imaging.

For example, the imaging unit may be equipped with an illumination light source that causes visible light to be incident on the imaging target, and during scanning movement, imaging performed while causing visible light to be incident on the imaging target and imaging performed while causing excitation light to be incident thereon may be repeated in alternation. With this configuration, imaging in the visible light range and fluorescence imaging can be performed in a single scan, thereby shortening the time required to obtain these different types of images. In addition, because the time difference between imaging is small, changes in the state of the imaging target between images can be avoided.

For example, when the light receiving element receives multiple fluorescent light having different wavelengths at different times, the tracking movement mechanism may be configured to move the light receiving element from the initial position in the tracking direction while the light receiving element receives light for one wavelength, and to return the light receiving element to the initial position before light reception for the next wavelength begins. With such a configuration, when multiple fluorescent components having different emission wavelengths are received in time series, the position at which the fluorescent light enters the light receiving element can be aligned for each wavelength component, and images with good image quality can be obtained. For example, imaging can be performed by effectively utilizing the field of view of the imaging optical system.

The present invention is particularly effective when the object to be imaged is cells cultured in a container or a pathological tissue specimen.

This invention can be applied to all imaging techniques that capture fluorescent images while scanning an object. It is particularly suitable for observing living specimens such as cells and tissue samples, and is useful in the fields of medicine and biochemistry.

1 Imaging device 20 Imaging unit (imaging section)
21 Light source (illumination light source)
22 Light source (excitation light source)
26 Imaging element (light receiving element)
27 Follow-up movement mechanism 30 Scanning movement mechanism Ds Scanning movement direction Dt Follow-up direction Ps Movement start position (initial position)
S: Raw sample (image subject)

Claims (8)

an imaging section that irradiates an imaging object with excitation light, receives fluorescence emitted from the imaging object, and images the imaging object;
a scanning movement mechanism that scans and moves the imaging unit at a constant speed in a direction intersecting with a direction of incidence of the excitation light relative to the imaging object,
The imaging unit includes:
An excitation light source that emits the excitation light;
a light receiving element that intermittently receives the fluorescence emitted from the image capturing object;
a tracking movement mechanism that is capable of moving the light receiving element within a predetermined range in a tracking direction and an opposite direction to the tracking direction, the direction being a direction in which an optical path of the fluorescent light moves in association with the scanning movement from a point on the image capture object to the light receiving element;
the light receiving element is a two-dimensional image sensor,
The following movement mechanism includes:
During a period in which the light receiving element receives the fluorescent light, the light receiving element is moved from a predetermined initial position along the tracking direction at a speed corresponding to the scanning movement speed , thereby causing the light receiving element to track the movement of an imaging position where an image of the imaged object is formed on the imaging unit ;
The imaging device moves the light receiving element in the reverse direction and returns it to the initial position during a period in which the light receiving element does not receive the fluorescent light. The imaging device according to claim 1, wherein the excitation light source intermittently irradiates the excitation light onto the imaging subject. The imaging device according to claim 2, wherein the tracking movement mechanism moves the light receiving element in synchronization with the incidence of the excitation light from the excitation light source on the imaging object. The imaging device according to claim 1 , wherein a speed at which the light receiving element moves in the tracking direction is equal to a speed at which the imaging position moves due to the constant speed scanning movement. The imaging device according to any one of claims 1 to 4, wherein the imaging unit includes an illumination light source that causes visible light to be incident on the imaging target, and during the scanning movement, imaging performed while the visible light is incident on the imaging target and imaging performed while the excitation light is incident thereon are alternately performed. the light receiving element receives the plurality of fluorescent lights having different wavelengths at different times;
6. The imaging device according to claim 1, wherein the tracking movement mechanism moves the light receiving element from the initial position in the tracking direction while the light receiving element receives light of one wavelength component, and returns the light receiving element to the initial position before light reception of a next wavelength component begins. 1. An imaging method for imaging an object by irradiating excitation light onto the object and receiving fluorescence emitted from the object, comprising:
an imaging unit having an excitation light source and a light receiving element which is a two-dimensional image sensor , and the imaging object are relatively scanned and moved at a constant speed in a direction intersecting with the incident direction of the excitation light;
a direction in which an optical path of the fluorescence from a point on the image capture object to the light receiving element moves in association with the scanning movement is defined as a tracking direction, and the light receiving element is reciprocated within a predetermined range in the tracking direction and an opposite direction thereto, while receiving the fluorescence from the image capture object during a period in which the light receiving element is moving in the tracking direction;
In the movement of the light receiving element in the tracking direction, the light receiving element is moved along the tracking direction at a speed corresponding to the scanning movement speed, thereby causing the light receiving element to track the movement of an imaging position where the object to be imaged is imaged on the imaging unit . The imaging method according to claim 7, wherein the imaging subject is a cell cultured in a container or a pathological tissue specimen.

Images