JP5562653B2 - Virtual slide creation apparatus and virtual slide creation method - Google Patents

Description

  The present invention relates to a virtual slide creation device and a virtual slide creation method.

Conventionally, in the field of medical treatment such as pathological diagnosis, it is necessary not only to grasp the limited fine structure of the observation target such as tissue and cells, but also to perform diagnosis while grasping the whole image of the observation target A virtual slide is known as a method for efficiently observing the whole image and a fine observation image (see, for example, Patent Document 1).
The virtual slide is a high-magnification microscopic image obtained by partially imaging each part of the observation target, repeatedly shifting the imaging range so as to partially overlap, and performing image processing on the acquired multiple microscope images. By pasting it on the content of the shift, it is possible to virtually observe a slide image as if the entire observation object was photographed.

  In the virtual slide creation method disclosed in Patent Document 1, an XY stage equipped with an observation target is driven, and a microscope image is repeatedly photographed at each position positioned by shifting the position of the observation target with respect to the optical axis of the objective lens. doing. Then, in order to paste together a plurality of acquired microscope images without deviation, adjacent microscope images are photographed with an overlap, image pattern recognition is performed within the overlapped range, and the microscope images are matched. Yes.

JP 2008-191427 A

  However, there is an inconvenience that the longer the overlap width, the longer the time required for image processing such as image pattern recognition. On the other hand, if an attempt is made to set the overlap width to be small, the XY stage must be precisely positioned, which increases the time required for positioning and increases the time required to acquire a microscope image. There is an inconvenience.

  The present invention has been made in view of the above-described circumstances, and reduces the time required for image processing without increasing the time required for acquiring a microscope image, and reduces the time required for obtaining a virtual slide. An object of the present invention is to provide a virtual slide creation device and a virtual slide creation method that can perform the above-described process.

In order to achieve the above object, the present invention provides the following means.
The present invention provides a stage on which a specimen is mounted, an objective lens that collects light from the specimen mounted on the stage, and a direction in which at least one of the stage and the objective lens intersects the optical axis of the objective lens A relative movement mechanism that moves relative to each other, a position detection unit that obtains relative position information between the stage and the objective lens by the relative movement mechanism, and photographs the light collected by the objective lens An imaging unit that acquires a partially enlarged image, and an enlarged image and the enlarged image acquired by the imaging unit during movement in a direction intersecting the optical axis of the objective lens of at least one of the stage and the objective lens a storage unit that stores an association and relative position information of the stage detected by said position detecting unit at the time of acquisition of the image and the objective lens, the stage Oyo At least one of the moving in a direction intersecting the optical axis of the objective lens, the relative position stored in association with a plurality of enlarged image stored in the storage unit to the enlarged image of the objective lens Provided is a virtual slide creation device including an image processing unit for pasting according to information.

  According to the present invention, by mounting the sample on the stage and operating the relative movement mechanism, the imaging unit moves the objective lens and the sample relative to each other in the direction intersecting the optical axis of the objective lens. A partial enlarged image of each part of the specimen can be acquired. Then, a partial enlarged image of the specimen is acquired by the operation of the imaging unit, and the relative position information between the objective lens and the stage is acquired by the operation of the position detection unit at the time of acquisition, and the enlarged image and the image thereof are stored in the storage unit. The relative position information at the time of acquisition is stored in association with each other.

  In this case, since the relative position information acquired at the time of acquiring the enlarged image is associated with the enlarged image, accurate position information is given to the enlarged image without precisely positioning the stage and the objective lens by the relative movement mechanism. be able to. In other words, since precise positioning between the stage and the objective lens is not performed, the time required for positioning can be shortened and accurate position information is given to the enlarged image, so that the enlarged images are arranged by the image processing unit. The time required for image processing at the time can be shortened. As a result, the time required to obtain the virtual slide can be greatly shortened.

In the above invention, the relative movement mechanism moves at least one of the stage and the objective lens to a preset relative imaging position by open loop control, and outputs a movement completion signal at the end of movement. The imaging unit may acquire a partially enlarged image of the specimen in response to a movement completion signal from the relative movement mechanism.
By doing so, since the relative movement mechanism is not feedback-controlled based on the detected relative position information, the stage and the objective lens are not precisely positioned, and the time required for positioning can be shortened.

In the above invention, the relative movement mechanism may store a movement pattern of at least one of the stage and the objective lens and move the movement according to the movement pattern.
In this way, by simply moving at least one of the stage and the objective lens according to a preset movement pattern, the coordinate designation and command communication time for moving to each imaging position can be shortened, and in a shorter time. Virtual slides can be created.

The relative movement mechanism may start moving at least one of the stage and the objective lens from one relative imaging position to another relative imaging position by a trigger signal.
In this way, just by moving at least one of the stage and objective lens according to the trigger signal, the coordinate specification and command communication time for moving to each imaging position can be shortened, and a virtual slide can be created in a shorter time. can do.

In the above invention, the relative imaging positions adjacent to each other are set to positions where enlarged images that overlap each other can be acquired, and the image processing unit is stored in association with the adjacent enlarged images. On the basis of the relative position information, the image pattern recognition process may be executed using the luminance information of the pixel columns arranged at the same distance from the center of the enlarged image.
By doing in this way, since the whole overlap part between adjacent enlarged images is not used for image pattern recognition, the amount of image processing can be reduced and a virtual slide can be created in a short time. That is, even if the width of the overlap portion is set to be large, it is not necessary to increase the time required for image pattern recognition, and only a rough positioning is required, so that the time required for positioning can be greatly shortened.

Further, the present invention provides a moving step of relatively moving at least one of a specimen and an imaging optical system that acquires a partially enlarged image of the specimen in a direction intersecting the optical axis of the imaging optical system, A detection step of acquiring relative position information with respect to the imaging optical system, an imaging step of acquiring an enlarged image of the specimen by the imaging optical system at the time when the relative position information is acquired in the detection step, and the movement step in the storing step stores an association with the relative position information between the specimen and the imaging optical system at the time of acquisition of the acquired magnified image and the enlarged image by the photographing step, during said moving step, the storage stored plurality of enlarged image association image align therefore stuck to the relative position information stored in processing the enlarged image in step step To provide a virtual slide creation methods, including door.

  In the above invention, the moving step moves at least one of the specimen and the imaging optical system so as to overlap adjacent enlarged images, and the image processing step is stored in association with the adjacent enlarged images. On the basis of the relative position information, the image pattern recognition process may be executed using the luminance information of the pixel columns arranged at the same distance from the center of the enlarged image.

  According to the present invention, the time required for image processing can be shortened without increasing the time required for acquiring a microscope image, and the time required for obtaining a virtual slide can be shortened.

It is a whole lineblock diagram showing the virtual slide creation device concerning one embodiment of the present invention. It is a block diagram which shows the memory | storage part and image processing part with which the control part of the virtual slide production apparatus of FIG. 1 was equipped. It is a time chart explaining the virtual slide creation method using the virtual slide creation apparatus of FIG. It is a flowchart explaining the virtual slide creation method of FIG. It is a figure explaining the image bonding step in the virtual slide creation method of FIG. It is a figure explaining the modification of the image bonding step of the virtual slide creation method of FIG.

A virtual slide creation device and a virtual slide creation method according to an embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the virtual slide creation device 1 according to the present embodiment displays a generated virtual slide, a microscope main body 2, a control unit 3 that controls the microscope main body 2 and generates a virtual slide. The display part 4 to be provided is provided.

  The microscope main body 2 is moved by a stage 5 on which a specimen A placed on a slide glass is mounted, motors (relative movement mechanisms) 6a and 6b that move the stage 5 in two horizontal directions, and the motors 6a and 6b. Sensors (position detection units) 7a and 7b that detect the position (relative position information) of the stage 5 relative to the origin, an objective lens 8 that is disposed to face the specimen A and collects light from the specimen A, and An imaging unit 9 such as a CCD that captures the light collected by the objective lens 8 and obtains an enlarged image of the specimen A is provided.

The stage 5 is moved in the arrow X direction by the motor 6a, and is moved in the arrow Y direction orthogonal to the arrow X by the motor 6b.
When the position of the stage 5 is detected, the sensors 7 a and 7 b temporarily store it and transmit it to the control unit 3. Specifically, the sensor 7a detects the position of the stage 5 in the arrow X direction, and the sensor 7b detects the position of the stage 5 in the arrow Y direction.
Further, when the enlarged image of the specimen A is acquired, the imaging unit 9 temporarily stores it and transmits it to the control unit 3.

As shown in FIG. 2, the control unit 3 includes a storage unit 10 that stores an image acquired by the microscope body 2 and an image processing unit that generates a virtual slide by arranging the images stored in the storage unit 10. 11.
The control unit 3 stores a preset movement pattern, and outputs a movement command signal to the motors 6a and 6b so as to move the stage 5 with respect to the objective lens 8 according to the movement pattern. Yes. The motors 6a and 6b are open-loop controlled by a movement command signal output from the control unit 3, and move the stage 5 by a movement amount that matches the movement command signal.

  When the control unit 3 operates the motors 6a and 6b by the amount of movement corresponding to the stage 5 placed at a position predetermined by the movement pattern, a movement completion signal is generated from the motors 6a and 6b. Upon receiving the movement completion signal, the control unit 3 instructs the sensors 7a and 7b to detect the position information of the stage 5, and instructs the imaging unit 9 of the microscope body 2 to acquire an image. It has become. Then, the control unit 3 instructs the motors 6a and 6b to move the stage 5 when the acquisition of the enlarged image by the imaging unit 9 is completed.

  The transmission of the position information to the control unit 3 by the sensors 7a and 7b and the transmission of the image information to the control unit 3 by the imaging unit 9 are performed while the stage 5 is moved by the motors 6a and 6b. The transmitted position information and image information are stored in association with the storage unit 10 provided in the control unit 3.

  The image processing unit 11 arranges the enlarged images of each part of the specimen A stored in the storage unit 10 so as to paste the enlarged images and generate a virtual slide. Specifically, since the enlarged image of each part of the specimen A stores the positional information of the stage 5 at the time when the enlarged image is acquired, the enlarged image is arranged according to the positional information of the stage 5. Thus, a virtual slide can be generated simply by compositing. The bonding process by the image processing unit 11 is also performed while the stage 5 is moved by the motors 6a and 6b.

A virtual slide generation method using the virtual slide generation device 1 according to the present embodiment configured as described above will be described below with reference to FIGS. 3 and 4.
In order to generate the virtual slide of the specimen A using the virtual slide generating apparatus 1 according to the present embodiment, first, the control unit 3 moves the stage 5 to the first imaging position according to a preset movement pattern. A movement command signal is output to the motors 6a and 6b (step S1).

  The motors 6a and 6b move the stage 5 according to the movement command signal from the control unit 3 (step S2). At this time, the motors 6a and 6b are not feedback-controlled by the position information detected by the sensors 7a and 7b, but are moved by the movement amount according to the movement command signal and stopped at each imaging position by the open loop control by the control unit 3. To do. Then, when stopped at the imaging position, the motors 6a and 6b output a movement completion signal to the control unit 3 (step S3).

  The control unit 3 that has received the movement completion signal instructs the sensors 7a and 7b to store the current position of the stage 5, and also obtains an enlarged image of the sample A at the current position with respect to the imaging unit 9. To Upon receiving the command, the sensors 7a and 7b temporarily store the current position, and the imaging unit 9 acquires and temporarily stores an enlarged image at the current position, and outputs an imaging completion signal to the control unit 3 (step S4). ).

And the control part 3 gives the movement command signal (trigger signal) with respect to motor 6a, 6b so that the stage 5 may be moved to the next imaging position according to the movement pattern set beforehand, when the imaging completion signal is received. Output (step S5).
The motors 6a and 6b move the stage 5 in response to a movement command signal from the control unit 3 (step S6). When the movement of the stage 5 by the motors 6a and 6b starts, the imaging unit 3 and the sensors 7a and 7b transmit the temporarily stored enlarged image information and position information to the control unit 3 (step S7). The control unit 3 associates the transmitted enlarged image information and the position information with each other and stores them in the storage unit 10, and associates the enlarged image information stored in the storage unit 10 with the image processing unit 11. A virtual slide is generated by pasting according to the stored position information (step S8).

  The transmission of the enlarged image information and the position information to the control unit 3, the storage to the storage unit 10, and the pasting process are performed during the movement of the stage 5 to the next imaging position by the motors 6 a and 6 b, and the movement is completed. End by. Thereafter, the operation from step S3 is repeated, and the virtual slide generation process ends when the pasting process of the magnified image information at all the imaging positions scheduled by the preset movement pattern is completed (step S3). S9).

  As described above, according to the virtual slide generation device 1 and the virtual slide generation method according to the present embodiment, the control of the motors 6a and 6b is open-loop control, and feedback control is not performed. However, accurate positioning to each imaging position is not performed, but positioning can be performed quickly as much as positioning accuracy is not required. On the other hand, since the image and the exact position where the image was acquired are stored in association with each other, as shown in FIG. 5, only by arranging the images according to the stored position, complicated image processing is performed. It is possible to easily generate a virtual slide without performing. That is, the time required for image processing can be shortened, and a virtual slide can be generated in a short time.

  Since the movement is completed in a short time, the cycle time for acquiring one enlarged image can be shortened, and the time required for image processing can be shortened, so that one imaging position can be changed to another imaging position. The image pasting process can be performed during the movement to. Therefore, there is an advantage that the time for generating a virtual slide by combining a plurality of enlarged images can be greatly reduced.

  Also, in this case, by setting each imaging position in the movement pattern so that the imaging ranges overlap as much as the error generation factor of the mechanical system is expected, a region that is not imaged due to rough positioning occurs. Can be prevented.

  In this embodiment, the motors 6a and 6b are controlled by open loop to prevent the positioning time from being extended by feedback. However, instead of this, feedback control of the motor may be performed. . Even in this case, the positioning accuracy can be lowered, and as a result, the positioning time can be shortened.

  In addition, the process of pasting the enlarged images is performed simply by arranging the images according to the positions stored in association with the acquired enlarged images, but instead, the enlarged images are converted by the following method. Image pattern recognition may be performed at the overlapping portion to correct the positional deviation between the enlarged images.

  That is, in the virtual slide image generation method according to the present embodiment in which the imaging position is roughly positioned, the width of the overlapping portion between the enlarged images G1 and G2 is relatively large compared to the case where positioning is performed with high accuracy. For this reason, enormous time is required for image processing when image pattern recognition is performed using the entire overlapping portion. Therefore, as shown in FIG. 6, using the coordinates of the center positions P1 and P2 of the enlarged images acquired by the sensors 7a and 7b, the numbers arranged at equal distances from the center positions P1 and P2 of the adjacent enlarged images. By performing image pattern recognition between pixel columns having a pixel width α, the time required for image processing can be shortened.

In the present embodiment, the control unit 3 moves the stage 5 in accordance with a preset movement pattern and stops it at each imaging position. Instead, the waiting time at each imaging position is previously set. May be set so that the movement to the next imaging position is automatically started after the elapse of the waiting time.
In this embodiment, the objective lens 8 is fixed and the stage 5 on which the specimen A is mounted is moved. Alternatively, the stage 5 may be fixed and the objective lens 8 may be moved. Further, both the objective lens 8 and the stage 5 may be relatively moved.

1 Virtual slide creation device 5 Stage 6a, 6b Motor (relative movement mechanism)
7a, 7b Sensor (position detection unit)
8 Objective Lens 9 Imaging Unit 10 Storage Unit 11 Image Processing Unit A Sample G1, G2 Enlarged Image P1, P2 Center S2 Movement Step S4 Detection Step, Imaging Step S7 Transmission Step (Storage Step)
S8 Image pasting step (image processing step)

Claims (7)

A stage with a specimen,
An objective lens that collects light from the specimen mounted on the stage;
A relative movement mechanism for relatively moving at least one of the stage and the objective lens in a direction intersecting the optical axis of the objective lens;
A position detection unit for acquiring relative position information between the stage and the objective lens by the relative movement mechanism;
An imaging unit that captures the light collected by the objective lens and obtains a partially enlarged image of the sample;
During the movement of at least one of the stage and the objective lens in a direction intersecting the optical axis of the objective lens, the enlarged image acquired by the imaging unit and the position detection unit at the time of acquisition of the enlarged image are detected. A storage unit for storing the relative position information of the stage and the objective lens in association with each other;
During the movement in the direction intersecting the optical axis of at least one of the stage and the objective lens, a plurality of enlarged images stored in the storage unit are stored in association with the enlarged image. And a virtual slide creation device including an image processing unit for pasting according to the relative position information. The relative movement mechanism moves at least one of the stage and the objective lens to a preset relative imaging position by open loop control, and outputs a movement completion signal at the end of movement,
The virtual slide creation apparatus according to claim 1, wherein the imaging unit acquires a partially enlarged image of a specimen according to a movement completion signal from the relative movement mechanism.   The virtual slide creation device according to claim 2, wherein the relative movement mechanism stores a movement pattern of at least one of the stage and the objective lens and moves the movement according to the movement pattern.   The virtual slide creation device according to claim 3, wherein the relative movement mechanism starts moving at least one of the stage and the objective lens from one relative imaging position to another relative imaging position by a trigger signal. . The adjacent imaging positions adjacent to each other are set to positions where enlarged images that overlap each other can be acquired,
The image processing unit performs image pattern recognition processing using luminance information of pixel rows arranged at equal distances from the center of the enlarged image based on relative position information stored in association with adjacent enlarged images. The virtual slide creation device according to any one of claims 1 to 4. A moving step of relatively moving at least one of the specimen and an imaging optical system that acquires a partially enlarged image of the specimen in a direction intersecting the optical axis of the imaging optical system;
A detection step of acquiring relative position information between the specimen and the imaging optical system;
An imaging step of acquiring an enlarged image of the specimen by the imaging optical system at the time when the relative position information is acquired in the detection step;
A storage step of storing the enlarged image acquired by the imaging step and the relative position information of the sample and the imaging optical system in association with each other during the moving step ; and
During said moving step, virtual slide generation method including a plurality of enlarged image imaging bonded Thus the relative position information stored in association with the the enlarged image the steps stored in said storing step. The moving step moves at least one of the specimen and the imaging optical system so as to overlap adjacent enlarged images;
The image processing step performs image pattern recognition processing using luminance information of pixel rows arranged at equal distances from the center of the enlarged image based on relative position information stored in association with adjacent enlarged images. The virtual slide creation method according to claim 6.

Images