JP2009063658A - Microscope system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily display an image even if an amount of data is extremely large. <P>SOLUTION: A connection/attribute creating section 164 creates first label information which is imparted to a plurality of pickup images picked up by respective several image sensors and includes at least the identifiers of the picked-up images and position information within the view field of an objective lens. A buffer 163 stores the picked-up images and the first label information while associated with each other. A connection/attribute creating section 166 creates second label information which is imparted to a one view field image when several stored picked-up images are connected as one view field image, i.e., an image of one field of view of the objective lens, and which includes at least the identifier of the one view field image and position information within an observation view field. A buffer 165 stores the one view field image and the second label information while associated with each other. This invention can be applied to a microscope system which has a microscope. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a microscope system, and more particularly to a microscope system that can acquire a wide-field and high-resolution specimen image.

  2. Description of the Related Art In recent years, attention has been focused on a technique that can digitize an image of a preparation to be observed with a microscope and display the image on an arbitrary monitor for observation.

In Patent Document 1, a digital camera is mounted on a normal microscope, a microscope image is acquired using a conventional objective lens while scanning an XY stage, and a plurality of these images are combined to obtain a large specimen image. A technique called “virtual slide” or “virtual microscopy” is disclosed.
US Patent Application Publication No. 2001/0050999

  However, in the conventional techniques including Patent Document 1, a large whole image is constructed while acquiring a captured image of a specimen image magnified by an objective lens with one image sensor, but when the amount of data becomes enormous There is a problem that it becomes difficult to display an image.

  Furthermore, in the present invention, the image of the specimen magnified by the objective lens is simultaneously picked up by a plurality of image pickup elements, so this problem becomes more remarkable.

  Specifically, the following two methods can be adopted as the final data storage form. That is, firstly, the method of saving all images as one file at the time of acquisition, and secondly, when saving an image for each acquisition unit, label information (identifier and position information) is also saved at the same time, This is a method of connecting images based on label information.

  If these methods are adopted, the former causes inconveniences such as inability to expand data when displaying an image when the amount of data becomes enormous. In the latter case, when the number of data divisions becomes enormous, inconveniences such as the time required for collecting label information at the time of image display and the image connecting process based on the label information become enormous. As described above, in the case of the conventional technique, if the amount of data for constructing the entire image becomes enormous, the image cannot be displayed easily.

  The present invention has been made in view of such a situation, and makes it possible to easily display an image even if the amount of data for constructing an entire image becomes enormous.

A microscope system according to one aspect of the present invention includes:
A microscope (for example, FIG. 2) having a stage (for example, the motorized XY stage 16 in FIG. 2) on which the specimen is placed and an objective lens (for example, the objective lens 12 in FIG. 2) are observed. In a microscope system having a large microscope 1) (for example, the microscope system of FIG. 13),
Imaging means (for example, the camera head 111 in FIG. 13) comprising a plurality of imaging elements provided so as to fall within the field of view of the objective lens;
1st label information given to a plurality of picked-up images picked up by each of the plurality of image pick-up elements, and includes at least an identifier of the picked-up image and position information in the field of view of the objective lens. First generation means for generating the label information (for example, the connection / attribute generation unit 164 in FIG. 13);
A first buffer (for example, buffer 163 in FIG. 13) that stores the captured image and the first label information in association with each other;
When a plurality of captured images stored in the first buffer are connected as one field image that is an image for one field of the objective lens, second label information to be given to the one field image, Second generation means (for example, the connection / attribute generation unit 166 in FIG. 13) that generates the second label information including at least the identifier of the one visual field image and position information in the observation visual field;
A second buffer (for example, a buffer 165 in FIG. 13) that stores the one-view image and the second label information in association with each other.

When an image for one field of view in the objective lens is picked up by the image pickup means, a first connection means for connecting the plurality of picked-up images stored in the first buffer as the one field-of-view image (for example, , The process of step S58 of FIG. 16 executed by the connection / attribute generation unit 164 of FIG.
A second connection for connecting a plurality of one-field images stored in the second buffer as specimen images that are images of the specimen when one field-of-view image for the observation field is stored in the second buffer; Means (for example, the process of step S64 of FIG. 17 executed by the connection / attribute generation unit 166 of FIG. 13).

The one field-of-view image has an area for connecting with another one field-of-view image adjacent when the sample image is connected;
The second connection means removes the region when connecting the plurality of one-field images as the specimen image (for example, the step of FIG. 17 executed by the connection / attribute generation unit 166 of FIG. 13). Process of S64).

A storage unit (for example, an image data storage unit 168 in FIG. 13) that stores the one field-of-view image from which the region stored in the second buffer is removed and the second label information in association with each other; Further prepare.

  Display means (for example, the display 116 in FIG. 13) is further provided for displaying the one field-of-view image corresponding to the second label information based on the stored second label information.

  In one aspect of the present invention, the first label information is provided to a plurality of captured images captured by each of the plurality of imaging elements, and includes at least an identifier of the captured image and position information in the field of view of the objective lens. The first label information included is generated, the captured image and the first label information are stored in association with each other, and the stored multiple captured images are images for one field of view of the objective lens. Second label information to be added to one field-of-view image, which includes at least an identifier of the one field-of-view image and position information in the observation field of view. The image and the second label information are stored in association with each other.

  According to an aspect of the present invention, management of divided and acquired images is facilitated, and accordingly, divided image collection and combination processing during image display is facilitated.

Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

  FIG. 1 is a schematic cross-sectional view showing the overall configuration of a large microscope.

  The large-sized microscope 1 is a microscope that observes a specimen by adjusting the distance between a stage on which a specimen such as a cell is placed and an objective lens to adjust the focus.

  The large microscope 1 includes an imaging unit 11, an objective lens 12, an AF (Auto Focus) unit 13, a frame / housing unit 14, a light source unit 15, an electric XY stage 16, a Z-axis driving unit 17, and a transmission illumination unit 18. Constructed to include.

  The imaging unit 11 includes a plurality of CCD (Charge Coupled Device) and CMOS (Complementary Metal Oxide Semiconductor) imaging elements, and a wide-field image of a specimen formed by the objective lens 12 having a high NA and a wide field of view. Take an image. Here, the number of image pickup devices mounted on the image pickup unit 11 is determined by the formation area of the image formed by the objective lens 12. For example, the imaging unit 11 includes 3 × 3, 3 × 2, 2 × 2, etc. imaging elements in the XY direction (horizontal direction) in the drawing according to the image formation area by the objective lens 12. Provided. In the present embodiment, in order to make the description easy to understand, it is assumed that 2 × 2 CCDs are mounted on the imaging unit 11.

  The imaging unit 11 can scan in the XY directions by an imaging unit drive system (not shown).

  The AF unit 13 adjusts the Z-axis direction (vertical direction (optical axis direction)) in the drawing by using a Z-axis drive unit 17 capable of adjusting the position in the Z direction with respect to the objective lens 12, and performs electric X- The sample is placed on the Y stage 16 (the cover glass placed on the sample on the slide) and is brought into focus. Although not shown, in the present embodiment, the imaging unit 11 is provided with a sensor, which autofocuses the sample image.

  The imaging unit 11, the objective lens 12, and the AF unit 13 are supported by a frame / housing unit 14.

  The electric XY stage 16 includes a drive system, a sensor, and the like (not shown), and moves the placed specimen in the XY direction. The specimens (preparations) are conveyed one by one to the electric XY stage 16 by a loader (not shown). The transmitted illumination unit 18 illuminates the specimen placed on the electric XY stage 16 with light from the light source unit 15 serving as a light source.

  The large microscope 1 is configured as described above.

  In such a large microscope 1, as shown in FIG. 2, specimens to be observed are placed on the electric XY stage 16, but as described above, the specimens are electrically driven XY one by one. It is conveyed to the stage 16. Thereafter, the AF unit 13 detects whether or not the conveyed sample is at a predetermined Z position, and the imaging unit 11 captures an image of the sample formed by the objective lens 12.

  In the present embodiment, a plurality of captured images captured by the imaging unit 11 (a plurality of CCDs) are connected (images are combined) to generate a sample image for one observation field. First, a method for connecting captured images in the present embodiment will be described with reference to FIGS. The operation described with reference to FIGS. 3 to 10 is an operation after focusing on the sample conveyed to the electric XY stage 16.

  First, a detailed configuration of the imaging unit 11 in FIG. 1 will be described with reference to FIG.

  As illustrated in FIG. 3, the imaging unit 11 includes an imaging unit 31 that includes four CCDs 41 to 44. For example, the CCDs 41 to 44 are arranged at predetermined intervals in consideration of margins based on errors such as a stage and a CCD, which are necessary when connecting a plurality of captured images. For example, a plurality of CCDs are arranged at intervals necessary for the package.

  FIG. 4 shows, for example, how the imaging unit 31 captures a field of view that can be observed with the objective lens 12 of FIG. That is, 4 × 4 squares are arranged in a lattice pattern inside the field of view 51 of the objective lens 12 in the upper diagram of FIG. 4, and these squares are CCD 41 to CCD 44 mounted on the imaging unit 31. It corresponds to either.

When the imaging unit 31 is in the initial position (hereinafter referred to as the first position), the CCD 41, the CCD 42, the CCD 43, and the CCD 44 are respectively 41 ₁ , 42 ₁ , 43 ₁ , 44 ₁ as shown in FIG. It is expressed. Next, when the imaging unit 31 moves from the first position to a position moved by 1 CCD in the X-axis direction (hereinafter referred to as a second position), the CCD 41 to CCD 44 are 41 ₂ , 42 ₂ , 43, respectively. ₂ and 44 ₂ .

When the imaging unit 31 moves from the second position to a position (hereinafter referred to as a third position) moved by −1 CCD in the X-axis direction and 1 CCD in the Y-axis direction, the CCDs 41 to 44 are , 41 ₃ , 42 ₃ , 43 ₃ , and 44 ₃ , respectively. Next, when the imaging unit 31 moves from the third position to a position moved by 1 CCD in the X-axis direction (hereinafter referred to as a fourth position), the CCDs 41 to 44 are respectively 41 ₄ , 42 ₄ , 43 ₄ , 44 ₄ .

That is, as shown in the lower diagram of FIG. 4, considering the case where the sample 52 is in the field of view 51 of the objective lens 12, the CCD 41 ₁ to the CCD 41 ₁ through the image pickup unit 31 are moved to the first position. the CCD 44 _1, a portion of the specimen 52 is imaged. Thereafter, the imaging unit 31 is moved from the first position in the order of the second position, the third position, and the fourth position, and the CCD 41 to the CCD 44 are moved by repeating the movement and imaging a total of four times. The captured image for one field of view of the objective lens 12 can be captured.

  Further, in the present embodiment, in order to acquire an image having high resolution over the entire specimen, it is necessary to acquire captured images for a plurality of visual fields. Therefore, while moving the visual field 51 of the objective lens 12, Imaging is performed. FIG. 5 is a schematic diagram showing how captured images for a plurality of fields of view are acquired.

  In FIG. 5, a specimen 61 is placed on a preparation 62, and the top is covered with a cover glass (not shown).

In FIG. 5, as shown in the upper diagram, using the motorized XY stage 16 of FIG. 1, the visual field 51 ₁ , the visual field 51 ₂ , the visual field 51 ₃ , the visual field 51 ₄ , the visual field 51 ₅ , the visual field 51 ₆ , field 51 _7, viewing 51 _8, by sequentially moving to the imaging position of the field 51 _9, as shown in the lower diagram, the sample image 71 is acquired.

That is, the sample image 71 was acquired 1 field image 71A obtained in field 51 _1, field 51 ₂ obtained 1-field image 71B, one visual field image 71C obtained by the field 51 _3, at a viewing 51 ₄ 1 field images 71D, 1 field image 71E obtained at a viewing 51 _5, 1 field image 71F obtained at a viewing 51 _6, 1 field image 71G obtained at a viewing 51 _7, 1 field image acquired by the field 51 ₈ 71H, and it consists of a total of 9 field of view of the image of one field image 71I obtained in field 51 _9.

  As described above, the sample image 71 is composed of images of nine fields of view. A method for capturing one field of view image for one field of view will be described as follows. That is, FIG. 6 is a schematic diagram illustrating a method for acquiring a captured image for one field of view.

FIG. 6 shows a change in the position of the imaging unit 31 in the visual field 51 ₁ . That is, the image pickup unit 31 moves in the XY directions, so that the first position (upper left diagram), the second position (upper right diagram), the third position (lower left diagram), the fourth position A captured image at the position (lower right figure) can be acquired.

In the first position, CCD 41 _1, CCD 42 _1, CCD 43 _1, and CCD 44 ₁ obtains a captured image of the different regions, obtaining four captured images. Then, moving from the first position to the second position, again, CCD 41 _2, CCD 42 _2, CCD 43 _2, and CCD 44 ₂ obtains a captured image of a region different than either respectively, four captured images Get.

Next, in the CCD 41 _3, CCD 42 _3, CCD 43 _3, and CCD 44 _3, 4 of captured images in the third position is obtained, CCD 41 _4, CCD 42 _4, CCD 43 _4, and the CCD 44 _4, the fourth position Four captured images are acquired. As a result, 16 captured images corresponding to one visual field in the visual field 51 ₁ are captured.

After that, as shown in FIG. 7, the electric field 51 of the objective lens 12 moves from the visual field 51 ₁ to the visual field 51 ₂ by moving the motorized XY stage 16. That is, FIG. 7 is a schematic diagram showing the acquisition of one field of view portion of the captured image in the next field 51 ₂ of after capturing a captured image of one field worth in the field of view 51 ₁ shown in FIG.

As shown in FIG. 7, in the visual field 51 ₂ , as in the visual field 51 ₁ , the imaging unit 31 is moved in the order of the first position, the second position, the third position, and the fourth position, By performing the imaging four times, a captured image for one field of view (a total of 16 captured images) is acquired. Then, also in the field of view 51 ₃ to field 51 _9, the imaging unit 31, a first position to be moved sequentially to the fourth position, respectively, captured images of one field of view worth is obtained.

  The 16 captured images acquired for each visual field are connected to the 16 captured images for each visual field to form one single visual field image. FIG. 8 is a schematic diagram illustrating a connection state of captured images for each visual field.

As shown on the left side of FIG. 8, 1 fields of view of the captured image in the field of view 51 _1, CCD 41 to CCD44 of the imaging unit 31, considering the overlapping portion for bonding the aspect of the captured image (margin) Are partly overlapped with adjacent captured images.

That is, in the first position of the field of view 51 _1, an image captured by the CCD 41 _1, CCD 42 _1, CCD 43 _1, CCD 44 _1, captured images 71A _11, captured image 71A _21, captured image 71A _31, the captured image 71A ₄₁ Then, these captured images have a margin for pasting with other captured images adjacent vertically and horizontally.

Similarly Further, in the second position, CCD 41 ₂ or CCD 44 ₂ captured image 71A ₁₂ to the captured image 71A ₄₂ captured by, in a third position, CCD 41 ₃ to CCD 44 captured images 71A ₁₃ through the imaging is imaged by ₃ image 71A _43, in the fourth position, the captured image 71A ₁₄ to the captured image 71A ₄₄ captured by the CCD 41 ₄ to CCD 44 _4, a margin for bonding with other captured image adjacent to the vertical and horizontal Yes.

Then, the captured image 71A ₁₁ to the captured image 71A ₄₄ have their margin, by being connected to the other captured images adjacent considering its margin, as shown on the right side of FIG. 8 One visual field image 71A for one visual field is obtained.

Portion Specifically, in the field of view 51 _1, when connecting the image, for example, first, focusing on the captured image 71A _11, which for the captured image 71A _11, overlaps the captured image 71A ₁₂ located laterally (Margin) is removed. Next, with respect to the captured image 71A _12, the margin of the captured image 71A ₂₁ located laterally is removed, further, with respect to the captured image 71A _21, the margin of the captured image 71A ₂₂ located laterally Removed. In this state, the row (first row) in the visual field 51 ₁ is connected.

Then, the captured image 71A _13, the margin of the captured image 71A ₁₁ a captured image 71A ₁₂ positioned thereon is removed. Then, the captured image 71A _14, the margin of the captured image 71A ₁₂ a captured image 71A ₂₁ positioned on are removed, but still overlap the captured image 71A ₁₃ located next to the captured image 71A ₁₄ Removed. Next, the margin of the captured image 71A _23, the captured image 71A ₁₄ that the margin between the captured image 71A ₂₁ a captured image 71A ₂₂ positioned on is removed, further located next to the captured image 71A ₂₃ Is removed. Then, the captured image 71A _24, the margin of the captured image 71A ₂₂ is removed which is located thereon, further margin of the captured image 71A ₂₃ located next to the captured image 71A ₂₄ are removed. In this state, two rows in the visual field 51 ₁ (first row and second row) are connected.

Then, the same processing as the processing in the first and second rows in the visual field 51 ₁ is also performed in the third and fourth rows, so that the captured images 71A _{11 to} 71A ₄₄ for one visual field are connected. A one-view image 71A as shown on the right side of FIG. 8 is generated.

Thereafter, as shown in FIG. 9, the same processing is performed in the next visual field 51 ₂ after the visual field 51 ₁ , whereby the captured image 71B _{11 to the} captured image 71B ₁₄ , the captured image 71B _{21 to the} captured image 71B ₂₄ , and the captured image. 71B _{31 to} captured image 71B ₃₄ and captured image 71B _{41 to} captured image 71B ₄₄ are connected to generate a one-view image 71B.

Furthermore, since the same processing is performed in the next field 51 ₃ to field 51 ₉ of the field 51 _2, 1-field image 71C to one visual field image 71I is generated.

  The 1-view image 71A to the 1-view image 71I have an overlap necessary for bonding with adjacent 1-view images, and one 1-view image is generated from the 16 captured images described above. By applying the same processing as the principle, one specimen image 71 is obtained from nine one-field images as shown in FIG.

As described above, in this embodiment, 16 pieces of captured image one for one field of view image by connecting (e.g., captured images 71A ₁₁ to the captured image 71A ₄₄₎ (e.g., 1-field image 71A) is generated, Furthermore, nine one-field images (for example, one field image 71A to one field image 71I) are connected to generate one final sample image (for example, sample image 71).

  Note that when the captured image is acquired, the imaging unit 31 sequentially moves from the first position to the fourth position. As the movement method, the imaging unit 11 is driven by an imaging unit drive system (not shown). May be moved in the XY direction, the electric XY stage 16 may be moved, or both of them may be moved.

  By the way, the specimen image 71 generated in this way is finally displayed on a display, but in order to realize it, for example, the following configuration is required. That is, in addition to the large microscope 1 described above, as shown in FIGS. 11 and 12, a camera head 111, a camera control unit 112, a microscope control unit 113, a pre-imaging unit 114, and a host corresponding to the imaging unit 11 of FIG. The present invention is realized by a system (hereinafter referred to as a microscope system) configured to include the PC 115, the display 116, and the controller 117. Next, such a microscope system will be described with reference to FIGS. 11 to 20.

  First, the configuration of the camera head 111, camera control unit 112, microscope control unit 113, pre-imaging unit 114, host PC 115, display 116, and controller 117 will be described with reference to the block diagrams of FIGS.

  If explaining in random order, in FIG. 11, the microscope control part 113 is connected to each part of a microscope system, such as the drive system and autofocus in the large sized microscope 1, for example, and controls these.

  The pre-imaging unit 114 is composed of, for example, a CCD or the like, captures the entire specimen immediately before entering the large microscope 1 at a low magnification, and supplies the captured pre-captured image to the microscope control unit 113. The microscope control unit 113 outputs the pre-captured image from the pre-imaging unit 114 to, for example, the host PC 115 in FIG.

  In FIG. 12, the host PC 15 causes the display 116 to display a pre-captured image input from the microscope control unit 113. Therefore, the user operates the controller 117 for control while viewing the image of the specimen immediately before entering the large microscope 1 (hereinafter referred to as a prior specimen image) displayed on the display 116, and generally observes it. The position and range to be observed can be indicated on the pre-sample image. That is, the user can confirm the pre-sample image before observing the sample image with the large microscope 1, and can specify the range of the sample to be observed.

  Thereafter, position information on the pre-sample image instructed by the user's operation of the controller 117 is output to the microscope control unit 113 via the host PC 115. The microscope control unit 113 controls each unit of the microscope system so that a sample image at a position corresponding to the position information on the prior sample image is acquired and displayed on the display 116.

  The camera head 111 has CCD 41 to CCD 44 corresponding to FIG. 3 as a plurality of imaging elements, and outputs image signals captured by these imaging elements to the camera control unit 112.

  The camera control unit 112 is provided with sensor processing units 121 to 124 corresponding to the CCDs 41 to 44, and image signals captured by the corresponding imaging elements are input thereto. The sensor processing unit 121 to the sensor processing unit 124 perform predetermined image processing on the input image signal, and supply captured image data obtained thereby to the multiplexing processing unit 125. At that time, image processing performed by the sensor processing unit 121 to the sensor processing unit 124 is performed in parallel.

  In this embodiment, since the camera head 111 composed of four CCDs is described as an example, four sensor processing units are provided. However, the number of sensor processing units corresponds to the number of CCDs. Provided.

  The multiplexing processing unit 125 multiplexes the captured image data supplied from each of the sensor processing unit 121 to the sensor processing unit 124 and outputs the multiplexed data to the host PC 115.

  The host PC 115 is composed of, for example, a personal computer or a dedicated server. In the host PC 115, a CPU (Central Processing Unit) 141 executes various processes according to programs loaded from the hard disk drive 143 to the main memory 142. The main memory 142 also appropriately stores data necessary for the CPU 141 to execute various processes.

  The CPU 141, the main memory 142, and the hard disk drive 143 are connected to each other via a bus 144. A communication interface 145 and a graphic controller 146 are also connected to the bus 144.

  The communication interface 145 receives captured image data input from the camera control unit 112 according to the control of the CPU 141 and supplies the captured image data to the main memory 142. The CPU 141 performs predetermined processing on the captured image data temporarily stored in the main memory 142, and the image data obtained thereby is finally stored in the hard disk drive 143.

  The graphic controller 146 performs predetermined image processing on the image data according to the control of the CPU 141, and displays an image corresponding to the image data obtained thereby on the display 116.

  Further, a drive 147 is connected to the bus 144 as necessary, and a removable medium 148 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is necessary. Is installed in the hard disk drive 143 accordingly. Further, the communication interface 145 may acquire a predetermined program by communicating with an external device via a communication network, the Internet, other networks, or a communication medium, and may record the program in the hard disk drive 143.

  In addition, the structure of each part which comprises a microscope system is not limited to the example of FIG. 11 and FIG. 12, What is necessary is just to have at least the functional structure of FIG.

  FIG. 13 is a block diagram illustrating an example of a functional configuration of each apparatus constituting the microscope system of FIGS. 11 and 12.

  In FIG. 13, portions corresponding to those in FIGS. 11 and 12 are denoted by the same reference numerals, and the description thereof will be repeated, and will be omitted as appropriate.

  The camera control unit 112 is configured to include a raw data conversion unit 151, an image generation unit 152, an image correction unit 153, and a serialization unit 154. In the camera control unit 112, the raw data conversion unit 151 to serialization unit 154 perform predetermined image processing on the image signal input from the camera head 111, and output captured image data obtained thereby to the host PC 115. Details of processing performed by the raw data conversion unit 151 to serialization unit 154 in the camera control unit 112 will be described with reference to a flowchart of FIG.

  The raw data conversion unit 151 through the serialization unit 154 correspond to, for example, the sensor processing unit 121 through the sensor processing unit 124 in FIG.

  The host PC 115 performs predetermined image processing on the captured image data input from the camera control unit 112 and causes the display 116 to display an image corresponding to the image data obtained thereby.

  The host PC 115 is configured to include a control unit 161, a reception buffer 162, a buffer 163, a connection / attribute generation unit 164, a buffer 165, a connection / attribute generation unit 166, an image compression unit 167, and an image data storage unit 168.

  In this embodiment, since the host PC 115 has a hardware configuration as shown in FIG. 12, the functions of the connection / attribute generation unit 164, the connection / attribute generation unit 166, and the image compression unit 167 are as follows. For example, it is executed by the CPU 141 that executes a program recorded in the hard disk drive 143. Further, the control unit 161 corresponds to, for example, the CPU 141 in FIG. 12, and the reception buffer 162 corresponds to, for example, a reception buffer (not shown) provided in the communication interface 145 in FIG. Further, the buffer 163 and the buffer 165 correspond to, for example, a buffer (not shown) or the main memory 142 provided in the host PC 115, and the image data storage unit 168 corresponds to, for example, the hard disk drive 143 in FIG.

  However, the connection / attribute generation unit 164, the connection / attribute generation unit 166, and the image compression unit 167 can be configured as hardware, or can be configured as a combination of software and hardware.

  Details of processing performed by the control unit 161 to the image data storage unit 168 in the host PC 115 will be described with reference to flowcharts of FIGS. 16 and 17 described later.

  The microscope system is configured as described above.

  By the way, when the captured image is acquired, if the shutter time in each CCD is different, the brightness of the captured image is also different, so that it may be difficult to see when an image of one field of view is constructed. Therefore, it is necessary to obtain a shutter time with a predetermined reference brightness and acquire all images with the same shutter time.

  Next, a process for setting an exposure time (hereinafter referred to as an exposure time setting process) executed by the microscope control unit 113 will be described with reference to the flowchart of FIG.

  In step S <b> 11, the microscope control unit 113 determines an imaging start location based on observation range information designated on the pre-sample image designated by the operation of the controller 117 by the user.

In step S12, the microscope control unit 113, in response to the determined imaging start location, by driving the electric X-Y stage 16 moves the sample to the first field of view of the example field 51 _1, and the like. At this time, the microscope control unit 113 controls the imaging unit drive system to position the CCD 41 to CCD 44 at the first position shown in FIG.

  In step S <b> 13, the microscope control unit 113 acquires exposure information of each CCD in the first visual field from the camera control unit 112.

  In step S14, the microscope control unit 113 selects the longest exposure time among the exposure times of each CCD based on the obtained exposure information of each CCD, sets the exposure time as the exposure time of each CCD, The exposure time setting process ends.

  As described above, the optimum CCD exposure time is selected from the exposure times of the plurality of CCDs, and the specimen is imaged by the plurality of CCDs using the exposure times. That is, in the present embodiment, a plurality of CCDs are used, and each CCD is arranged at intervals, so that at least one of the plurality of CCDs is sufficiently filled with a sample. The CCD exposure time is used for immobilization. That is, for the exposure time as the imaging condition, a parameter obtained with the best imaging element among a plurality of imaging elements is used. This parameter is used without change until the entire observation range is shot.

  As a result, the entire observation range has the same brightness, and even if the images acquired by the CCDs are joined to form a single image, a clear image with no conspicuous connection is obtained. In addition, since the CCD is in the first position in advance, imaging can be started immediately after setting the exposure time. Thereby, the positioning operation is reduced and the throughput is improved.

  In the present embodiment, it has been described that the exposure information of each CCD is acquired in step S13 and step S14, and the exposure time of each CCD is set based on the exposure information. Obtain the location of the sample that exists within the range specified by the user in the sample image by a predetermined image analysis, identify the CCD at the position where the sample can be imaged, and specify the exposure time of the specified CCD for each It may be set as the CCD exposure time.

  In the present embodiment, the example in which the longest exposure time is selected so that the captured image does not become dark and is not too bright has been described as the exposure time selected in step S14. Not only the exposure time but also an optimal time is appropriately selected such as, for example, a time obtained by averaging the exposure times of the CCDs, or a time obtained by averaging two long exposure times. It is also preferable to change the exposure time determination method depending on whether the specimen is illuminated by epi-illumination or transmission illumination. In the above-described example, the entire observation range is described as being imaged with the same exposure time. However, the present embodiment is not limited to this, and every time the field of view of the objective lens 12 changes, the flowchart illustrated in FIG. Processing may be performed to reset the parameters.

  Next, a process of generating a captured image (hereinafter referred to as a captured image generation process) executed by the camera control unit 112 will be described with reference to the flowchart of FIG.

  In step S31, the raw data conversion unit 151 performs predetermined conversion on image data represented by a digital signal obtained by A / D (Analog / Digital) conversion of an analog image signal supplied from the camera head 111. By performing processing, it is converted into Raw data and supplied to the image generation unit 152. Here, the conversion processing is, for example, processing such as pixel defect correction of an image sensor or gain adjustment for adjusting brightness.

  The A / D conversion process may be performed by the camera head 111 instead of the raw data converter 151. In this case, since the raw data conversion unit 151 is supplied with image data from the camera head 111, the raw data conversion unit 151 executes only conversion processing on the image data.

  In step S <b> 32, the image generation unit 152 performs processing such as pixel interpolation on the raw data supplied from the raw data conversion unit 151, generates RGB image data, and supplies the RGB image data to the image correction unit 153.

  In step S <b> 33, the image correction unit 153 performs image correction processing on the RGB image data supplied from the image generation unit 152 by shading, distortion, and the like, and uses the corrected captured image data obtained thereby. This is supplied to the serialization unit 154.

  In step S34, the serialization unit 154 serializes the corrected captured image data supplied from the image correction unit 153, outputs the serialized image data to the host PC 115, and the captured image generation process by the camera controller 112 ends.

  As described above, in the camera control unit 112, captured image data captured by a plurality of CCDs is generated and output to the host PC 115.

  Next, a process for connecting captured images (hereinafter referred to as an image connection process) executed by the host PC 115 will be described with reference to the flowcharts of FIGS. 16 and 17.

  The captured image data output from the camera control unit 112 is deserialized by the control unit 161 before the reception buffer 162. Subsequently, in step S51, the reception buffer 162 stores the deserialized captured image data as reception data.

  In step S52, the control unit 161 determines whether or not the reception data stored in the reception buffer 162 has been captured image data for one CCD.

  If it is determined in step S52 that the captured image data for one CCD element is not obtained, the process returns to step S51, and the processes in steps S51 and S52 are repeated until the captured image data for one CCD is stored.

  Thereafter, when it is determined in step S52 that the captured image data for one CCD has been stored, the control unit 161 acquires the captured image data for one CCD stored in the reception buffer 162 in step S53, and in step S54. Then, a predetermined coordinate conversion process required for each CCD is performed.

  Here, the coordinate conversion process refers to, for example, a process of rotating image data or correcting enlargement / reduction. In step S55, the control unit 161 stores the captured image data subjected to the coordinate conversion processing in the buffer 163.

  The received data for one CCD determined in step S52 is determined for each CCD 41 to CCD44. That is, in the processing from step S53 to step S55 after step S52, the captured image data acquired for each CCD is processed in parallel.

  In step S <b> 56, the connection / attribute generation unit 164 refers to the image data stored in the buffer 163 and determines whether image data for one field of view has been acquired. If it is determined in step S56 that the image data for one field of view has not been acquired, the process returns to step S51, and the processing in steps S51 to S56 is repeated until the image data for one field of view is stored.

  Thereafter, when it is determined in step S56 that image data for one field of view has been acquired, the connection / attribute generation unit 164 acquires captured image data for one field of view stored in the buffer 163 in step S57, In step S58, image connection processing is performed on the captured image data for one field of view.

  Here, the image connection processing executed by the connection / attribute generation unit 164 determines a connection position necessary for connecting a captured image of one visual field region, and further connects the captured image by removing margins. This is a process of generating a single visual field image from a plurality of captured images.

  In the present embodiment, data is taken into the buffer 163 in order according to the imaging position, so that it can be seen which image data and which image data are adjacent in the taken order. It has become.

As described above, for example, in the visual field 51 ₁ , the captured images 71A _{11 to} 71A ₁₄ , the captured images 71A _{21 to} 71A ₂₄ , and the captured images are captured by the CCD 41 to CCD 44 four times. 71A _{31 to} captured image 71A ₃₄ and captured image 71A _{41 to} captured image 71A ₄₄ are acquired. Captured image 71A _12, since the captured image 71A ₁₁ to the left is present, the margin on the left, which is determined as a connection position is removed, is connected to the captured image 71A _11. That is, the connection / attribute generation unit 164, that the same process is repeatedly performed for each captured image, from the captured image 71A ₁₁ to the captured image 71A ₄₄ stored in the buffer 163, margin is removed 1 One single-view image 71A is generated.

  Returning to the flowchart of FIG. 16, in step S <b> 59, the buffer 165 stores one-field-of-view image data obtained by performing the image connection process in accordance with the control of the control unit 161.

  The display 116 acquires the one field-of-view image data stored in the buffer 165 in step S60 according to the control of the control unit 161, and displays the one field-of-view image corresponding to the acquired one field-of-view image data in step S61. Thereby, the user can confirm one visual field image for one visual field sequentially.

  That is, in the present embodiment, since the buffer 163 for constructing an image for one field of view and the buffer 165 for constructing an image for the entire observation region are taken, the acquisition status is acquired during image acquisition. It can be displayed easily.

  In step S62, the connection / attribute generation unit 166 refers to the 1 field-of-view image data stored in the buffer 165 and determines whether connection is necessary for the stored 1 field-of-view image data.

  When it is determined in step S62 that it is necessary to connect one field-of-view image data, the connection / attribute generation unit 166 acquires the one field-of-view image data stored in the buffer 165 in step S63, and in step S64, An image connection process is performed on the acquired one-field-of-view image data.

  Here, the image connection process executed by the connection / attribute generation unit 166 determines a connection position necessary for connection of one field-of-view image, and further removes a margin to connect one field-of-view image. This is a process for generating a specimen image from a single visual field image.

  As described above, for example, in the specimen image 71, since the one-view image 71B has the one-view image 71A on the left side, the left margin determined as the connection position is removed and connected to the one-view image 71A. .

  The image connection process performed by the connection / attribute generation unit 164 (the process of step S58) and the image connection process performed by the connection / attribute generation unit 166 (the process of step S64) are basically the same algorithm. The same thing can be used repeatedly. As a result, the speed can be increased by a combination of simple processes, and the load on the CPU can be reduced.

  In step S65, the connection / attribute generation unit 166 generates label information corresponding to one visual field image. Here, the label information is information that is given to various images and indicates, for example, attributes and positions.

  In step S <b> 66, the image compression unit 167 compresses the single-view image data subjected to the image connection processing by a predetermined method such as JPEG (Joint Photographic Experts Group) compression, and stores the compressed data in the image data storage unit 168. At this time, label information corresponding to one field-of-view image data is also stored. Thereby, the one-view image data stored in the image data storage unit 168 can be displayed on the display 116 based on the label information.

  In the present embodiment, description will be made on the assumption that one field-of-view image data is stored together with label information. However, only final specimen image data may be stored without storing one field-of-view image data.

  On the other hand, if it is determined in step S62 that connection is not necessary, steps S63 to S66 are skipped, and the process proceeds to step S67.

  In step S <b> 67, the connection / attribute generation unit 166 determines whether to use one field-of-view image data for subsequent connection.

  If it is determined in step S67 that it will be used for the subsequent connection, in step S68, the buffer 165 stores the one-field-of-view image data subjected to the image connection process. On the other hand, if it is determined in step S67 that it will not be used for the subsequent connection, the process of step S68 is skipped, and the process proceeds to step S69.

  In step S69, it is determined whether or not the final sample image is completed, and the above-described processing in steps S51 to S69 is repeated until the sample image is completed. That is, by performing the processing from step S51 to step S69, for example, one field-of-view image to which a plurality of captured images are connected is generated, and further, a sample image to which a plurality of one-field images are connected is generated.

  Thereafter, when it is determined in step S69 that the final sample image is completed, in step S70, the image compression unit 167 converts the final sample image data stored in the buffer 165 into, for example, JPEG compression or the like. The image data is compressed and stored in the image data storage unit 168, and the image connection process ends.

  As described above, in the present embodiment, the processing efficiency can be improved by performing image connection processing for each visual field after photographing one visual field.

  In addition, by providing two buffers 163 and 165 as buffers for storing image data, the connection / attribute generation unit 164 performs image connection processing (step S58 processing) and the connection / attribute generation unit 166. The image connection process to be performed (the process of step S64) can be performed in parallel, and the processing time can be shortened.

  Next, the data structure of the image data and the details of the buffer for storing the image data in the image connection process executed by the host PC 115 will be described with reference to FIGS.

  FIG. 18 is a schematic diagram illustrating an example of acquiring an image in one visual field region.

  In FIG. 18, the diagram on the left side is a schematic diagram showing how a single visual field image, which is an image of the single visual field area, is acquired by shifting four image sensors four times in the single visual field area.

As illustrated in FIG. 18, the CCD 41 to the CCD 44 perform the movement and imaging of the CCD four times in the field of view 51 ₁ , so that the captured images 71A _{11 to} 71A ₁₄ , the captured images 71A _{21 to} 71A ₂₄ , captured image 71A ₃₁ to the captured image 71A _34, acquires a captured image 71A ₄₁ to the captured image 71A _44. These captured images are output to the host PC 115 and stored by the buffer 163 for each CCD.

At this time, in the buffer 163, the captured image acquired for each imaging is stored as shown in FIG. That is, for example, captured image 71A ₁₁ acquired by CCD 41 ₁ is stored in the leftmost buffer (hereinafter, referred to as buffer 163 _1).

In addition, the connection / attribute generation unit 164 generates label information h _{A11 that} is given to the captured image 71A ₁₁ and includes an identifier, position information in the field of view of the objective lens 12, and the like. The connection / attribute generation unit 164 stores the generated label information h _A11 in the buffer 163 in association with the captured image 71A ₁₁ .

Similarly, in the first imaging, the captured image 71A ₂₁ and label information h _A21 is a second buffer from the left in FIG. 19 are stored (hereinafter, referred to as buffer 163 _2), the captured image 71A ₃₁ and label information h _A31 is left third buffer from Figure 19 are stored (hereinafter, referred to as buffer 163 _3), the captured image 71A ₄₁ and label information h _A41 is, rightmost buffer 19 (hereinafter, a buffer 163 ₄ Stored).

Subsequently, in the second, third, and fourth imaging, the captured image and the label information are stored in the buffer 163 as in the first imaging. When the imaging of four times is performed, as shown in FIG. 19, the buffer 163 _1, captured images 71A ₁₁ to the captured image 71A ₁₄ obtained by CCD41 are sequentially stored together with the corresponding label information .

Similarly, the captured image 71A _{21 to the} captured image 71A ₂₄ by the CCD 42 is stored in the buffer 163 ₂ , the captured image 71A _{31 to the} captured image 71A ₃₄ by the CCD 43 is stored in the buffer 163 ₃ , and the captured image by the CCD 44 is stored in the buffer 163 _4. 71A _{41 to} 71A ₄₄ are sequentially stored together with corresponding label information.

  Thereafter, the connection / attribute generation unit 164 performs the image connection process on the captured image stored in the buffer 163 because the captured image for one field of view is aligned in the buffer 163 after four times of imaging. The one field-of-view image 71 </ b> A is stored in the buffer 165.

  FIG. 20 is a schematic diagram showing a one-field image stored in the buffer 165. As shown in FIG. For example, the first-view image 71 </ b> A constructed first is stored in the lowermost level, which is the first storage position of the buffer 165.

In addition, the connection / attribute generation unit 166 generates label information h _{A that} is given to the one visual field image 71A and includes an identifier, position information in the observation visual field, and the like. The connection / attribute generation unit 166 stores the generated label information h _A in the buffer 165 in association with the one visual field image 71A.

Thus, by connecting the respective captured images 71A ₁₁ to the captured image 71A _44, when generating the 1-field picture 71A, a plurality granted on their captured image label information (in this case, the captured image It is possible to combine 16 pieces of label information equal to the number into one piece of label information of one visual field image. Thereby, the data amount of label information can be reduced.

Further, the image compression unit 167, the 1-field image 71A stored in the buffer 165 is compressed by the JPEG compression or the like, is stored in the image data storage unit 168 together with the label information h _A. Thereby, when displaying one field-of-view image on display 116, it becomes possible to display one field-of-view image quickly by referring to the corresponding label information.

At this time, stored in the buffer 163, the captured image 71A ₁₁ to the captured image 71A ₄₄ as part Connected for one field of view image 71A is erased.

Then, the processing performed by the field of view 51 ₁ described above is, that also sequentially repeated in field 51 ₂ to field 51 _9, the buffer 163, as shown on the right side of FIG. 18, one field of view image 71B to one field of view Captured images for constructing the image 71I are sequentially stored.

The connection / attribute generation unit 164 performs image connection processing on the captured images sequentially stored in the buffer 163 when the captured images for one field of view are prepared, and the one field-of-view image obtained thereby is stored in the buffer 165. To store. As a result, as shown in FIG. 20, the buffer 165 stores the one-view image 71B constructed next to the one-view image 71A stored at the lowest stage of the first storage position. . Further, the label information h _B is given to the one-field-of-view image 71B, similarly to the one-field-of-view image 71A.

  After that, the buffer 165 stores the one-view image 71C to the one-view image 71I one by one in order. The timing for deleting the stored one-view image is as follows, for example. become. That is, the one-view image shown in FIG. 20 is deleted when the connection processing of all adjacent target images is completed. For example, as shown in the right side of FIG. 18, the 1-view image 71A is adjacent to the 1-view image 71B and the 1-view image 71D, and thus is deleted when the connection process of the 1-view image 71B and the 1-view image 71D is completed. The

  Then, the connection / attribute generation unit 166 connects each of the nine 1-view images of the 1-view image 71A to the 1-view image 71I to obtain one sample image 71.

  In the present embodiment, the label information is given for each visual field unit. However, for example, the label information may be given for every other unit such as a range designated by the user.

  As described above, in the present embodiment, after the captured image acquired for each CCD is subjected to image processing that can be performed independently, label information is added to the captured image obtained thereby. Keep buffering. Thereafter, when captured images necessary for constructing an image for one field of view are prepared, image connection processing of these captured images is performed, and label information is also integrated.

  By performing this series of processing, the subsequent processing becomes a problem of division or integration of adjacent images, and the data storage unit is based on this, and further considering the performance of the host PC 115, the adjacent processing is performed sequentially. We will consider connecting multiple images.

  In this embodiment, since the order of the imaging positions for each field of view is determined, by storing the data of the captured images in a predetermined buffer area in a predetermined order according to the order input to the buffer, Accurate label information can be added. However, when the order of the imaging positions of one field of view is changed, a command for instructing the CPU 141 to change the correspondence between the order of input to the buffer and the buffer area according to the change of the order may be given. . In particular, in setting the exposure time, the imaging unit 31 is moved under the control of the microscope control unit 113 so as to be close to the center of the observation range in order to improve the probability that any CCD is located in the imaging range of the specimen. It may be better to set the exposure time. When this is done, it is better to change the imaging order of each position in one field of view so that imaging can be performed immediately after setting the exposure time.

  According to the present invention, since it is a system that acquires a specimen image magnified by a wide-field objective lens using a plurality of image sensors, a stable virtual image can be obtained as compared with a system that has conventionally been acquired by one image sensor. A slide image can be obtained.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  This recording medium is distributed to provide a program to users separately from a computer, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk (CD-ROM (Compact Disc-Read Only Memory). ), DVD (Digital Versatile Disc)), magneto-optical disk (including MD (Mini-Disk) (trademark)), or the removable media 148 of FIG. The hard disk drive 143 of FIG. 12 in which a program is recorded, which is provided to the user in a state of being pre-installed in the computer, and a ROM (Read Only Memory) (not shown) are configured.

  The program for executing the above-described series of processing is installed in a computer via a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via an interface such as a router or a modem as necessary. You may be made to do.

  In the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

It is a schematic sectional drawing which shows the whole structure of a large sized microscope. It is a figure which shows the principal part which images a microscope image. It is a figure which shows the detailed structure of an imaging part. It is a schematic diagram which shows the visual field of an objective lens. It is a schematic diagram which shows the mode of the movement of the visual field of an objective lens. It is a schematic diagram which shows the mode of the change of the position of the imaging unit in the visual field of an objective lens. It is a schematic diagram which shows the mode of the movement from a certain visual field to the next visual field. It is a schematic diagram which shows the mode of the connection of the captured image for every visual field. It is a schematic diagram which shows the mode of the connection of the captured image for every visual field. It is a schematic diagram which shows a mode that 1 visual field image is connected and a sample image is produced | generated. It is a block diagram which shows the example of a structure of a microscope system. It is a block diagram which shows the example of a structure of a microscope system. It is a block diagram which shows the example of a functional structure of each apparatus which comprises a microscope system. It is a flowchart explaining an exposure time setting process. It is a flowchart explaining a captured image generation process. It is a flowchart explaining an image connection process. It is a flowchart explaining an image connection process. It is a schematic diagram which shows the example of acquisition of the captured image in 1 visual field of an objective lens. It is a schematic diagram showing an example of a buffer that holds data for each CCD and its data structure. It is a schematic diagram which shows the example of the buffer which hold | maintains data for every visual field image, and its data structure.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Large microscope, 11 Imaging part, 12 Objective lens, 13 AF unit, 14 Frame / housing part, 15 Light source part, 16 Electric XY stage, 17 Z-axis drive part, 18 Transmission illumination unit, 31 Imaging unit, 41 thru | or 41 thru | or 44 CCD, 111 camera head, 112 camera control unit, 113 microscope control unit, 114 pre-imaging unit, 115 host PC, 116 display, 117 controller, 121 to 124 sensor processing unit, 125 multiplexing processing unit, 141 CPU, 142 main Memory, 143 hard disk drive, 145 communication interface, 146 graphic controller, 151 Raw data conversion unit, 152 image generation unit, 153 image correction unit, 154 serialization unit, 161 control unit, 162 reception buffer, 163 buffer, 164 connection / attribute generation unit, 165 buffer, 166 connection / attribute generation unit, 167 image compression unit, 168 image data storage unit

Claims (5)

In a microscope system having a stage on which a specimen is placed and an objective lens, and having a microscope for observing the specimen,
Imaging means comprising a plurality of imaging elements provided to be within the field of view of the objective lens;
1st label information given to a plurality of picked-up images picked up by each of the plurality of image pick-up elements, and includes at least an identifier of the picked-up image and position information in the field of view of the objective lens. First generation means for generating the label information of
A first buffer for storing the captured image and the first label information in association with each other;
When a plurality of captured images stored in the first buffer are connected as one field image that is an image for one field of the objective lens, second label information to be given to the one field image, Second generating means for generating the second label information including at least the identifier of the one visual field image and position information within the observation visual field;
A microscope system comprising: a second buffer that stores the one-field image and the second label information in association with each other. A first connection unit that connects the plurality of captured images stored in the first buffer as the one field-of-view image when an image for one field of view of the objective lens is captured by the imaging unit;
A second connection for connecting a plurality of one-field images stored in the second buffer as specimen images that are images of the specimen when one field-of-view image for the observation field is stored in the second buffer; The microscope system according to claim 1, further comprising means. The one field-of-view image has an area for connecting with another one field-of-view image adjacent when the sample image is connected;
The microscope system according to claim 2, wherein the second connection unit removes the region when the plurality of one-field images are connected as the specimen image. The microscope system according to claim 3, further comprising: a storage unit that stores the first field image from which the region stored in the second buffer is removed and the second label information in association with each other. The microscope system according to claim 4, further comprising display means for displaying the one visual field image corresponding to the second label information based on the stored second label information.

Images