JP6962725B2 - Observation device and observation method - Google Patents

Description

The present invention relates to an observation device and an observation method.

Conventionally, a scanning laser microscope or a camera is known as an observation device for acquiring an image of a sample (see, for example, Patent Document 1). In these observation devices such as scanning laser microscopes and cameras, there is a universal demand for high-resolution and high-speed acquisition of sample images.

Japanese Unexamined Patent Publication No. 2011-090248

However, in a conventional scanning laser microscope, it is possible to capture a high-speed phenomenon of a sample by reducing the number of pixels to a predetermined number or less, but if the number of pixels is reduced to a predetermined number or less, the resolution becomes insufficient and the resolution becomes insufficient. There is an inconvenience that the details of a high-speed phenomenon cannot be analyzed. Further, although the camera can acquire the sample image at high speed, it has a disadvantage that the resolution is low. It is also possible to achieve high resolution by synthesizing a plurality of images captured by shifting the pixels (pixel shift) in the camera, but in this case, since it is necessary to shoot multiple times, the sample speed is high. There is a problem that the phenomenon cannot be captured.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an observation device and an observation method capable of observing a sample at both high resolution and high speed.

In order to achieve the above object, the present invention provides the following means.
A first aspect of the present invention is a resolution setting unit that sets a first resolution for creating a high-resolution image and a second resolution lower than the first resolution, and the first resolution set by the resolution setting unit. A high-resolution dictionary consisting of a collection of a plurality of high-resolution base images that serve as a base for constructing a desired high-resolution image by linear coupling is downsampled according to the ratio of the second resolution to the high-resolution image. A dictionary creation unit that creates a low-resolution dictionary that is a collection of a plurality of low-resolution base images whose resolution is lower than the resolution base image, and a sample is photographed at the second resolution set by the resolution setting unit to obtain a low resolution. Among the image acquisition unit for acquiring an image, the plurality of low-resolution base images of the low-resolution dictionary represented by developing the low-resolution image acquired by the image acquisition unit, and their respective coefficients, each of them. It is provided with an image construction unit that replaces the plurality of low-resolution base images with a plurality of corresponding high-resolution base images of the high-resolution dictionary without changing the coefficients and constructs the high-resolution image of the sample by linear coupling. It is an observation device.

According to this aspect, the high-resolution image can be represented by the linear sum of a plurality of high-resolution base images constituting the high-resolution dictionary, and the low-resolution image is a plurality of low-resolution base images constituting the low-resolution dictionary. It can be expressed by the linear sum of. In addition, since the low-resolution dictionary is created by downsampling the high-resolution dictionary by the dictionary creation unit, the plurality of high-resolution base images constituting the high-resolution dictionary and the plurality of low-resolution base images constituting the low-resolution dictionary are correlated. Have a relationship. Further, the low-resolution image obtained by photographing the sample at the second resolution having a lower resolution by the image acquisition unit cannot analyze the details of the phenomenon as compared with the case of photographing with the first resolution having a higher resolution. It is possible to capture the high-speed phenomenon of the sample.

Therefore, the image construction unit constructs a high-resolution image of the sample from the low-resolution image of the sample acquired by the image acquisition unit at the second resolution using the low-resolution dictionary and the high-resolution dictionary, thereby sampling at the first resolution. It is possible to create a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample, as compared with the case of acquiring a high-resolution image by photographing the image. As a result, the sample can be observed at both high resolution and high speed.

In the above aspect, the image acquisition unit acquires a plurality of high- resolution captured images at the first resolution set by the resolution setting unit, and the dictionary creation unit acquires a plurality of high-resolution captured images acquired by the image acquisition unit. The high-resolution dictionary may be created by aggregating a plurality of the high-resolution base images that can represent the high-resolution captured image by a linear sum.
With this configuration, the observation device can perform everything from the acquisition of a high-resolution photographed image and the creation of a high-resolution dictionary to the construction of a high-resolution image using the high-resolution dictionary.

In the above aspect, the dictionary creator can represent a plurality of the high-resolution captured images acquired by another microscope at a resolution higher than the first resolution by a linear sum of the plurality of high-resolution base images. The high-resolution dictionaries may be created by assembling them.
With this configuration, the image construction unit can construct a high-resolution image with a high resolution that cannot be acquired by this observation device alone.

In the above aspect, the dictionary creating unit can represent a plurality of the high resolution captured images obtained by photographing the same sample as the low resolution image acquired by the image acquisition unit by a linear sum. The high-resolution base image of the above may be aggregated to create the high-resolution dictionary.

With this configuration, in the image construction unit, the low-resolution image of the sample acquired by the image acquisition unit is developed by a plurality of low-resolution base images of the sample, and the high-resolution image of the sample is a plurality of the sample. Constructed by a linear combination of high resolution base images. Therefore, it is necessary to improve the accuracy of the high-resolution image constructed by the image construction unit as compared with the case of using a high-resolution dictionary in which a sample of a low-resolution image and a plurality of high-resolution base images of different samples are aggregated. Can be done.

In the above aspect, a dictionary selection unit for selecting the high-resolution dictionary to be used from the sample of the high-resolution base image or a plurality of high-resolution dictionaries having different resolutions may be provided.

With this configuration, it is only necessary to prepare a plurality of high-resolution dictionaries in advance and select the high-resolution dictionary to be used from among them by the dictionary selection unit, so that it takes time to create a high-resolution dictionary. Can be omitted. Further, if the dictionary selection unit selects the optimum high-resolution dictionary according to the sample, the accuracy of the high-resolution image constructed by the image construction unit can be improved.

In the above aspect, the dictionary selection unit may select the high-resolution dictionary to be used from the network.
With this configuration, an appropriate high-resolution dictionary can be selected and used from many options.

In the above aspect, the image construction unit captures the sample at the second resolution and at a predetermined time interval by time-lapse photography by the image acquisition unit, and based on the plurality of low-resolution images obtained. A plurality of the high resolution images of the sample may be constructed.
With this configuration, it is possible to observe the time change of the sample in detail without missing the high-speed phenomenon of the sample.

In the above aspect, the image construction unit divides the sample into a plurality of imaging ranges by the image acquisition unit by the image acquisition unit and photographs the specimens at the second resolution to obtain the plurality of low resolution images. Based on this, a plurality of partial high-resolution images of the sample may be constructed, and the plurality of partial high-resolution images may be joined to construct an overall high-resolution image of the sample. ..
With this configuration, a wide range of the sample can be observed in detail without missing the high-speed phenomenon of the sample.

A second aspect of the present invention is to construct the desired high resolution image by linear coupling according to the ratio of the first resolution for creating the high resolution image to the second resolution lower than the first resolution. A high-resolution dictionary consisting of a collection of a plurality of high-resolution base images, which is the basis of the above, is downsampled to create a low-resolution dictionary consisting of a collection of a plurality of low-resolution base images having a lower resolution than the high-resolution base image. A dictionary creation step, a time-lapse observation step of taking a sample at the second resolution and a predetermined time interval to acquire a plurality of low-resolution images, and a plurality of the low-resolution images acquired by the time-lapse observation step are developed. The plurality of low-resolution base images of the low-resolution dictionary and their respective coefficients are represented by the respective coefficients, and the plurality of the low-resolution base images are represented by the plurality of corresponding high-resolution bases of the high-resolution dictionary without changing the respective coefficients. It is an observation method including an image construction step of constructing a plurality of the high resolution images of the sample by linear coupling in place of an image.

According to this aspect, the high resolution image can be expressed as a linear sum of a plurality of high resolution base images constituting the high resolution dictionary, and the low resolution image is a plurality of low resolution base images constituting the low resolution dictionary. Can be expressed as a linear sum of. In addition, since the low-resolution dictionary is created by downsampling the high-resolution dictionary in the dictionary creation process, the plurality of high-resolution base images constituting the high-resolution dictionary and the plurality of low-resolution base images constituting the low-resolution dictionary are correlated. Have a relationship. Furthermore, a plurality of low-resolution images obtained by photographing a sample at a second resolution having a lower resolution by a time-lapse observation process captures a high-speed phenomenon of the sample as compared with a case where a sample is photographed at a first resolution having a high resolution. Can be done.

Therefore, by constructing a high-resolution image of the sample using the low-resolution dictionary and the high-resolution dictionary from the low-resolution image of the sample acquired at the second resolution by the time-lapse observation step by the image construction step, the first resolution and the predetermined value are obtained. Compared with the case of taking a sample at the time interval of the above and acquiring a plurality of high-resolution images, it is possible to acquire a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample. As a result, both high resolution and high speed can be achieved, and the time change of the sample can be observed in detail without missing the high-speed phenomenon of the sample.

A third aspect of the present invention is to construct the desired high resolution image by linear coupling according to the ratio of the first resolution for creating the high resolution image to the second resolution lower than the first resolution. A high-resolution dictionary consisting of a collection of a plurality of high-resolution base images, which is the basis of the above, is downsampled to create a low-resolution dictionary consisting of a collection of a plurality of low-resolution base images having a lower resolution than the high-resolution base image. A dictionary creation step, a map observation step of dividing the imaging range of a sample into a plurality of images, photographing each imaging range at the second resolution to acquire a plurality of low-resolution images, and a plurality of images acquired by the map observation process. The low-resolution images of the above are expanded and represented by the plurality of the low-resolution base images of the low-resolution dictionary and their respective coefficients, and the plurality of the low-resolution base images of the high-resolution dictionary are expressed without changing the respective coefficients. Replacing the plurality of corresponding high-resolution base images, a plurality of the partial high-resolution images of the sample are constructed by linear coupling, and the plurality of partial high-resolution images are joined together to form the entire sample. This is an observation method including an image construction step of constructing the above-mentioned high-resolution image.

According to this aspect, the image construction step constructs a high-resolution image of the sample from the low-resolution image of the sample acquired at the second resolution by the map observation step using a low-resolution dictionary and a high-resolution dictionary. Compared to the case where each shooting range of a sample is photographed at one resolution and a plurality of partial high-resolution images are acquired, a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample is displayed. Can be obtained. As a result, it is possible to observe a wide range of the sample in detail without missing the high-speed phenomenon of the sample while achieving both high resolution and high speed.

According to the present invention, there is an effect that a sample can be observed at both high resolution and high speed.

It is a block diagram which shows the observation apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows an example of a high-resolution dictionary. It is a figure which shows an example of a low resolution dictionary. It is a flowchart explaining the process of creating a high-resolution image of a sample by the observation apparatus of FIG. It is a figure which shows an example of the shooting condition shown on the monitor screen. It is a figure which shows an example of the low-resolution image of the sample acquired by the observation apparatus of FIG. FIG. 6 is a diagram showing an example of a linear sum represented by a plurality of low-resolution base images and their respective coefficients by expanding the low-resolution image of FIG. 6. It is a figure which shows an example which replaced a plurality of low-resolution base images with a plurality of high-resolution base images without changing the respective coefficients of the linear sum of FIG. 7. It is a figure which shows an example of the high-resolution image constructed by linearly combining the linear sum of FIG. It is a flowchart explaining the process of making a high-resolution image of a sample by the observation apparatus which concerns on the modification of 1st Embodiment of this invention. It is a flowchart explaining the process of making a high-resolution image of a sample by the observation apparatus which concerns on another modification of 1st Embodiment of this invention. It is a block diagram which shows the observation apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart explaining the process of making a high-resolution image of a sample by the observation apparatus of FIG. It is a figure which shows an example of the shooting condition shown on the monitor screen. It is a flowchart explaining the step of observing a sample by time-lapse observation and creating a high-resolution image thereof by the observation apparatus and observation method which concerns on the modification of 1st and 2nd Embodiment of this invention. It is a flowchart explaining the process of observing a sample by map observation and creating a high-resolution image thereof by the observation apparatus and observation method which concerns on modification of 1st and 2nd Embodiment of this invention.

[First Embodiment]
The observation device according to the first embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the observation device 1 according to the present embodiment is based on a microscope 3 for observing a specimen (not shown), a user interface 5 for inputting an instruction from a user, and an instruction from the user interface 5. It is provided with a control calculation device 7 that controls the microscope 3 and performs calculation processing. Further, the observation device 1 is connected to an input unit such as a mouse or keyboard for inputting instructions by the user, and a monitor for displaying shooting conditions, images, and the like (both are not shown).

The microscope 3 collects the scanning optical system 9 such as a galvanometer mirror that scans the laser beam emitted from the light source (not shown) two-dimensionally on the sample, and the observation light from the sample irradiated with the laser light. It includes an objective lens (not shown) and an image pickup element (image acquisition unit) 11 that captures observation light focused by the objective lens and acquires an image.

The user interface 5 includes a high-resolution image size setting unit (resolution setting unit) 13 and a low-resolution image size setting unit (resolution setting unit) 15 for setting a resolution desired by the user as an image size, and a control calculation device according to user input. A shooting start instruction unit 17 that instructs 7 to start shooting, and a high-resolution image designation unit that specifies a plurality of high-resolution shot images used to create a high-resolution dictionary for constructing a high-resolution image by user input. It has 19 and. A high-resolution dictionary is a collection of a plurality of high-resolution base images that serve as a basis for constructing a desired high-resolution image by linear combination.

The high-resolution image size setting unit 13 sets the first resolution for creating a high-resolution image as the first image size.
The low-resolution image size setting unit 15 sets a second resolution lower than the first resolution as the second image size.

The shooting start instruction unit 17 instructs the control calculation device 7 to start shooting at the first image size set by the high resolution image size setting unit 13, or is a low resolution image size setting unit, depending on the user's choice. It is designed to instruct the start of shooting with the second image size set by 15.

The control calculation device 7 creates a photographing control unit 21 that controls the scanning optical system 9 and the image pickup element 11 of the microscope 3, a high-resolution dictionary creation unit (dictionary creation unit) 23 that creates a high-resolution dictionary, and a low-resolution dictionary. It includes a low-resolution dictionary creation unit (dictionary creation unit) 25 for creating a high-resolution image, and a high-resolution image creation unit (image construction unit) 27 for creating a high-resolution image.

The image pickup control unit 21 takes a sample in the first image size or a sample in the second image size in response to an instruction from the image pickup start instruction unit 17 to start photography. And the image sensor 11 is controlled.

The high-resolution dictionary creation unit 23 internally generates and generates a plurality of high-resolution base images that can represent a plurality of different high-resolution captured images designated by the high-resolution image designation unit 19 by a linear sum. A high-resolution dictionary as shown in FIG. 2, for example, is created by aggregating a plurality of high-resolution base images. The plurality of high-resolution base images constituting the high-resolution dictionary may have different shooting parts of the same subject or different subjects themselves, and the larger the number, the more preferable. In FIG. 2, Dh indicates a high-resolution dictionary and Bh indicates a high-resolution base image.

The low-resolution dictionary creation unit 25 downloads the high-resolution dictionary according to the ratio of the first image size set by the high-resolution image size setting unit 13 to the second image size set by the low-resolution image size setting unit 15. A low-resolution dictionary as shown in FIG. 3, for example, is created by sampling and aggregating a plurality of low-resolution base images having a resolution lower than that of the high-resolution base image. In FIG. 3, Dl indicates a low resolution dictionary and Bl indicates a low resolution base image.

By downsampling a high-resolution dictionary to create a low-resolution dictionary, a plurality of high-resolution base images constituting the high-resolution dictionary and a plurality of low-resolution base images constituting the low-resolution dictionary have a correlation. The downsampling here is converted into a plurality of low-resolution base images whose resolution is lowered by, for example, adding blur or noise to a plurality of high-resolution base images constituting the high-resolution dictionary to perform pseudo-deterioration. It is designed to do.

The high-resolution image creation unit 27 develops a low-resolution image acquired by the image pickup device 11 with a second image size set by the low-resolution image size setting unit 15, and the developed low-resolution image constitutes a low-resolution dictionary. It is represented by a plurality of low-resolution base images and their respective coefficients. Further, the high-resolution image creation unit 27 uses a high-resolution dictionary for a plurality of low-resolution base images represented by developing the low-resolution images and a plurality of low-resolution base images without changing each coefficient among the respective coefficients. By substituting a plurality of corresponding high-resolution base images in the above and linearly combining them, a high-resolution image of a sample is constructed.

The operation of the observation device 1 configured in this way will be described below with reference to the flowchart of FIG.
In order to create a high-resolution image of a sample by the observation device 1 according to the present embodiment, the user first wants to create a high-resolution image by the high-resolution image size setting unit 13 on a monitor screen as shown in FIG. 5, for example. A desired first image size (512 × 512 in the example shown in FIG. 5) is set as the first resolution (high resolution) of the image (step SA1).

Subsequently, the user sets a desired second image size (64 × 64 in the example shown in FIG. 5) as a second resolution (low resolution) lower than the first resolution by the low resolution image size setting unit 15 ( Step SA2).

Next, when the user inputs an instruction to start shooting at a high resolution (first image size) on the monitor screen as shown in FIG. 5, the imaging control unit 21 microscopes according to the instruction from the shooting start instruction unit 17. The scanning optical system 9 and the image pickup device 11 of No. 3 are controlled to take a sample at the first image size.

In the microscope 3, laser light is emitted from a light source (not shown) and scanned two-dimensionally on the sample by the scanning optical system 9, and the first image size set by the high-resolution image size setting unit 13 by the image sensor 11 is used. The observation light from the specimen is photographed. Similarly, a plurality of samples are photographed with the first image size by the image sensor 11, and a plurality of high-resolution photographed images are acquired (step SA3). The plurality of high-resolution captured images acquired by the image sensor 11 are sent to the high-resolution dictionary creation unit 23 via the imaging control unit 21.

Next, the user confirms a plurality of high-resolution captured images sent to the high-resolution dictionary creation unit 23 on a monitor or the like, and on the monitor screen as shown in FIG. 5, the high-resolution image designation unit 19 among them. A plurality of high-resolution captured images used to create a high-resolution dictionary from are specified (step SA4).

Next, when the user instructs to create a dictionary on the monitor screen as shown in FIG. 5, first, the high-resolution dictionary creation unit 23 creates a plurality of high-resolution captured images designated by the user by the high-resolution image designation unit 19. A plurality of high-resolution base images that can be represented by a linear sum are generated, and the generated plurality of high-resolution base images are aggregated to create a high-resolution dictionary as shown in FIG. 2 (step SA5).

Next, the low-resolution dictionary creation unit 25 sets a high resolution according to the ratio of the first image size set by the high-resolution image size setting unit 13 to the second image size set by the low-resolution image size setting unit 15. The high-resolution dictionary created by the dictionary creation unit 23 is downsampled, and a plurality of high-resolution base images constituting the high-resolution dictionary are converted into a plurality of low-resolution low-resolution base images. Then, the low-resolution dictionary creation unit 25 aggregates these plurality of low-resolution base images to create a low-resolution dictionary as shown in FIG. 3 (step SA6).

Next, when the user inputs an instruction to start shooting at a low resolution (second image size) on the monitor screen as shown in FIG. 5, the microscope 3 is operated by the shooting control unit 21 in accordance with the instruction from the shooting start instruction unit 17. The scanning optical system 9 and the image sensor 11 of the above are controlled to take a sample with a second image size.

Then, the image sensor 11 captures the observation light from the sample with the second image size set by the low-resolution image size setting unit 15, and a low-resolution image of the sample as shown in FIG. 6 is acquired (step). SA7). The low-resolution image obtained by photographing the sample at the second resolution with a low resolution by the image sensor 11 cannot analyze the details of the phenomenon as compared with the case of photographing with the first resolution with a high resolution, but the sample is It is possible to capture high-speed phenomena.

Subsequently, the high-resolution image creation unit 27 develops a low-resolution image of the sample acquired by the image sensor 11, for example, as shown in FIG. 7, and a plurality of low-resolution base images of the low-resolution dictionary and their respective. It is represented by a coefficient. In FIG. 7, a, b, c, and d are coefficients. The same applies to FIG.

Then, by the high-resolution image creation unit 27, for example, as shown in FIG. 8, the plurality of low-resolution base images and the plurality of low-resolution base images have high resolution without changing the respective coefficients among the plurality of low-resolution base images and their respective coefficients. It is replaced by a plurality of corresponding high-resolution base images in the dictionary, and the plurality of high-resolution base images and their respective coefficients are linearly combined. As a result, a high-resolution image of the sample as shown in FIG. 9 is created (step SA8).

As described above, according to the observation device 1 according to the present embodiment, the high-resolution image can be represented by the linear sum of a plurality of high-resolution base images constituting the high-resolution dictionary, and the low-resolution image can be represented by a linear sum. It can be expressed by the linear sum of a plurality of low-resolution base images constituting the low-resolution dictionary. Then, the high-resolution image creation unit 27 constructs a high-resolution image of the sample from the low-resolution image of the sample acquired by the image pickup element 11 at the second resolution using the low-resolution dictionary and the high-resolution dictionary. Compared with the case of taking a sample at a resolution and acquiring a high-resolution image, it is possible to create a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample. As a result, the sample can be observed at both high resolution and high speed.

This embodiment can be modified as follows.
As a first modification, for example, as shown in the flowchart of FIG. 10, after setting the second image size by the low resolution image size setting unit 15 (step SA2), a high resolution captured image acquired in advance is used. A plurality of high-resolution captured images may be prepared (step SB3), and a plurality of high-resolution captured images to be used may be specified (step SA4) to create a high-resolution dictionary (step SA5).

As the high-resolution photographed image acquired in advance, a natural image may be used, or an image acquired in advance by the microscope 3 may be used. Further, a high-resolution photographed image acquired by another microscope different from the microscope 3 may be used as a pre-acquired high-resolution photographed image. In this case, as the high-resolution captured image acquired in advance, a plurality of high-resolution captured images acquired at a resolution higher than the first resolution by another microscope such as an electron microscope may be used.

By using a plurality of high-resolution captured images acquired in advance, it is possible to save the trouble of acquiring a plurality of high-resolution captured images immediately before observation. As a result, it is possible to prevent unnecessary damage to the specimen. In addition, the high-resolution dictionary creation unit 23 creates a high-resolution dictionary in which a plurality of high-resolution base images are aggregated using a plurality of high-resolution captured images acquired by another microscope at a resolution higher than the first resolution. If this is the case, the high-resolution image creation unit 27 can construct a high-resolution high-resolution image that cannot be acquired by the observation device 1 alone.

As a second modification, for example, as shown in the flowchart of FIG. 11, the high-resolution image designation unit 19 obtains the same sample as the low-resolution image from a plurality of high-resolution images. A plurality of high-resolution captured images may be specified (step SC4). For example, when observing a nuclear portion of a specimen such as a cultured cell, a high-resolution photographed image including the nuclear portion may be specified.

In this case, the microscope 3 captures the same sample as the low-resolution image at the first image size to acquire a plurality of high-resolution captured images (see step SA3 in the flowchart of FIG. 4), or the microscope 3 or the microscope. The same sample as the low-resolution image is photographed with a first image size or a higher resolution by another microscope having a resolution higher than 3, and a plurality of high-resolution captured images are acquired in advance and prepared. This may be the case (see step SB3 in the flowchart of FIG. 10).

The high-resolution dictionary creation unit 23 internally generates and generates a plurality of high-resolution base images that can represent a plurality of high-resolution captured images obtained by photographing the same sample as the low-resolution image by a linear sum. By creating a high-resolution dictionary by aggregating a plurality of high-resolution base images, the low-resolution image of the sample acquired by the image pickup element 11 in the high-resolution image creation unit 27 is a plurality of low-resolution base images of the sample. The high-resolution image of the sample is constructed by a linear combination of multiple high-resolution base images of the sample.

Therefore, the accuracy of the high-resolution image constructed by the high-resolution image creation unit 27 is improved as compared with the case of using a high-resolution dictionary in which a plurality of high-resolution base images of a sample different from the low-resolution image sample are aggregated. Can be improved.

[Second Embodiment]
Next, the observation device according to the second embodiment of the present invention will be described.
In the observation device 31 according to the present embodiment, as shown in FIG. 12, the control calculation device 7 does not include the high-resolution dictionary creation unit 23, and the user interface 5 is high in place of the high-resolution image designation unit 19. It differs from the first embodiment in that it includes a resolution image dictionary designation unit (dictionary selection unit) 33.
Hereinafter, the parts having the same configuration as the observation device 1 according to the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

The high-resolution image dictionary designation unit 33 stores a sample of a high-resolution base image or a plurality of high-resolution dictionaries having different resolutions, and the user can specify any of the high-resolution dictionaries. .. Further, the high-resolution image dictionary designation unit 33 sends the high-resolution dictionary designated by the user to the low-resolution dictionary creation unit 25.

The operation of the observation device 31 configured in this way will be described below with reference to the flowchart shown in FIG.
In the observation device 31 according to the present embodiment, after the second image size is set (step SA2), the user monitors a plurality of high-resolution dictionaries stored in the high-resolution image dictionary designation unit 33 on a monitor or the like. After confirmation, a desired high-resolution dictionary is designated by the high-resolution image dictionary designation unit 33 on the monitor screen as shown in FIG. 14 (step SD3). As a result, the high-resolution dictionary specified by the user is sent to the low-resolution dictionary creation unit 25, and the low-resolution dictionary creation unit 25 creates the low-resolution dictionary (step SA6).

According to the observation device 31 according to the present embodiment, it is sufficient to prepare a plurality of high-resolution dictionaries in advance and select the high-resolution dictionary to be used from the high-resolution image dictionary designation unit 33. You can save time to create a high resolution dictionary. Further, if the high-resolution image dictionary designation unit 33 selects the optimum high-resolution dictionary according to the sample, the accuracy of the high-resolution image constructed by the high-resolution image creation unit 27 can be improved. For example, if a high-resolution dictionary containing a high-resolution base image of a sample of the same type as a sample of a low-resolution image is selected, the time required for constructing the high-resolution image can be shortened and the accuracy of the high-resolution image can be improved.

In the present embodiment, the high-resolution image dictionary designation unit 33 may select the high-resolution dictionary to be used from the network instead of selecting the high-resolution dictionary to be used from the stored high-resolution dictionaries. ..
By doing so, it is possible to select and use a high-resolution dictionary suitable for the sample to be observed from many options.

In each of the above embodiments, a plurality of low-resolution images are acquired by photographing a sample with a second image size and a predetermined time interval by time-lapse photography by the image sensor 11, and the high-resolution image creation unit 27 performs time-lapse photography. A plurality of high-resolution images of the sample may be constructed based on the plurality of acquired low-resolution images.

In this case, as shown in the flowchart of FIG. 15, the observation method may include an image size setting step SE1, a dictionary creation step SE2, a time lapse observation step SE3, and an image construction step SE4.

The image size setting step SE1 sets the first image size and the second image size.
The dictionary creation step SE2 downsamples the high-resolution dictionary according to the ratio of the first image size to the second image size to create a low-resolution dictionary.

In the time-lapse observation step SE3, the image sensor 11 takes a sample with a second image size and at a predetermined time interval, and acquires a plurality of low-resolution images of the same sample.
In the image construction step SE4, a plurality of low-resolution images acquired in the time-lapse observation step SE3 are expanded and represented by a plurality of low-resolution base images in a low-resolution dictionary and their respective coefficients, and a plurality of low-resolution images are not changed. The low-resolution base image is replaced with the corresponding high-resolution base images in the high-resolution dictionary, and multiple high-resolution images of the sample are constructed by linear coupling.

By doing so, by constructing a high-resolution image of the sample from the low-resolution image of the sample acquired by the time-lapse observation step SE3 using the low-resolution dictionary and the high-resolution dictionary, the first resolution and a predetermined time interval Compared with the case of taking a sample with and acquiring a plurality of high-resolution images, it is possible to take a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample. As a result, both high resolution and high speed can be achieved, and the time change of the sample can be observed in detail without missing the high-speed phenomenon of the sample.

Further, in each of the above embodiments, the image pickup element 11 divides the sample into a plurality of imaging ranges by tying imaging and captures the sample with a second image size to acquire a plurality of low-resolution images, and a high-resolution image creation unit. 27 builds multiple partial high resolution images of the sample based on the multiple low resolution images obtained and stitches these multiple partial high resolution images together to provide the overall high resolution of the sample. You may also build an image.

In this case, as shown in the flowchart of FIG. 16, the observation method includes an image size setting step SE1, a dictionary creation step SE2, a map observation step SF3, and an image construction step SE4. The image size setting step SE1 and the dictionary creation step SE2 are the same as the above-described modification.

In the map observation step SF3, the imaging range of the sample is divided into a plurality of images, and each imaging range is photographed at a second resolution to acquire a plurality of low-resolution images.
In the image construction step SE4, a plurality of low-resolution images acquired by the map observation step SF3 are developed and represented by a plurality of low-resolution base images in a low-resolution dictionary and their respective coefficients, and a plurality of low-resolution images are not changed. The low-resolution base image is replaced with the corresponding high-resolution base images in the high-resolution dictionary, and multiple high-resolution images of the sample are constructed by linear coupling.

By doing so, by constructing a high-resolution image of the sample from the low-resolution image of the sample acquired by the map observation step SF3 using the low-resolution dictionary and the high-resolution dictionary, each photographing of the sample at the first resolution is performed. Compared with the case of capturing a range and acquiring a plurality of partial high-resolution images, it is possible to create a high-resolution image showing the details of the high-speed phenomenon while capturing the high-speed phenomenon of the sample. As a result, it is possible to observe a wide range of the sample in detail without missing the high-speed phenomenon of the sample while achieving both high resolution and high speed.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included. For example, the present invention is not limited to the one applied to each of the above embodiments and modifications, and may be applied to an embodiment in which these embodiments and modifications are appropriately combined, and the present invention is not particularly limited. ..

1,31 Observation device 11 Image sensor (image acquisition unit)
13 High-resolution image size setting section (resolution setting section)
15 Low resolution image size setting section (resolution setting section)
23 High-resolution dictionary creation department (dictionary creation department)
25 Low resolution dictionary creation department (dictionary creation department)
27 High-resolution image creation department (image construction department)
33 High-resolution image dictionary designation section (dictionary selection section)
SE2 dictionary creation process SE3 time lapse observation process SE4 image construction process SF3 map observation process

Claims (10)

A resolution setting unit that sets a first resolution for creating a high-resolution image and a second resolution lower than the first resolution, and a resolution setting unit.
A plurality of high-resolution base images that serve as a base for constructing the desired high-resolution image by linear coupling are assembled according to the ratio of the first resolution to the second resolution set by the resolution setting unit. A dictionary creation unit that downsamples a high-resolution dictionary and creates a low-resolution dictionary by collecting a plurality of low-resolution base images whose resolution is lower than that of the high-resolution base image.
An image acquisition unit that captures a sample at the second resolution set by the resolution setting unit to acquire a low-resolution image, and an image acquisition unit.
Among the plurality of the low-resolution base images of the low-resolution dictionary represented by developing the low-resolution image acquired by the image acquisition unit and their respective coefficients, the plurality of the lows without changing the respective coefficients. An observation device including an image construction unit that replaces a resolution base image with a plurality of corresponding high resolution base images of the high resolution dictionary and constructs the high resolution image of the sample by linear coupling. The image acquisition unit acquires a plurality of high- resolution captured images at the first resolution set by the resolution setting unit.
Claim 1 in which the dictionary creation unit creates the high-resolution dictionary by aggregating a plurality of the high-resolution base images that can represent a plurality of the high-resolution captured images acquired by the image acquisition unit by a linear sum. The observation device described in. The dictionary creation unit aggregates a plurality of the high-resolution base images that can represent a plurality of high-resolution captured images acquired by another microscope at a resolution higher than the first resolution by a linear sum, and the high-resolution image. The observation device according to claim 1, wherein a dictionary is created. A plurality of high-resolution bases capable of representing a plurality of high-resolution captured images obtained by photographing the same sample as the low-resolution image acquired by the image acquisition unit by a linear sum. The observation device according to claim 2 or 3, wherein images are assembled to create the high-resolution dictionary. The observation device according to any one of claims 1 to 4, further comprising a dictionary selection unit for selecting the high-resolution dictionary to be used from the sample of the high-resolution base image or a plurality of high-resolution dictionaries having different resolutions. The observation device according to claim 5, wherein the dictionary selection unit selects the high-resolution dictionary to be used from the network. A plurality of the above-mentioned specimens based on the plurality of low-resolution images obtained by the image-constructing unit photographing the specimens at the second resolution and at predetermined time intervals by time-lapse photography by the image acquisition unit. The observation device according to any one of claims 1 to 6, which constructs a high-resolution image. The sample is based on the plurality of low-resolution images obtained by the image construction unit dividing the sample into a plurality of imaging ranges by the image acquisition unit and photographing the sample at the second resolution. 1. The observation device described in the image. A plurality of high resolutions that are the basis for constructing the desired high resolution image by linear coupling, depending on the ratio of the first resolution for acquiring the high resolution image to the second resolution lower than the first resolution. A dictionary creation process of downsampling a high-resolution dictionary consisting of a collection of base images and creating a low-resolution dictionary consisting of a collection of a plurality of low-resolution base images having a lower resolution than the high-resolution base image.
A time-lapse observation step of taking a sample at the second resolution and at a predetermined time interval to acquire a plurality of low-resolution images, and
The plurality of the low-resolution images acquired in the time-lapse observation step are developed and represented by the plurality of the low-resolution base images of the low-resolution dictionary and their respective coefficients, and the respective coefficients are not changed and the plurality of the low-resolution images are expressed. An observation method including an image construction step of replacing a resolution base image with a plurality of corresponding high resolution base images of the high resolution dictionary and constructing a plurality of the high resolution images of the sample by linear coupling. A plurality of high resolutions that are the basis for constructing the desired high resolution image by linear coupling, depending on the ratio of the first resolution for acquiring the high resolution image to the second resolution lower than the first resolution. A dictionary creation process of downsampling a high-resolution dictionary consisting of a collection of base images and creating a low-resolution dictionary consisting of a collection of a plurality of low-resolution base images having a lower resolution than the high-resolution base image.
A map observation step in which the imaging range of the sample is divided into a plurality of images, each imaging range is photographed at the second resolution, and a plurality of low-resolution images are acquired.
The plurality of the low-resolution images acquired by the map observation step are developed and represented by the plurality of the low-resolution base images of the low-resolution dictionary and their respective coefficients, and the respective coefficients are not changed and the plurality of the low-resolution images are expressed. The resolution base image is replaced with the corresponding high resolution base images of the high resolution dictionary, and a plurality of partial high resolution images of the sample are constructed by linear coupling, and these multiple partial high resolutions are constructed. An observation method that includes an image construction step of stitching together images to construct the overall high resolution image of the specimen.

Images