CN112748559B - Microscope and method for displaying microscopic images and computer program product - Google Patents
Microscope and method for displaying microscopic images and computer program product Download PDFInfo
- Publication number
- CN112748559B CN112748559B CN202011184461.0A CN202011184461A CN112748559B CN 112748559 B CN112748559 B CN 112748559B CN 202011184461 A CN202011184461 A CN 202011184461A CN 112748559 B CN112748559 B CN 112748559B
- Authority
- CN
- China
- Prior art keywords
- image
- points
- level
- microscopic
- image points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000004590 computer program Methods 0.000 title abstract description 9
- 239000011159 matrix material Substances 0.000 claims abstract description 26
- 230000008859 change Effects 0.000 claims description 12
- 238000000399 optical microscopy Methods 0.000 claims description 8
- 239000000523 sample Substances 0.000 description 28
- 230000003287 optical effect Effects 0.000 description 10
- 239000011164 primary particle Substances 0.000 description 8
- 230000003993 interaction Effects 0.000 description 6
- 150000002500 ions Chemical class 0.000 description 6
- 239000002245 particle Substances 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 3
- 230000010365 information processing Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000010894 electron beam technology Methods 0.000 description 2
- 238000010884 ion-beam technique Methods 0.000 description 2
- 238000000386 microscopy Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 239000000969 carrier Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001493 electron microscopy Methods 0.000 description 1
- 238000001000 micrograph Methods 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000004626 scanning electron microscopy Methods 0.000 description 1
- 238000004621 scanning probe microscopy Methods 0.000 description 1
- 238000010408 sweeping Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/002—Scanning microscopes
- G02B21/0024—Confocal scanning microscopes (CSOMs) or confocal "macroscopes"; Accessories which are not restricted to use with CSOMs, e.g. sample holders
- G02B21/0052—Optical details of the image generation
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
- G02B21/367—Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J37/00—Discharge tubes with provision for introducing objects or material to be exposed to the discharge, e.g. for the purpose of examination or processing thereof
- H01J37/26—Electron or ion microscopes; Electron or ion diffraction tubes
- H01J37/28—Electron or ion microscopes; Electron or ion diffraction tubes with scanning beams
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J37/00—Discharge tubes with provision for introducing objects or material to be exposed to the discharge, e.g. for the purpose of examination or processing thereof
- H01J37/30—Electron-beam or ion-beam tubes for localised treatment of objects
- H01J37/302—Controlling tubes by external information, e.g. programme control
- H01J37/3023—Programme control
- H01J37/3026—Patterning strategy
-
- H—ELECTRICITY
- H01—ELECTRIC ELEMENTS
- H01J—ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS
- H01J2237/00—Discharge tubes exposing object to beam, e.g. for analysis treatment, etching, imaging
- H01J2237/26—Electron or ion microscopes
- H01J2237/28—Scanning microscopes
Landscapes
- Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Optics & Photonics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Microscoopes, Condenser (AREA)
Abstract
The present invention relates to a method of displaying at least one digital microimage. In one step of the method, a first microscopic image of the sample is recorded, the first microscopic image being formed from a matrix of image points and being recorded row by row. The image points are obtained one by one. An image pyramid is determined from the first microscopic image while the first microscopic image is recorded. The image pyramid is formed from at least two image levels having progressively lower resolutions. The first microscopic image to be recorded forms a first image level of the image levels. The image points of the second of these image levels are determined by: the first filter is applied to a plurality of image points in the newly obtained image points of the first image level and to a plurality of image points of the first image level adjacent to the image points. At least one portion of one of the image levels is determined at least in part by display according to a user selection. In addition, the invention relates to a microscope and a computer program product.
Description
Technical Field
The invention relates firstly to a method for displaying at least one digital microimage, wherein the microimage can have a very high resolution. In addition, the invention relates to a microscope having an image recording unit, an image processing unit and an image display unit, wherein the microscope allows recording microscopic images with a very high resolution. The microscope is formed, for example, by a particle-optical microscopy device in the form of a scanning electron microscope or an ion microscope. Another subject of the invention forms a computer program product comprising a series of control commands.
Background
US 2014/0225902 A1 shows an image pyramid processor and a method for image processing at multiple resolutions. The image pyramid processor includes a hierarchy multiplexer configured to process multiple hierarchies of the image pyramid in separate work units using separate processing elements. The image pyramid processor also includes a buffer pyramid with memory that can be used to store the corresponding intermediate results of the individual work units.
From WO 2019/091570 A1, a device for microscopic observation of a sample in a time-resolved manner is known, which device comprises a sample holder, a data receiving unit with a device for receiving data from a recording area defined by a microscope, a setting unit for variably setting the relative position between the recording area and the sample holder, and a control device for actuating the setting unit and the data receiving unit. For receiving the reference data set, the data is repeatedly and alternately received and the relative position between the receiving area and the sample holder is changed. The corresponding offset vectors may be determined iteratively, which may be done at different resolution levels of the image pyramid.
EP 2,362,344 A1 teaches an information processing apparatus that causes a display apparatus to display a first image of an observation target object in a display area. In response to a request for a change in the display area, the information processing apparatus changes the display area at a first speed and causes the display area to display a second image. In response to a request to end the display area change, the information processing apparatus changes the display area at a first delay speed and causes the display area to display a third image. For this purpose an image pyramid structure is used.
US 2004/0167806 A1 shows a method for viewing virtual object carriers which are large images having a data size of at most 4GB and which contain at most 100GB of raw image data prior to compression. The server system invokes a compressed block of image data and sends the data block to the client, the block containing the image data at a resolution that approximates the desired resolution. The client scales the image data and renders the image at the desired resolution.
EP 1,84,763 B1 shows a method for recording image data of a linear array based object carrier scanner. The plurality of image strips are combined into a single combined baseline image. The image data is organized in an image pyramid.
Disclosure of Invention
Starting from the prior art, the object of the present invention is to be able to record high-resolution microscopic images recorded row by row while at the same time displaying these images at different resolutions with reduced effort.
The object is achieved by a method according to the following, by a computer program product according to the following and by a microscope according to the following.
The method of the invention is used to display at least one microscopic image. The microscopic image is preferably recorded with an optical microscope or alternatively preferably with a particle optical microscopy device in the form of a scanning electron microscope (REM) or an ion microscope. The microscopic image is preferably digital and preferably has a very high resolution. The resolution is thus characteristic of the microscopic image and forms the image resolution. The resolution of the microscopic image can be given in particular by the image points, for example by the number of columns and rows of image points. Thus, when the number of image points is large, the resolution of the microscopic image is high. Therefore, when the image concerned has more image points than another image, the image concerned has a higher resolution than the other image. Correspondingly, when the image in question has fewer image points than another image, the image in question has a lower resolution than the other image. The resolution of the microscopic image is not the optical resolution of the microscope, which is defined as the distance that the two structures must have at least in order to remain perceived as separate image structures after optical imaging.
The user of the microscope may select the part of the microscopic image that should be displayed. Thereby, the user also selects the resolution of the illustration. This is preferably a plurality of microscopic images, that is to say a series of microscopic images or microscopic videos, in particular microscopic live videos.
In one step of the method of the invention, at least one first microscopic image of the sample is recorded. The first microscopic image is formed by a matrix of image points and recorded row by row. The progressive scanning is thus performed, as is done for example in the case of electron microscopes or laser scanning microscopes. The image points are obtained one by one. The image points are preferably obtained individually one by one. The recording of the first microscopic image is thus carried out image-point by image-point.
According to the invention, an image pyramid has been determined from the first microscopic image while the first microscopic image is still being recorded. The image pyramid is formed from at least two of the image levels having progressively lower resolutions. The resolution of the image hierarchy is the image resolution as described above and in particular given in terms of image points. Since the resolution of the image levels decreases in an upward direction toward the top of the pyramid, the image pyramid has a smaller and smaller number of image points from one image level to another from bottom to top, such that the uppermost image level has the smallest resolution. The image pyramid is preferably formed of at least four image levels with progressively lower resolutions. The number of image levels with progressively lower resolutions should preferably be chosen such that the last image level, i.e. the uppermost image level, has a resolution equal to or close to the resolution of the display used to display the microscopic image, such that the uppermost image level achieves the minimum scaling factor to be reached. The first microscopic image to be recorded forms a first image level of the image levels, which forms the lowest image level of the image pyramid. The image points of the second of these image levels are determined by: a first filter is applied to a plurality of image points in the newly obtained image points of the first image level and to a plurality of image points of the first image level adjacent to the image points. These adjacent image points may be already recorded and/or not yet recorded image points. The number of image points is reduced by applying the first filter so that the next higher image levels have respectively smaller resolutions.
The image pyramid has been determined from the first microscopic image during recording of the first microscopic image. The image pyramid is preferably already determined from the first microimage during the recording of a row of these image points of the first microimage, so that the image pyramid is already determined during the recording of the remaining image points of the respective row.
According to the invention, at least one part of one of the image levels of the at least partially determined image pyramid is displayed, depending on the selection that the user has made for the display of the microscopic image. This is preferably already done during the recording of the first microimage and during the determination of the image pyramid, so that it is not necessary to completely record the first microimage nor to completely determine the image pyramid, but only to contain the user-selected part.
A particular advantage of the method according to the invention is that the first filter has to be applied only to a small region of the recorded or yet-to-be-recorded first microimage, respectively, so that its application can be low-cost and fast in order to be able to display the first microimage or the already recorded part of the first microimage in a low-delay manner in response to a user selection. In any event, the first microscopic image need not be fully recorded and the image pyramid need not be fully determined at the user-selected local point in time at which the first microscopic image is displayed, as the partially determined image pyramid is already available for display. Thereby allowing the user to rotate and zoom to more easily select the part.
In a preferred embodiment of the method of the invention, after an image point of the second image level has been obtained, a plurality of image points of the second image level adjacent to this image point are each determined by: the first filter is applied to one of the newly acquired image points of the first image level and to an image point adjacent to the image point. This results in the previously acquired image points of the second image level being retrieved, i.e. updated. The reason for this is that one image point just obtained in the first image level affects a plurality of image points in the region from the second image level by the action of the first filter. The image point that has been obtained previously may also belong to this region of the second image level.
The process of the invention is preferably carried out row by row. Thus, in a preferred embodiment of the method of the invention, after one of the rows of the image point matrix of the first image or of the first image level has been obtained, the image points of a plurality of adjacent rows of the second image level are determined by applying the first filter a plurality of times. The action of the first filter by one line of image points just obtained in the first image level affects a plurality of lines of image points of the second image level. The lines for which image points have been previously obtained may also belong to those of the second image level.
In a preferred embodiment of the method according to the invention, the first filter is applied to the plurality of image points of the newly acquired image points of the first image level and to a plurality of still to be acquired image points of the first image level surrounding the image points, respectively. These image points still to be obtained are each predefined with an initial value. The initial value preferably corresponds to an image point of black. The initial values may also correspond to gray, white or other image points.
In a preferred embodiment of the method of the invention, the first filter is applied to a sub-matrix of image points of the first image level, respectively, the sub-matrix comprising the plurality of image points of the newly acquired image points. This sub-matrix is preferably rectangular or square. This sub-matrix preferably has between three and ten columns and between three and ten rows. This sub-matrix particularly preferably has a number of columns between three and seven and a number of rows between three and seven.
The image pyramid is preferably formed from at least three image levels of image levels having progressively lower resolutions. Correspondingly, the method preferably comprises a further step in which an image point of a third one of the image levels is determined. This determination is made by the following means: a second filter is applied to a plurality of image points in the newly determined image points of the second image level and to a plurality of image points of the second image level adjacent to the image points. The application of the second filter to the second image level preferably takes place in the same way as the application of the first filter to the first image level, so that the preferred features given above for the application of the first filter also correspondingly apply for the application of the second filter.
The image pyramid is preferably formed of more than three image levels of image levels having progressively lower resolutions. Correspondingly, the method comprises the further step in which image points of at least one further image level of the image levels are determined. This determination is made by the following means: the further filter is applied to a plurality of image points in the newly determined image points of the respectively previously acquired image level, that is to say of the respectively deeper image level, and to a plurality of image points of the respectively previously acquired image level which are adjacent to these image points. The application of the further filter to the respective previously obtained image hierarchy preferably takes place in the same way as the application of the first filter to the first image hierarchy, so that the preferred features given above for the application of the first filter also correspondingly apply to the application of the further filter.
The method of the invention is particularly suitable also for samples that vary over time. Therefore, it is preferred to record not only the first microscopic image, but also at least one second microscopic image of the sample after the first microscopic image has been recorded. The second microscopic image is formed from a matrix of image points like the first microscopic image and is recorded row by row, wherein the image points are acquired one by one. The recording of the second microscopic image is thus carried out image-point by image-point. The image pyramid is continuously updated while the second microscopic image is still recorded. The obtained image points of the second microscopic image to be recorded each replace a corresponding image point of the first image level after its acquisition. Updating at least some of the image points of the second image level by: the first filter is applied to a plurality of image points in the newly replaced image point of the first image level and to a plurality of image points of the first image level adjacent to the image points. The updated image points replace corresponding image points of the second image hierarchy. The image points adjacent to the newly replaced image point may be image points that have not been replaced, such that they originate from the first microscopic image. The display content is updated as well. Thus, selected portions of respective ones of the image levels of the at least partially updated image pyramid are displayed.
A particular advantage of the method according to the invention is that not only the first microimage can be displayed with low delay corresponding to the user's selection, but also the second microimage can be displayed. The low cost and fast application of the first filter allows for low delay updating of the image pyramid.
The structure of the image pyramid preferably remains unchanged when the second microscopic image is recorded, so that the features given to the image pyramid of the first microscopic image are preferably also given for the second microscopic image.
In most cases, the first and second microscopic images have only a small difference. Typically, these variations involve only one or a few small areas of the microscopic image. The vast majority of these image points of the second microscopic image are unchanged relative to the first microscopic image. Thus, in a preferred embodiment of the method according to the invention, the change region is searched. In the region of the changes, the first microscopic image is different from the second microscopic image, wherein a threshold value can be applied to identify the changes. The variation region or regions generally form a component of the second microimage. No update of the image pyramid is required outside the change region. This saves computational effort and minimizes the data to be transmitted to the display. Updating at least a number of the image points of the second image level is performed exclusively in the at least one change region. Within the at least one change region, the image points of the second image level are updated by: the first filter is applied to a plurality of image points in the newly replaced image point of the first image level and to a plurality of image points of the first image level adjacent to the image points.
The at least one variation area is preferably selected to be rectangular or square.
It is preferred to record not only the first and second microscopic images, but also further microscopic images. The recording of these further microscopic images is carried out in a manner corresponding to the recording of the second microscopic image. The step of updating the image pyramid and the step of displaying the selected part are likewise carried out in a corresponding manner as in the case of the second microscopic image. The plurality of microscopic images preferably form a microscopic video, in particular a microscopic live video.
In a preferred embodiment of the method of the invention, the first filter comprises a low pass filter. In the same way, the second filter and, where appropriate, the other filters preferably each comprise a low-pass filter. However, in theory, the low-pass filter may also be omitted.
In a preferred embodiment of the method of the invention, the first filter comprises a low pass filter and a decimation filter to reduce the resolution. In the same way, the second filter and, where appropriate, the other filters preferably comprise a low-pass filter and a decimation filter, respectively, to reduce resolution. In other preferred embodiments of the method of the present invention, the first filter comprises a decimation filter to reduce resolution, but does not comprise a low pass filter. In the same way, the second filter and, where appropriate, the other filters preferably each include a decimation filter to reduce resolution, but do not include a low pass filter.
The low-pass filter or filters, respectively, are preferably formed by a convolution matrix. The convolution matrix preferably comprises between three and ten columns and between three and ten rows. The convolution matrix preferably has between three and seven columns and between three and seven rows.
Alternatively or additionally, the first filter, the second filter, if appropriate, and the third filter, if appropriate, preferably each comprise a gaussian filter, a filter for weighted averaging, and/or a filter for weighted averaging over distance.
The first filter and the second filter and, where appropriate, the further filters are preferably identical.
The method of the invention is suitable for microscopic images with very high resolution. Correspondingly, the matrix of the first microimages preferably has at least 10,000 columns and 10,000 rows. The matrix of first microscopy images further preferably has at least 32,000 columns and 32,000 rows. The same applies to the matrix of the second and, where appropriate, the other microimages.
The matrix of first microimages is preferably square. The same applies to the matrix of the second and, where appropriate, the other microimages.
The matrix of the first and the matrix of the second and, if appropriate, the matrix of the further microimages are preferably of the same size, that is to say they have the same number of columns and the same number of rows.
In the method of the invention, when the user changes the resolution of the image to be displayed, it is preferred to display at least one part of another one of the image levels of the image pyramid. The image hierarchy to be displayed is selected corresponding to the part selected by the user.
The computer program product of the invention comprises a control command sequence by means of which the particle-optical device implements the method of the invention for displaying microscopic images. The described preferred embodiment of the method of the invention is preferably implemented by the control command sequence for the particle-optical device. The computer program product is formed in particular by software or by a computer program.
The microscope of the invention is preferably an optical microscope or alternatively preferably a particle-optical microscopy apparatus. Charged particles are used in particle-optical microscopy equipment to perform microscopy. The particle-optical microscopy apparatus is preferably a particle beam microscope and is preferably formed by a scanning electron microscope (REM) or by an ion microscope. The microscope includes an image recording unit, an image processing unit, and an image display unit. The image recording unit is preferably formed for progressive scanning. The image recording unit is preferably formed for particle-optical microscopy, in particular for electron microscopy, particularly preferably for scanning electron microscopy (scanning electron microscope, SEM). Alternatively, the image recording unit is preferably formed for a laser scanning microscopy method. The image recording unit can theoretically also be formed for other microscopy methods. The image processing unit is configured, where appropriate, together with the image display unit for carrying out the method of the invention. The image processing unit is configured, where appropriate, together with the image display unit, preferably for implementing the described preferred embodiment of the method of the invention.
Drawings
Further details and improvements of the invention emerge from the following description of a preferred embodiment of the invention, with reference to the accompanying drawings. In the drawings:
fig. 1: a schematic representation of three image levels of an image pyramid is shown, which can be determined according to a preferred embodiment of the method of the invention;
fig. 2: illustrating the principle of formation of an image pyramid according to a preferred embodiment of the method of the present invention;
fig. 3: a partial display of a microscopic image is shown according to a preferred embodiment of the method according to the invention;
fig. 4: another partial display of a microscopic image is shown according to a preferred embodiment of the method according to the invention;
fig. 5: another partial display of a microscopic image is shown according to a preferred embodiment of the method according to the invention; and
fig. 6: a preferred embodiment of the microscope of the invention in the form of a scanning electron microscope is shown.
Detailed Description
Fig. 1 shows a schematic representation of three image levels of an image pyramid, which can be determined according to a preferred embodiment of the inventive method. The recorded first microimage forms a first image level 01. A second image level 02 is determined from the first image level 01. A third image level 03 is determined from the second image level 02.
After recording the second microscopic image, the second microscopic image forms the first image level 01. The second microscopic image is changed only in the change region 04 compared to the first microscopic image. According to the invention, the changes immediately propagate into the change regions 04 of the second image level 02 and the third image level 03. Thereby ensuring that the image pyramid is in a consistent state at any time.
Fig. 2 shows the principle of formation of an image pyramid according to a preferred embodiment of the method of the invention. The first three image levels 01, 02, 03 and a fourth image level 06 are again formed. The image levels 01, 02, 03, 06 each consist of rows 07 of image points, only a few of which rows 07 are shown. The first image level 01 illustratively has a resolution of 32,000 by 32,000 image points. The second image level 02 has a resolution of, for example, 16,000 by 16,000 image points. The third image level 03 has a resolution of, for example, 8,000 by 8,000 image points. The fourth image level 06 has a resolution of, for example, 4,000 by 4,000 image points.
The first image level 01 illustratively comprises only one row 08 in which a change has occurred between the first and second microimages. Row 08 with changes is shown in dots. However, these changes in line 08 illustratively act on three lines 09 of the second image level 02 when a filter (not shown) to be used in accordance with the present invention is applied. These three affected rows 09 of the second image level 02 are shown in dashed lines. A filter (not shown) is applied to the five shown rows 07 of the first image level 01, such that unchanged ones of these rows 07 are also reconsidered by applying the filter.
In applying the filter (not shown) to be used according to the invention, the changes in these three lines 09 of the second image level 02 exemplarily act on four lines 11 of the third image level 03. These four affected rows 11 of the third image level 03 are shown in dashed lines. A filter (not shown) is applied to the seven shown rows 07 of the second image level 02, thereby also reconsidering unchanged ones of these rows 07 by applying the filter.
In applying the filter (not shown) to be used according to the invention, the changes in these four rows 11 of the third image level 03 illustratively act on four rows 12 of the fourth image level 04. These four affected rows 12 of the fourth image level 04 are shown in dashed lines. A filter (not shown) is applied to the eight shown rows 07 of the third image level 03, such that unchanged ones of these rows 07 are also taken into account by applying the filter.
Fig. 3 shows a partial display of a microscopic image according to a preferred embodiment of the method according to the invention. These four image levels 01, 02, 03, 06 are again shown. In the example shown, the user has selected the smallest degree of zoom so that he observes the whole microscopic image, but at a smaller resolution. The minimum degree of scaling is illustratively 5%. The method according to the invention proposes to display the content of the uppermost image level, that is to say of the fourth image level 06. Thus, the complete section 13 of the fourth image level 06 is displayed on the monitor 14.
Fig. 4 shows a further partial display of a microscopic image according to a preferred embodiment of the method according to the invention. In the illustrated example, the user has selected a medium degree of zoom such that he observes a medium-sized part of the microscopic image with a medium resolution. The intermediate degree of scaling is illustratively 25%. The method of the invention proposes to display the content of the second image level from the most bottom, that is to say the second image level 02. Thus, a portion 16 of the second image level 02 is displayed on the monitor 14, wherein the portion 16 cannot be fully displayed due to the selected resolution.
Fig. 5 shows a further partial display of a microscopic image according to a preferred embodiment of the method according to the invention. In the example shown, the user has selected the maximum degree of zoom so that he observes a small part of the microscopic image with the maximum resolution. The maximum degree of scaling is 100%. The method of the invention proposes to display the content of the lowest hierarchical level, that is to say the first image level 01. Thus, a portion 17 of the first image level 01 is displayed on the monitor 14, wherein the portion 17 cannot be fully displayed due to the selected resolution.
Fig. 6 shows a preferred embodiment of the microscope of the invention in the form of a scanning electron microscope (REM). Scanning electron microscopes are examples of particle-optical devices 101 suitable for carrying out the method of the invention. However, the practice of the invention is not limited to scanning electron microscopes, but can also be practiced with other microscopes, especially with optical or particle-optical equipment (e.g., with ion microscopes).
The particle-optical apparatus 101 is provided with an electron source 104 which is located in the electron-optical column 103 and which can generate a primary particle beam (in this case a primary electron beam). The primary particle beam is a beam of charged particles, in the present example an electron beam.
In sample chamber 102, sample 113 is located on sample stage 114, which is formed to receive the sample and the vacuum conditions prevail. The sample stage 114 is formed so as to be movable in a plurality of directions, preferably at least in three spatial directions x, y, and z perpendicular to each other, perpendicular to and parallel to the optical axis 106 of the electron optical column 103.
The primary particle beam is directed towards the sample 113, accelerated along the optical axis 106 of the electron-optical column 103, bundled by the lens system 105, 107 and passed through at least one aperture stop 108. Furthermore, the electron optical column 103 comprises a deflection system 109 by means of which the primary particle beam can be directed in a defined manner (e.g. row by row) over the sample surface, which is referred to as "sweeping", "scanning" or "sampling".
When the particles of the primary particle beam hit the surface of the sample 113, secondary electrons, e.g. Secondary Electrons (SE), may be detected by means of a detector, due to the release of interaction products, e.g. Secondary Electrons (SE), by the interaction between the hit particles and the sample material. For this purpose the particle-optical apparatus comprises at least one first detector 110. By means of the analysis and control unit 111, an image of the sample can be generated from the signals detected by the detector 110.
Since the primary particle beam is directed stepwise over the sample surface, different locations on the sample are irradiated such that the interaction products are emitted at the irradiated locations, respectively. Image information of the respective sample locations is obtained by detecting the interaction products, wherein the image information may be presented as image points (pixels).
The primary particle beam may be swept (line scanned) over the sample, for example, in a line. This means that the primary particle beam is directed along separate rows on the sample surface. After the end of the line is reached, the primary beam is diverted to the starting position of the next line and the primary beam is subsequently swept along the next line.
The particle-optical apparatus is not limited to an electron microscope, but may also be formed as an ion microscope, such as a FIB (focused ion beam) system. Here, ions (FIB) focused into a beam sample the sample surface. An image of the sample may be generated by detecting the interaction products released due to the interaction of the ion beam with the sample material.
The analysis and control unit 111 of the particle-optical apparatus is configured for implementing a series of control commands, which are included in the computer program product. The particle-optical apparatus 101 is caused to implement the method of the invention by implementing control commands.
List of reference numerals
01. First image hierarchy
02. Second image hierarchy
03. Third image hierarchy
04. Change region
05 -
06. Fourth image hierarchy
07. Row of lines
08. With varying rows
09. Affected rows
10 -
11. Affected rows
12. Affected rows
13. Complete local area
14. Monitor
15 -
16. Local area
17. Local area
101. Particle-optical device
102. Sample chamber
103. Electron optical column
104. Electron source
105. First lens system
106. Optical axis of electron column
107. Second lens system
108. Aperture diaphragm
109. Deflection yoke
110. Detector for detecting a target object
111. Analysis and control unit
112 -
113. Sample of
114. Sample stage
Claims (12)
1. A method for displaying microscopic images; the method comprises the following steps:
-recording a first microscopic image of the sample, which is formed by a matrix of image points and is recorded row by row, wherein the image points are acquired one by one;
-determining an image pyramid from the first microimage while recording the first microimage, wherein the image pyramid is formed of at least two image levels (01, 02, 03, 06) having progressively lower resolutions, wherein the first microimage to be recorded forms a first (01) of the image levels, wherein the image points of a second (02) of the image levels are determined by: applying a first filter to a plurality of image points of the newly obtained image points of the first image level (01) and to a plurality of image points of the first image level (01) adjacent to the image points; and
displaying at least one part (13; 16; 17) of one of the image levels (01, 02, 03, 06) of the at least partially determined image pyramid in accordance with a selection by a user,
wherein the image points are individually obtained one by one, an
The image pyramid has been determined from the first microscopic image during the recording of a row of the image points of the first microscopic image such that the image pyramid has been determined during the recording of the remaining image points of the respective row.
2. Method according to claim 1, characterized in that after an image point of the second image level (02) has been obtained, a plurality of image points of the second image level (02) adjacent to this image point are each determined by: the first filter is applied to one of the newly acquired image points of the first image level (01) and to an image point adjacent to the image point.
3. Method according to claim 1, characterized in that after one of the rows of the image point matrix of the first image level (01) has been obtained, the image points of adjacent rows of the second image level (02) are determined by applying the first filter a plurality of times.
4. A method according to one of claims 1 to 3, characterized in that the first filter is applied to the plurality of image points of the newly acquired image points of the first image level (01) and to a plurality of still-acquired image points of the first image level (01) surrounding the image points, wherein the still-acquired image points are each predefined with an initial value.
5. A method according to one of claims 1 to 3, characterized in that the first filter is applied to a sub-matrix of image points of the first image level (01), respectively, the sub-matrix comprising the plurality of image points of the newly acquired image points.
6. The method of claim 5, wherein the sub-matrix has between three and seven columns and between three and seven rows.
7. A method according to one of claims 1 to 3, characterized in that the method comprises the following further steps:
-determining image points of a third image level (03) of the image levels, wherein the determination is made by: a second filter is applied to a plurality of image points of the newly determined image points of the second image level (02) and to a plurality of image points of the second image level (02) adjacent to the image points.
8. A method according to one of claims 1 to 3, characterized in that the method comprises the following further steps:
-recording a second microscopic image of the sample, which second microscopic image is formed by a matrix of image points like the first microscopic image and is recorded row by row, wherein the image points of the second microscopic image are acquired one by one;
-updating the image pyramid while recording the second microscopic image, wherein the obtained image points of the second microscopic image to be recorded replace the corresponding image points of the first image level (01) after their respective acquisition, wherein at least several of the image points of the second image level (02) are each updated by: applying the first filter to a plurality of image points of the newly replaced image points of the first image level (01) and to a plurality of image points of the first image level (01) adjacent to the image points; and
-displaying selected parts (13; 16; 17) of respective ones of the image levels (01, 02, 03, 06) of the at least partially updated image pyramid.
9. The method according to claim 8, wherein the first and second microimages differ only in at least one change region of the second microimage, wherein updating of at least several of the image points of the second image hierarchy is performed exclusively in the at least one change region.
10. A computer-readable storage medium comprising a sequence of control commands by which a particle-optical device (101) is caused to implement a method for presenting microscopic images according to one of claims 1 to 9.
11. Microscope (101) having an image recording unit (103, 104, 105, 107, 108, 109, 110), an image processing unit (111) and an image display unit, characterized in that the image processing unit (111) is configured for carrying out the method according to one of claims 1 to 9.
12. The microscope (101) according to claim 11, characterized in that the microscope is formed by a particle-optical microscopy device (101), wherein the particle-optical microscopy device (101) is formed by a scanning electron microscope (101) or by an ion microscope.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE102019129160.7 | 2019-10-29 | ||
| DE102019129160 | 2019-10-29 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112748559A CN112748559A (en) | 2021-05-04 |
| CN112748559B true CN112748559B (en) | 2024-01-02 |
Family
ID=75378964
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011184461.0A Active CN112748559B (en) | 2019-10-29 | 2020-10-29 | Microscope and method for displaying microscopic images and computer program product |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112748559B (en) |
| DE (1) | DE102020127118A1 (en) |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5589366B2 (en) * | 2009-11-27 | 2014-09-17 | ソニー株式会社 | Information processing apparatus, information processing method, and program thereof |
| DE102017101188B4 (en) * | 2017-01-23 | 2021-09-30 | Carl Zeiss Microscopy Gmbh | Microscope and method of microscopy on a sample |
-
2020
- 2020-10-15 DE DE102020127118.2A patent/DE102020127118A1/en active Pending
- 2020-10-29 CN CN202011184461.0A patent/CN112748559B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112748559A (en) | 2021-05-04 |
| DE102020127118A1 (en) | 2021-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9536702B2 (en) | Multi-beam particle microscope and method for operating same | |
| JP4426871B2 (en) | Image noise removal of FIB / SEM combined device | |
| JP7311487B2 (en) | Improved navigation for electron microscopy | |
| EP3915131A2 (en) | System comprising a multi-beam particle microscope and method for operating the same | |
| US8907280B1 (en) | Fast electron microscopy via compressive sensing | |
| EP0533330B1 (en) | A scanning microscope and a method of operating such a scanning microscope | |
| US20200025696A1 (en) | Adaptive specimen image acquisition using an artificial neural network | |
| EP2735867B1 (en) | Method of sampling a sample and displaying obtained information | |
| KR20200011870A (en) | Training an artificial neural network using simulated specimen images | |
| US10811217B2 (en) | Crystal orientation figure creating device, charged particle beam device, crystal orientation figure creating method, and program | |
| US20170323763A1 (en) | Charged Particle Beam Device | |
| US10614998B2 (en) | Charge reduction by digital image correlation | |
| JP2022501793A (en) | Image recording method using a particle microscope | |
| CN112748559B (en) | Microscope and method for displaying microscopic images and computer program product | |
| JP2022505064A (en) | Multi-beam electronic characterization tool with telecentric lighting | |
| US8153967B2 (en) | Method of generating particle beam images using a particle beam apparatus | |
| WO2019152585A2 (en) | Orientation determination and mapping by stage rocking electron channeling and imaging reconstruction | |
| WO2023053237A1 (en) | Charged particle beam device | |
| KR20210147917A (en) | Method and system for dynamic band contrast imaging | |
| US20060169894A1 (en) | Method of forming images in a scanning electron microscope | |
| US11982634B2 (en) | Adaptive specimen image acquisition | |
| JPH10162766A (en) | Focusing ion beam work observing device | |
| Browning et al. | Controlling the Dose Distribution in Scanning Transmission Electron Microscope (STEM) Images by Subsampling and Inpainting | |
| EP4343812A1 (en) | Data processing device and method, charged particle assessment system and method | |
| Höschen et al. | Dynamic implantation–an improved approach for a large area SIMS measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |