CN112396608A - Biological tissue electron microscope image correction method, system and device based on X-ray image - Google Patents
Biological tissue electron microscope image correction method, system and device based on X-ray image Download PDFInfo
- Publication number
- CN112396608A CN112396608A CN202011372160.0A CN202011372160A CN112396608A CN 112396608 A CN112396608 A CN 112396608A CN 202011372160 A CN202011372160 A CN 202011372160A CN 112396608 A CN112396608 A CN 112396608A
- Authority
- CN
- China
- Prior art keywords
- electron microscope
- image sequence
- blood vessel
- image
- mass center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001000 micrograph Methods 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012937 correction Methods 0.000 title claims abstract description 40
- 238000013507 mapping Methods 0.000 claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 6
- 230000011218 segmentation Effects 0.000 claims abstract description 4
- 210000004204 blood vessel Anatomy 0.000 claims description 107
- 210000004027 cell Anatomy 0.000 claims description 87
- 210000005056 cell body Anatomy 0.000 claims description 25
- 238000003384 imaging method Methods 0.000 claims description 12
- 238000003709 image segmentation Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 7
- 230000037303 wrinkles Effects 0.000 abstract description 2
- 239000000523 sample Substances 0.000 description 5
- 239000012472 biological sample Substances 0.000 description 4
- 238000003325 tomography Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005013 brain tissue Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011109 contamination Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
- G06T2207/10061—Microscopic image from scanning electron microscope
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the field of image processing, and particularly relates to a biological tissue electron microscope image correction method, a biological tissue electron microscope image correction system and a biological tissue electron microscope image correction device based on an X-ray image, aiming at solving the problem of difficult sequence image registration caused by distortion, wrinkles, pollution and damage during biological tissue segmentation. The invention comprises the following steps: the method comprises the steps of obtaining an X-ray image sequence through an X-ray microscope, cutting biological tissues into biological slices, obtaining an electron microscope image sequence through an electron microscope based on the biological slices, dividing the X-ray image and the electron microscope image into a cell mass center and a vessel bifurcation, matching the cell mass center and the vessel bifurcation of the X-ray image and the electron microscope image, obtaining a mapping relation according to a matching point pair successfully matched, and correcting the electron microscope image sequence according to the mapping relation. The invention can eliminate the influence caused by distortion, fold, pollution and damage in the process of biological slicing, and accurately obtain the three-dimensional structure information of the biological tissue with nanometer resolution.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a biological tissue electron microscope image correction method, system and device based on an X-ray image.
Background
The biological tissue three-dimensional reconstruction technology based on continuous slicing and electron microscope imaging can give consideration to both the scale and the resolution, and is an effective means for reconstructing the microstructure of the biological tissue. According to the method, a biological sample is cut into sequence slices with the thickness of 30-70 nanometers by using an ultrathin slice device, the sequence slices are placed into an electron microscope to carry out 2-5 nanometer resolution high-precision imaging on the slices, and finally, the three-dimensional electron microscope imaging result of the biological sample is recovered in an image registration mode. The disadvantage of this method is that there is a loss of structural information in the slice direction due to slice thickness limitations compared to nanoscale resolution on the imaging plane. In addition, slice distortion, wrinkling, contamination and breakage further destroy the original structure of the sample, making registration of the sequence images very difficult. At present, although the result obtained by the existing sequence slice three-dimensional electron microscope registration method can maintain the continuity of the form to a certain degree, due to the change of the three-dimensional structure of the biological sample, the corresponding point between adjacent slices obtained by adopting the similarity measurement is not necessarily the real corresponding point, and the difference between the registration result and the actual structure of the sample cannot be ensured within a reasonable range, which also leads some students to question the precision representation of the biological tissue three-dimensional reconstruction result based on the continuous slice and the electron microscope imaging, and limits the further popularization and application of the method.
With the development of the X-ray tomography technology, it becomes possible to observe three-dimensional morphological structures inside a sample without loss at high resolution. The imaging mode can avoid the damage to the biological sample, can obtain the three-dimensional fine structure in the tissue under micron resolution, and an X-ray microscope at a laboratory level can even achieve the same resolution imaging result of submicron resolution, thereby promoting the three-dimensional structure research of the biological tissue.
At present, in the field of biological tissue three-dimensional electron microscope reconstruction, an X-ray tomography technology is mostly used for quality screening of a three-dimensional electron microscope imaged sample and improving photoelectric correlation efficiency. The applications are still preliminary, the technical advantages of X-ray three-dimensional images with the same resolution and lossless imaging are not fully exerted, and only the X-ray microscope is taken as an auxiliary tool to improve the efficiency of biological tissue three-dimensional electron microscope reconstruction. Aiming at the problem that the registration result of a three-dimensional electron microscope of a biological tissue sequence slice cannot reproduce the actual structure of a sample, the invention uses an X-ray tomography image sequence as a reference to carry out nonlinear correction on the registration result of the three-dimensional electron microscope of the sequence slice so as to accurately obtain a three-dimensional image of a biological tissue with nanometer resolution.
Disclosure of Invention
In order to solve the above-mentioned problem in the prior art, that is, the problem that the sequence image registration is difficult due to distortion, wrinkle, pollution and damage caused by cutting the biological tissue, the present invention provides a biological tissue electron microscope image correction method based on an X-ray image, the method includes:
step S10, acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
step S20, cutting the biological tissue into biological slices, and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
step S30, segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binary image sequence;
segmenting based on the electron microscope image sequence to obtain a second cell soma image sequence and a second blood vessel binaryzation image sequence;
step S40, calculating and recording a first cell mass center set and a second cell mass center set based on the first cell body image sequence and the second cell body image sequence;
performing image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
step S50, point cloud matching is carried out on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and the first blood vessel bifurcation point set and the second blood vessel bifurcation point set are matched to generate a second matching point pair set;
and step S60, acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the center of mass corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
Further, step S50 includes: through a point cloud matching method, if the first cell mass center and the second cell mass center belong to the same cell mass center, setting the corresponding first cell mass center and the second cell mass center as a first matching point pair set; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set.
Further, the mapping relation is calculated by:
step B10, calculating the proportion between the physical sizes of the biological tissues contained in the single pixel based on the X-ray image sequence and the electron microscope image sequence;
step B20, adjusting a first cell centroid coordinate and a first vessel bifurcation coordinate in the X-ray image sequence based on the ratio between the single-pixel physical dimensions;
and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
Further, the method for generating the corrected three-dimensional electron microscope image by correcting the electron microscope image sequence by taking the X-ray image sequence as a standard comprises the following steps: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
In another aspect of the present invention, a biological tissue electron microscope image correction system based on X-ray image is provided, the system includes: the system comprises an X-ray image sequence acquisition module, an electron microscope image acquisition module, an image segmentation module, a key point coordinate acquisition module, a key point pairing module and a key point correction module;
the X-ray image sequence acquisition module is used for acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
the electron microscope image acquisition module is used for cutting the biological tissue into biological slices and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
the image segmentation module is used for segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binaryzation image sequence;
the image sequence of the electron microscope is further used for obtaining a second cell soma image sequence and a second blood vessel binaryzation image sequence by segmentation based on the image sequence of the electron microscope;
the key point coordinate acquisition module is used for calculating a first cell mass center set and a second cell mass center set based on the first cell mass image sequence and the second cell mass image sequence and recording mass center three-dimensional coordinates;
the image refinement method is also used for carrying out image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first blood vessel bifurcation point set and a second blood vessel bifurcation point set based on the first blood vessel network skeleton image sequence and the second blood vessel network skeleton image sequence and recording three-dimensional coordinates of the blood vessel bifurcation points;
the key point pairing module is used for performing point cloud matching on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and matching the first blood vessel bifurcation point set and the second blood vessel bifurcation point set to generate a second matching point pair set;
and the key point correction module is used for acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the mass center corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
Further, the key point pairing module is configured to, by using a point cloud matching method, set the corresponding first cell mass center and second cell mass center as a first matching point pair set if the first cell mass center and the second cell mass center belong to the same cell mass center; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set.
Further, the key point correction module comprises a mapping relation generation unit;
the mapping relation generating unit is used for calculating the proportion between the physical sizes of the biological tissues contained in the single pixel based on the X-ray image sequence and the electron microscope image sequence;
further for adjusting a first cell centroid coordinate and a first vessel bifurcation coordinate in the sequence of X-ray images based on a ratio between the single-pixel physical dimensions; and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
Further, the keypoint correction module includes a transformation correction unit, configured to correct the electron microscope image sequence using the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image, where the method includes: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
In a third aspect of the present invention, a storage device is provided, in which a plurality of programs are stored, the programs being suitable for being loaded and executed by a processor to implement the above-mentioned biological tissue electron microscope image correction method based on X-ray images.
In a fourth aspect of the present invention, a processing apparatus is provided, which includes a processor, a storage device; the processor is suitable for executing various programs; the storage device is suitable for storing a plurality of programs; the program is suitable for being loaded and executed by a processor to realize the biological tissue electron microscope image correction method based on the X-ray image.
The invention has the beneficial effects that:
(1) according to the biological tissue electron microscope image correction method based on the X-ray image, the electron microscope image sequence of the biological section is guided to correct by taking the X-ray image sequence as the template, and the three-dimensional structure information of the biological tissue with the nanometer resolution can be accurately obtained;
(2) according to the biological tissue electron microscope image correction method based on the X-ray image, the X-ray image sequence is obtained before slicing, and the key points are selected to be matched with the key points of the electron microscope image after slicing, so that the mapping relation is generated, and the intact biological tissue structure can be restored under the condition of high resolution.
(3) The biological tissue electron microscope image correction method based on the X-ray image searches the cell mass center and the bifurcation of the blood vessel in the X-ray image and the electron microscope image, selects the key points, and then matches the key points, and is particularly applied to complex biological tissues, particularly brain tissues, and can restore more accurate three-dimensional structures of the biological tissues, so that the generated biological electron microscope image is more accurate and real.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a schematic flow chart of an embodiment of a method for correcting an electron microscope image of a biological tissue based on an X-ray image according to the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
The invention provides a biological tissue electron microscope image correction method based on an X-ray image, which comprises the steps of S10-S60;
step S10, acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
step S20, cutting the biological tissue into biological slices, and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
step S30, segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binary image sequence;
segmenting based on the electron microscope image sequence to obtain a second cell soma image sequence and a second blood vessel binaryzation image sequence;
step S40, calculating a first cell centroid set and a second cell centroid set based on the first cell image sequence and the second cell image sequence;
performing image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
step S50, point cloud matching is carried out on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and the first blood vessel bifurcation point set and the second blood vessel bifurcation point set are matched to generate a second matching point pair set;
and step S60, acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the center of mass corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
In order to more clearly describe the method for correcting the electron microscope image of biological tissue based on the X-ray image, the following describes the steps in the embodiment of the method of the present invention in detail with reference to fig. 1.
The method for correcting the biological tissue electron microscope image based on the X-ray image comprises the following steps of S10-S60, wherein the steps are described in detail as follows:
step S10, acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
all data of an X-ray image sequence of biological tissues acquired under an X-ray microscope are lossless with the same resolution, and the real structural information of the biological tissues with micron resolution is reflected.
Step S20, cutting the biological tissue into biological slices, and acquiring an electron microscope image sequence based on the biological slices through an electron microscope; the biological slice is a serial slice which is cut into 30 to 70 nanometers thick by an ultrathin slice device, the serial slice is placed in an electron microscope to carry out 2 to 5 nanometers resolution high-precision imaging on the slice, and finally, a serial slice three-dimensional electron microscope image of the biological tissue is obtained in an image registration mode; the image registration method used in the step is to align the two-dimensional electron microscope image into a three-dimensional image;
before slicing, a thickness value such as 40nm is set, the ultrathin slicing device slices the biological tissue according to the set thickness, but errors usually exist in actual operation, the errors are in a range of plus or minus 2nm, and 30-70 nm is a numerical value selection range in the slicing thickness setting of the current common slicing mode microscopic imaging.
Step S30, segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binary image sequence;
segmenting based on the electron microscope image sequence to obtain a second cell soma image sequence and a second blood vessel binaryzation image sequence;
step S40, calculating a first cell centroid set and a second cell centroid set based on the first cell image sequence and the second cell image sequence;
performing image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
in the embodiment, the centroid method and the Gaussian fitting method can be adopted to calculate the centroid of the image; calculating a bifurcation point of the blood vessel by a line segment fitting method and a line segment intersection point method;
step S50, point cloud matching is carried out on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and the first blood vessel bifurcation point set and the second blood vessel bifurcation point set are matched to generate a second matching point pair set;
in this embodiment, a point cloud matching method is used to select a matching point pair, specifically, a transformation relation is found based on a target point set and a source point set, so that the difference between the transformed source point set and the target point set is minimized. It is preferable that the squared euclidean distance between the transformed pairs of points be set to less than
Then, the pair of points is identified as a matching pair of points belonging to the same cell body or the same vessel bifurcation.
In this embodiment, the point cloud matching method is that, if the first cell mass center and the second cell mass center belong to the same cell mass center, the corresponding first cell mass center and the second cell mass center are set as a first matching point pair set; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set. .
The matching in step S50 refers to registration between the electron microscope image blocks in three dimensions and the three-dimensional X-ray image blocks.
And step S60, acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the center of mass corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
In this embodiment, the calculation method of the mapping relationship is as follows:
a step B10 of calculating the ratio between the physical dimensions of the biological tissue contained in a single pixel, based on said sequence of X-ray images and sequence of electron microscope images;
in this embodiment, the resolution of the X-ray image is different from the resolution of the physical voxel of the electron microscope image, for example, the resolution of the X-ray image is 1 micron, and the xy resolution of the electron microscope image is 5 nm; the registration of the foregoing step S20 is simply the determination of matching and deformation relationships at the pixel level, and the step B10 here adjusts according to the ratio of their actual physical dimensions.
A step B20 of adjusting a first coordinates of the centroid of the cell volume and a first coordinates of the bifurcation of the blood vessel in said sequence of X-ray images, based on the ratio between the physical dimensions of the biological tissue contained by said single pixel;
and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
In this embodiment, the method for generating a corrected three-dimensional electron microscope image by correcting the electron microscope image sequence with the X-ray image sequence as a standard includes: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
The biological tissue electron microscope image correction system based on the X-ray image comprises an X-ray image sequence acquisition module, an electron microscope image acquisition module, an image segmentation module, a key point coordinate acquisition module, a key point pairing module and a key point correction module;
the X-ray image sequence acquisition module is used for acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
the electron microscope image acquisition module is used for cutting the biological tissue into biological slices and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
the image segmentation module is used for segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binaryzation image sequence;
the image sequence of the electron microscope is further used for obtaining a second cell soma image sequence and a second blood vessel binaryzation image sequence by segmentation based on the image sequence of the electron microscope;
the key point coordinate acquisition module is used for calculating a first cell body mass center set and a second cell body mass center set based on the first cell body image sequence and the second cell body image sequence;
the image refinement method is also used for carrying out image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
the key point pairing module is used for performing point cloud matching on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and matching the first blood vessel bifurcation point set and the second blood vessel bifurcation point set to generate a second matching point pair set;
in this embodiment, the key point pairing module is configured to, by using a point cloud matching method, if a first cell mass center and a second cell mass center belong to the same cell mass center, set the corresponding first cell mass center and second cell mass center as a first matching point pair set; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set.
And the key point correction module is used for acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the mass center corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
In this embodiment, the keypoint correction module includes a mapping relationship generation unit;
the mapping relation generating unit is used for calculating the proportion between the physical sizes of the biological tissues contained in the single pixel based on the X-ray image sequence and the electron microscope image sequence;
further for adjusting a first cell centroid coordinate and a first vessel bifurcation coordinate in the sequence of X-ray imaging images based on a ratio between the single-pixel physical dimensions; and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
In this embodiment, the keypoint correction module includes a transformation correction unit, configured to correct the electron microscope image sequence using the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image, where the method includes: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the biological tissue electron microscope image correction system based on an X-ray image provided in the foregoing embodiment is only illustrated by the division of the above functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
A storage device according to a third embodiment of the present invention stores a plurality of programs, and the programs are suitable for being loaded and executed by a processor to realize the above-mentioned method for correcting an image of a biological tissue electron microscope based on an X-ray image.
A processing apparatus according to a fourth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is suitable for being loaded and executed by a processor to realize the biological tissue electron microscope image correction method based on the X-ray image.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A biological tissue electron microscope image correction method based on an X-ray image is characterized by comprising the following steps:
step S10, acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
step S20, cutting the biological tissue into biological slices, and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
step S30, segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binary image sequence;
segmenting based on the electron microscope image sequence to obtain a second cell soma image sequence and a second blood vessel binaryzation image sequence;
step S40, calculating a first cell centroid set and a second cell centroid set based on the first cell image sequence and the second cell image sequence;
performing image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
step S50, point cloud matching is carried out on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and the first blood vessel bifurcation point set and the second blood vessel bifurcation point set are matched to generate a second matching point pair set;
and step S60, acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the center of mass corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
2. The method for correcting an electron microscope image of biological tissue based on an X-ray image according to claim 1, wherein the step S50 includes: the point cloud matching method comprises the steps that if the first cell mass center and the second cell mass center belong to the same cell mass center, the corresponding first cell mass center and the second cell mass center are set as a first matching point pair set; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set.
3. The method for correcting the biological tissue electron microscope image based on the X-ray image according to claim 1, wherein the mapping relation is calculated by:
a step B10 of calculating the ratio between the physical dimensions of the biological tissue contained in a single pixel, based on said sequence of X-ray images and sequence of electron microscope images;
a step B20 of adjusting a first coordinates of the centroid of the cell volume and a first coordinates of the bifurcation of the blood vessel in the sequence of X-ray images, based on the ratio between the physical dimensions of the biological tissue contained by the single pixel;
and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
4. The method for correcting the biological tissue electron microscope image based on the X-ray image according to claim 3, characterized in that the corrected three-dimensional electron microscope image is generated by correcting the electron microscope image sequence with the X-ray image sequence as a standard, and the method comprises: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
5. A system for correcting an image of a biological tissue electron microscope based on an X-ray image, the system comprising: the system comprises an X-ray image sequence acquisition module, an electron microscope image acquisition module, an image segmentation module, a key point coordinate acquisition module, a key point pairing module and a key point correction module;
the X-ray image sequence acquisition module is used for acquiring an X-ray image sequence of the biological tissue through an X-ray microscope;
the electron microscope image acquisition module is used for cutting the biological tissue into biological slices and acquiring an electron microscope image sequence based on the biological slices through an electron microscope;
the image segmentation module is used for segmenting based on the X-ray image sequence to obtain a first cell body image sequence and a first blood vessel binaryzation image sequence;
the image sequence of the electron microscope is further used for obtaining a second cell soma image sequence and a second blood vessel binaryzation image sequence by segmentation based on the image sequence of the electron microscope;
the key point coordinate acquisition module is used for calculating a first cell body mass center set and a second cell body mass center set based on the first cell body image sequence and the second cell body image sequence;
the image refinement method is also used for carrying out image refinement operation on the first blood vessel binarization image sequence and the second blood vessel binarization image sequence to obtain a first blood vessel network skeleton image sequence and a second blood vessel network skeleton image sequence; searching a first vessel bifurcation point set and a second vessel bifurcation point set based on the first vessel network skeleton image sequence and the second vessel network skeleton image sequence;
the key point pairing module is used for performing point cloud matching on the first cell mass center set and the second cell mass center set to generate a first matching point pair set, and matching the first blood vessel bifurcation point set and the second blood vessel bifurcation point set to generate a second matching point pair set;
and the key point correction module is used for acquiring the mapping relation of the matching points based on the three-dimensional coordinates of the mass center corresponding to the first matching point pair set and the three-dimensional coordinates of the vessel bifurcation corresponding to the second matching point pair set, and correcting the electron microscope image sequence by taking the X-ray image sequence as a standard to generate a corrected three-dimensional electron microscope image.
6. The system for correcting an image of a biological tissue electron microscope based on an X-ray image is characterized in that the key point matching module is used for setting the corresponding first cell mass center and the second cell mass center as a first matching point pair set if the first cell mass center and the second cell mass center belong to the same cell mass center by a point cloud matching method; and if the first blood vessel bifurcation point and the second blood vessel bifurcation point belong to the same blood vessel network bifurcation point, setting the corresponding first blood vessel bifurcation point and the second blood vessel bifurcation point as a second matching point set.
7. The system for correcting biological tissue electron microscope images based on X-ray images is characterized in that the key point correction module comprises a mapping relation generation unit;
the mapping relation generating unit is used for calculating the proportion between the physical sizes of the biological tissues contained in the single pixel based on the X-ray image sequence and the electron microscope image sequence;
further for adjusting a first cell centroid coordinate and a first vessel bifurcation coordinate in the sequence of X-ray imaging images based on a ratio between physical dimensions of biological tissue contained by the single pixel; and the mapping relation is that the adjusted first cell mass center coordinate and the adjusted first blood vessel bifurcation point coordinate are used as correction positions of the second cell mass center coordinate and the second blood vessel bifurcation point coordinate.
8. The system according to claim 7, wherein the keypoint correction module comprises a transformation correction unit for correcting the sequence of electron microscope images to generate a corrected three-dimensional electron microscope image based on the sequence of X-ray images, and the method comprises: and deforming the electron microscope image sequence by adopting a three-dimensional image deformation method based on the control points, so that the second cell mass center coordinate and the second vessel bifurcation coordinate are in corrected positions, generating a corrected three-dimensional electron microscope image sequence, and generating a corrected three-dimensional electron microscope image according to the corrected electron microscope image sequence.
9. A storage device, in which a plurality of programs are stored, wherein the programs are adapted to be loaded and executed by a processor to implement the method for correcting an image of a biological tissue electron microscope based on an X-ray image according to any one of claims 1 to 4.
10. A processing apparatus comprising a processor adapted to execute programs; and a storage device adapted to store a plurality of programs; characterized in that said program is adapted to be loaded and executed by a processor to implement the method for correcting an image of a biological tissue electron microscope based on an X-ray image according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011372160.0A CN112396608B (en) | 2020-11-30 | 2020-11-30 | Biological tissue electron microscope image correction method, system and device based on X-ray image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011372160.0A CN112396608B (en) | 2020-11-30 | 2020-11-30 | Biological tissue electron microscope image correction method, system and device based on X-ray image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112396608A true CN112396608A (en) | 2021-02-23 |
| CN112396608B CN112396608B (en) | 2021-05-04 |
Family
ID=74605637
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011372160.0A Active CN112396608B (en) | 2020-11-30 | 2020-11-30 | Biological tissue electron microscope image correction method, system and device based on X-ray image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112396608B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113192137A (en) * | 2021-04-27 | 2021-07-30 | 北京阅影科技有限公司 | Light source correction method and device |
| CN117611524A (en) * | 2023-10-26 | 2024-02-27 | 北京声迅电子股份有限公司 | Express item security inspection method based on multi-source image |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545567A (en) * | 2017-07-31 | 2018-01-05 | 中国科学院自动化研究所 | The method for registering and device of biological tissue's sequence section micro-image |
| CN108038874A (en) * | 2017-12-01 | 2018-05-15 | 中国科学院自动化研究所 | Towards the real-time registration apparatus of scanning electron microscope image and method of sequence section |
| CN111292310A (en) * | 2020-02-14 | 2020-06-16 | 广州再生医学与健康广东省实验室 | Method and device for acquiring biological tissue image, electronic equipment and storage medium |
| KR20200106763A (en) * | 2019-03-05 | 2020-09-15 | 서울대학교산학협력단 | Method for analyzing electron microscopy image |
-
2020
- 2020-11-30 CN CN202011372160.0A patent/CN112396608B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107545567A (en) * | 2017-07-31 | 2018-01-05 | 中国科学院自动化研究所 | The method for registering and device of biological tissue's sequence section micro-image |
| CN108038874A (en) * | 2017-12-01 | 2018-05-15 | 中国科学院自动化研究所 | Towards the real-time registration apparatus of scanning electron microscope image and method of sequence section |
| KR20200106763A (en) * | 2019-03-05 | 2020-09-15 | 서울대학교산학협력단 | Method for analyzing electron microscopy image |
| CN111292310A (en) * | 2020-02-14 | 2020-06-16 | 广州再生医学与健康广东省实验室 | Method and device for acquiring biological tissue image, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| SCHOLL, A.等: "Hartmann characterization of the PEEM-3 aberration-corrected X-ray photoemission electron microscope", 《ULTRAMICROSCOPY》 * |
| 宗云兵: "同步辐射X射线多模式成像及其在科技考古与肿瘤学中的应用研究", 《中国博士学位论文全文数据库 基础科学辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113192137A (en) * | 2021-04-27 | 2021-07-30 | 北京阅影科技有限公司 | Light source correction method and device |
| CN117611524A (en) * | 2023-10-26 | 2024-02-27 | 北京声迅电子股份有限公司 | Express item security inspection method based on multi-source image |
| CN117611524B (en) * | 2023-10-26 | 2024-05-31 | 北京声迅电子股份有限公司 | Express item security inspection method based on multi-source image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112396608B (en) | 2021-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107545567B (en) | Registration method and device for biological tissue sequence section microscopic image | |
| CN112396608B (en) | Biological tissue electron microscope image correction method, system and device based on X-ray image | |
| Gürsoy et al. | Rapid alignment of nanotomography data using joint iterative reconstruction and reprojection | |
| Lawrence et al. | Transform-based backprojection for volume reconstruction of large format electron microscope tilt series | |
| Ress et al. | Automatic acquisition of fiducial markers and alignment of images in tilt series for electron tomography | |
| EP2648206B1 (en) | Transmission electron microscope for reconstructing a three-dimensional structure and transmission electron microscope image reconstruction method | |
| CN106570484B (en) | MIcrosope image acquisition method based on sequence section | |
| US8554016B2 (en) | Image registration system and method for registering images for deformable surfaces | |
| CN110133014B (en) | Method and system for detecting internal defects of chip | |
| CN110736747A (en) | cell liquid based smear under-mirror positioning method and system | |
| CN110956630A (en) | Method, device and system for detecting plane printing defects | |
| Winkler et al. | Marker-free dual-axis tilt series alignment | |
| CN115578320A (en) | Full-automatic space registration method and system for orthopedic surgery robot | |
| Fox et al. | Complementary use of optical metrology and x-ray computed tomography for surface finish and defect detection in laser powder bed fusion additive manufacturing | |
| CN106023094B (en) | Bone tissue microstructure repair system based on image and its restorative procedure | |
| CN118115694A (en) | Reconstruction method of three-dimensional microscopic image of three-dimensional biological sample | |
| Lu et al. | EMDiffuse: a diffusion-based deep learning method augmenting ultrastructural imaging and volume electron microscopy | |
| CN114693959A (en) | Image target matching positioning method, storage medium and computer | |
| Radermacher et al. | Three-dimensional reconstruction of single particles in electron microscopy | |
| CN115880305A (en) | Visual detection method and device for surface defects of shaft parts | |
| Nguyen et al. | The alignment and fusion of multimodal 3D serial sectioning datasets | |
| CN114049252A (en) | Scanning electron microscope three-dimensional image acquisition system and method for sequence slicing | |
| Lawrence et al. | Electron tomography and multiscale biology | |
| Chen et al. | Morphology-retained non-linear image registration of serial electron microscopy sections | |
| CN114219702A (en) | Different-staining pathological section image matching and displaying method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |