CN115690183A - Image registration data processing method and system - Google Patents
Image registration data processing method and system Download PDFInfo
- Publication number
- CN115690183A CN115690183A CN202211472016.3A CN202211472016A CN115690183A CN 115690183 A CN115690183 A CN 115690183A CN 202211472016 A CN202211472016 A CN 202211472016A CN 115690183 A CN115690183 A CN 115690183A
- Authority
- CN
- China
- Prior art keywords
- image
- ghost
- images
- floating
- registration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention provides an image registration data processing method and system, relating to the technical field of image registration and obtaining a plurality of structural feature images in a plurality of monochromatic laser images; acquiring a plurality of ghost areas with overlarge eyeball movement according to the structural feature images, wherein each ghost area comprises a corresponding ghost floating image and a ghost reference image; acquiring a corresponding relation of the characteristic points; acquiring a plurality of pairs of feature points with accuracy greater than a preset threshold, and performing feature point matching processing to obtain a ghost processing floating image; carrying out pixel registration processing to obtain a region registration image; and carrying out alignment fusion processing on the multiple regional registration images, and combining the reference image and the floating image to obtain a registration image. The invention solves the technical problems of eye movement of a patient, inconsistent image offset and low whole image registration precision in the image acquisition process in the prior art, and achieves the technical effects of correcting partial image offset caused by the eye movement of the patient and improving real-time registration precision.
Description
Technical Field
The invention relates to the technical field of image registration, in particular to a method and a system for processing image registration data.
Background
With the change of life style and the increase of overuse of eyes, the demand for the treatment of ophthalmic diseases has been increasing and the market scale has been rapidly expanding in recent years. In the process of diagnosing ophthalmic diseases, medical staff check whether pathological changes exist in optic nerves, retinas and the like of eyegrounds through an eyeground camera because the blood vessels of the eyegrounds are the only blood vessels which can be directly observed by a human body through the body surface.
At present, by a laser scanning confocal imaging technology, a laser point light source, a retina and a point detector are positioned at a conjugate position by introducing a pinhole into an imaging focal plane, and stray light of a retina non-focal plane is effectively filtered. The optical tomography capability and high-resolution dynamic imaging which are not possessed by the fundus camera can be provided, and the transverse resolution reaches the micron level.
However, images obtained by monochromatic laser shooting are all gray-scale images, and the gray-scale images cannot represent fundus lesions. Thus, the same portion of the fundus is usually imaged by a plurality of laser sources, and a fundus color photograph is synthesized by an image processing technique. The monochromatic laser images are inconsistent in gray scale distribution, tissue response and the like because different tissues of the eyeground respond to laser with different wavelengths, and an image registration algorithm based on feature point matching is usually adopted to extract the same structural features of the two images for matching. The condition that partial double images can appear in the final synthesized fundus color photos can not ensure the color photo quality, and the characteristic point searching time is long due to overlarge offset, so that the real-time requirement can not be met. In the prior art, the technical problems of eye movement of a patient, inconsistent image offset and low whole image registration precision exist in the image acquisition process.
Disclosure of Invention
The application provides an image registration data processing method and system, which are used for solving the technical problems of eye movement of a patient, inconsistent image offset and low whole image registration precision in the image acquisition process in the prior art.
In view of the above, the present application provides an image registration data processing method and system.
In a first aspect of the present application, an image registration data processing method is provided, where the method includes: acquiring a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise a reference image and a floating image; acquiring a plurality of ghost areas with overlarge eyeball movement according to the plurality of structural feature images, wherein each ghost area comprises a corresponding ghost floating image and a corresponding ghost reference image; for each ghost region, acquiring a feature point correspondence relationship between the ghost floating image and a ghost reference image, wherein the feature point correspondence relationship comprises a plurality of pairs of feature points; acquiring a plurality of pairs of feature points of which the accuracy is greater than a preset threshold value in the feature point correspondence relationship, and performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image; carrying out pixel registration processing on the ghost floating image to obtain a region registration image; and carrying out alignment fusion processing on the registration images of the areas in the multiple ghost areas, and combining the reference image and the floating image to obtain a registration image.
In a second aspect of the present application, there is provided an image registration data processing system, the system comprising: a feature image obtaining module, configured to obtain a plurality of structural feature images in a plurality of monochromatic laser images, where the plurality of monochromatic laser images include a reference image and a floating image; a ghost region obtaining module, configured to obtain a plurality of ghost regions with excessive eye movement according to the plurality of structural feature images, where each ghost region includes a corresponding ghost floating image and a ghost reference image; the corresponding relation obtaining module is used for obtaining the corresponding relation of the characteristic points in the ghost floating image and the ghost reference image for each ghost area, wherein the corresponding relation of the characteristic points comprises a plurality of pairs of characteristic points; the floating image obtaining module is used for obtaining a plurality of pairs of feature points of which the accuracy is greater than a preset threshold value in the feature point corresponding relation, and performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image; the area image obtaining module is used for carrying out pixel registration processing on the ghost floating image to obtain an area registration image; a registered image obtaining module, configured to perform alignment fusion processing on the multiple region registered images in the multiple ghost regions, and obtain a registered image by combining the reference image and the floating image.
In a third aspect of the present application, there is provided a fundus laser camera comprising an image registration data processing system as described in the second aspect.
One or more technical solutions provided in the present application have at least the following technical effects or advantages:
the method provided by the embodiment of the application obtains a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise reference images and floating images, then obtains a plurality of ghost areas with overlarge eyeball movement according to the plurality of structural feature images, each ghost area comprises a corresponding ghost floating image and a corresponding ghost reference image, further obtains a feature point corresponding relation between the ghost floating image and the ghost reference image for each ghost area, wherein the feature point corresponding relation comprises a plurality of pairs of feature points, then obtains a plurality of pairs of feature points with accuracy greater than a preset threshold value in the feature point corresponding relation, performs feature point matching processing on the ghost floating images to obtain processed ghost floating images, further performs pixel registration processing on the processed ghost floating images to obtain area registration images, then performs alignment fusion processing on the plurality of area registration images in the plurality of ghost areas, and combines the reference images and the floating images to obtain the registration images. The technical effects of improving the efficiency of image registration and correcting the offset of the whole image caused by eye movement are achieved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Fig. 1 is a schematic flow chart of an image registration data processing method provided in the present application;
fig. 2 is a schematic flowchart of acquiring a plurality of structural feature images in a plurality of monochromatic laser images in an image registration data processing method provided by the present application;
fig. 3 is a schematic flowchart of acquiring multiple ghost image regions with excessive eye movement in an image registration data processing method provided by the present application;
fig. 4 is a schematic structural diagram of an image registration data processing system provided in the present application.
Description of reference numerals: the image registration method comprises a characteristic image obtaining module 11, a ghost image region obtaining module 12, a corresponding relation obtaining module 13, a floating image obtaining module 14, a region image obtaining module 15 and a registration image obtaining module 16.
Detailed Description
The application provides an image registration data processing method and system, which are used for solving the technical problems that in the prior art, in the image acquisition process, a patient moves eyes, the image offset is inconsistent, and the whole image registration precision is low. The method and the device have the advantages that partial image deviation caused by the eyeball motion of the patient is effectively corrected, the problem of double images after synthesis is effectively solved, and the whole image registration accuracy and efficiency are improved.
In the technical scheme of the invention, the data acquisition, storage, use, processing and the like all conform to relevant regulations of national laws and regulations.
In the following, the technical solutions in the present invention will be clearly and completely described with reference to the accompanying drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments of the present invention, and it should be understood that the present invention is not limited by the example embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention. It should be further noted that, for the convenience of description, only some but not all of the elements associated with the present invention are shown in the drawings.
Example one
As shown in fig. 1, the present application provides an image registration data processing method, wherein the method comprises:
step S100: acquiring a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise a reference image and a floating image;
further, as shown in fig. 2, a plurality of structural feature images in a plurality of monochromatic laser images are acquired, and step S100 in this embodiment of the present application further includes:
step S110: performing Gaussian filtering smoothing noise processing on the plurality of monochromatic laser images to obtain a plurality of noise reduction processing images;
step S120: judging whether the gray values of the plurality of noise reduction processing images are smaller than a preset gray value threshold value or not; if yes, the noise reduction processing image is processed by adopting an adaptive histogram enhancement algorithm and gamma curve stretching histogram processing, and if not, the processing is not carried out, and a plurality of processing images are obtained;
step S130: and carrying out fuzzy enhancement processing on the plurality of processed images to obtain a plurality of structural feature images.
Specifically, a fundus camera is used to acquire images of the fundus of a patient from a plurality of angles and at different positions by using a plurality of laser sources, and the plurality of monochromatic laser images are obtained. The plurality of monochromatic laser images are obtained by imaging the fundus for a plurality of times. The plurality of structural feature images are images reflecting structural features of blood vessels and optic disc information of the fundus from a plurality of different angles and different positions. The reference image refers to an image which is kept still in the process of carrying out image registration on the plurality of single-color laser images. The floating image refers to an image which is moved or rotated and changed in the process of carrying out image registration on the plurality of single-color laser images. The Gaussian filtering smooth noise processing is to perform weighted average on a plurality of monochromatic laser images respectively, each pixel point in the image is obtained by performing weighted average on the pixel point and other pixel values in the neighborhood, and therefore the accumulated transmitted error is corrected, noise is suppressed, and the plurality of noise reduction processing images are obtained. The plurality of noise reduction processed images are images obtained by processing noise in the images. The preset gray value threshold is the minimum value that the gray value of each pixel point in the preset image needs to meet, and is set by a worker, and the preset gray value threshold is not limited herein. When the gray value of the pixel point in the image is lower than the preset gray value threshold, the image at the moment can not clearly express the details of the image. The self-adaptive histogram enhancement algorithm processing is to comprehensively consider the entropy and brightness average value difference of an image, self-adaptively select a proper threshold value to divide the image into 2 sub-images to carry out double histogram equalization and gray level homogenization processing, thereby effectively enhancing the image and avoiding the over-enhancement phenomenon. And performing nonlinear operation on the gray value of the input image on the image during the gamma curve stretching histogram processing, so that the gray value of the output image and the gray value of the input image are in an exponential relationship, and the image is enhanced. The blurring processing is to superimpose the plurality of processed images to enhance the features of the images. And then extracting the plurality of structural feature images according to the result of the fuzzy enhancement processing.
Specifically, by performing gaussian filtering smoothing noise processing on the plurality of monochromatic laser images, it is possible to correct the accumulated error in the image by processing the inconsistency of the image grayscale detail distribution in the image. Furthermore, whether the quality of the processed images can meet the requirements or not can be obtained by judging the gray value of the plurality of noise reduction processing images, the images are processed in batches, the follow-up operation can be directly carried out according to the requirements, and the gray value of the images cannot be improved by carrying out image enhancement processing according to the requirements. Then, the obtained multiple processed images are subjected to fuzzy enhancement processing, the structural features in the images are extracted, and a cushion is laid for subsequent structural analysis and calibration.
Further, performing blur enhancement processing on the plurality of processed images, in step S130 of this embodiment of the present application, further includes:
step S131: performing fuzzy filter processing on the plurality of processed images to obtain a plurality of fuzzy processed images;
step S132: superposing the plurality of blurred processing images on the plurality of single-color laser images according to a first preset proportion to obtain a plurality of first enhanced processing images;
step S133: superposing the plurality of processed images and the plurality of first enhanced processed images according to a second preset proportion to obtain a plurality of second enhanced processed images;
step S134: obtaining a plurality of gray value thresholds according to the average gray value of the plurality of monochromatic laser images;
step S135: and extracting the plurality of second enhancement processing images according to the plurality of gray value thresholds to obtain the plurality of structural feature images.
Specifically, the blur filter processing of the plurality of processed images means removing high frequency components, such as noise and boundaries, in the images, thereby achieving the purpose of image blurring. The plurality of blurred images are images obtained by blurring the plurality of processed images so that the boundaries in the images are not so strong. The first preset proportion is the proportion of corresponding blurred images when a plurality of monochromatic laser images are enhanced, namely the number of blurred images corresponding to one monochromatic laser image. The plurality of first enhancement processing images are obtained by carrying out image enhancement on the monochromatic laser images. The second preset proportion is a preset superposition proportion between the images when the plurality of processing images are superposed on the plurality of first enhancement processing images. The second enhancement processing image is an image obtained by superposing the processing image and the enhancement processing image according to a preset proportion. The average gray value is obtained by respectively calculating the gray value corresponding to each pixel point in the plurality of single-color laser images and carrying out averaging processing. And then, determining the multiple gray value thresholds corresponding to the multiple monochromatic laser images according to the average gray value, thereby determining the corresponding gray value thresholds when the characteristics of different monochromatic laser images are extracted.
Specifically, the plurality of processed images are subjected to fuzzy processing, so that the boundary feeling of the images is not obvious enough, but larger and brighter pixel points in the images are reserved, and noise can be effectively removed. Therefore, the plurality of blurred images are superposed on the plurality of monochromatic laser images, so that the plurality of monochromatic laser images are subjected to image enhancement, and the plurality of first enhanced images are superposed with the plurality of processed images, so that the structure in the images is clearer. And then, extracting the second enhanced image according to the gray value thresholds, and extracting features according to different gray value thresholds, so as to obtain the structural feature images reflecting different structures. The technical effects of carrying out feature analysis on the images, obtaining a plurality of high-quality structural feature images and laying a cushion for subsequent image registration are achieved.
Step S200: acquiring a plurality of ghost areas with overlarge eyeball movement according to the structural feature images, wherein each ghost area comprises a corresponding ghost floating image and a ghost reference image;
further, as shown in fig. 3, a plurality of ghost areas with excessive eye movement are obtained according to the plurality of structural feature images, and step S200 in the embodiment of the present application further includes:
step S210: longitudinally partitioning the structural feature images into a plurality of regions to obtain a plurality of { block1, block2, block3, … };
step S220: according to the plurality of { block1, block2, block3, … }, calculating mutual information value difference values of each reference image and each floating image in the plurality of areas, and obtaining a plurality of mutual information value difference value sequences { d1, d2, d3, … };
step S230: and taking the areas when the mutual information value difference value is continuously smaller than or larger than a preset difference value threshold value as the plurality of ghost areas.
Specifically, the multiple ghost areas are areas in which images are overlapped and blurred due to eyeball motion in the process of image acquisition, and mutual information value differences corresponding to the areas continuously do not meet a preset difference threshold. The ghost floating image is an image that rotates or moves during image registration. The ghost reference image is a ghost image that remains stationary during image registration. The plurality of regions are obtained by longitudinally partitioning the plurality of structural feature images respectively. The mutual information value difference values between the reference image and the floating image in each region are arranged according to the sequence of the regions, and the mutual information value difference value sequences are in one-to-one correspondence with the regions.
Specifically, the structural feature images are divided in equal proportion according to the longitudinal distance to obtain a plurality of regions, namely a plurality of { block1, block2, block3, … }. And then, according to the mutual information value between the reference image and the floating image corresponding to each region, calculating a difference value, analyzing the difference value of the mutual information values, and when the difference value of the mutual information values is continuously smaller than or larger than a preset difference threshold value, indicating that the difference degree of the mutual information values between the reference image and the floating image of the corresponding region at the moment can not meet the requirement, so that the ghost phenomenon occurs. Therefore, the technical effects of accurately positioning the shadowed area and improving the image registration efficiency are achieved.
Step S300: for each ghost area, acquiring a corresponding relation of feature points in the ghost floating image and a ghost reference image, wherein the corresponding relation of the feature points comprises a plurality of pairs of feature points;
further, for each ghost region, obtaining a corresponding relationship of feature points in the ghost floating image and the ghost reference image, where step S300 in the embodiment of the present application further includes:
step S310: for each ghost area, extracting the characteristic points of a ghost reference image based on a characteristic point extraction algorithm to obtain a reference characteristic point set;
step S320: constructing a reference image pyramid and a floating image pyramid by respectively carrying out down-sampling on a ghost reference image and a ghost floating image, wherein the ghost reference image and the ghost floating image are respectively positioned at the bottom layers of the reference image pyramid and the floating image pyramid, and an upper layer image is obtained by carrying out 1/2 proportion down-sampling on the basis of a lower layer image;
step S330: corresponding the plurality of feature points in the reference feature point set to the highest layer in the reference image pyramid, and then searching the plurality of feature points in the corresponding layer of the floating image pyramid to obtain a highest-layer floating feature point set;
step S340: according to the highest layer floating feature point set, pre-translation and residual calculation are carried out on the next layer floating image, and a first feature point corresponding relation of the next layer floating image is obtained;
step S350: and performing iterative computation on each layer of floating images based on the first feature point corresponding relation to obtain the feature point corresponding relation.
Specifically, the feature point extraction algorithm is an algorithm for extracting extreme points, that is, key points in an image to obtain directions of feature points. The reference characteristic point set refers to characteristic points serving as image registration references in the ghost reference images. The step of constructing the reference image pyramid by downsampling is to reduce the original image, reduce the dimensionality of features on the basis of keeping effective information, reduce the reference image, and perform continuous downsampling to obtain multiple layers, wherein the image is continuously reduced from the bottom to the top to form the reference image pyramid. Illustratively, a reference image is down-sampled, and after the down-sampling step is performed, the image becomes one-half the original length and width, and the entire image becomes one-fourth of the original length and width. The floating image pyramid is obtained by reducing the original image, reducing the dimension of the features on the basis of keeping effective information, reducing the floating image, and continuously performing downward iterative sampling to obtain multiple layers, wherein the image is continuously reduced from the bottom to the top.
Specifically, a plurality of feature points in the reference feature point set are set in the highest layer of the reference image pyramid through continuous iterative sampling, and then, according to the one-to-one correspondence relationship between the reference image and the floating image, searching is performed in the corresponding layer of the floating image pyramid, so that points corresponding to the plurality of feature points in the highest layer of the reference image pyramid are found, and the highest-layer floating feature point set is obtained. And the highest-layer floating feature point set is a basic feature point for adjusting the image.
Specifically, according to the feature points in the highest-layer floating feature point set, pre-translation and residual calculation are performed on the next-layer floating image, so that an accurate feature point corresponding relation in the next-layer floating image is obtained. And the first feature point corresponding relation indicates the feature point corresponding situation between the reference image and the floating image, so that the reference image pyramid is iterated layer by layer until the bottom of the reference image pyramid is reached, and the iteration is stopped, so that the corresponding relation between the feature points of the original reference image and the floating image, namely the feature point corresponding relation, is obtained. The technical effects of accurately registering the images for bedding and solving the problem that corresponding characteristic points cannot be found due to overlarge offset of the reference image and the floating image are achieved.
Step S400: acquiring a plurality of pairs of feature points of which the accuracy is greater than a preset threshold value in the feature point correspondence relationship, and performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image;
further, a plurality of pairs of feature points of which the accuracy is greater than a preset threshold in the feature point correspondence relationship are obtained, and feature point matching processing is performed on the ghost floating image, in the step S400 in the embodiment of the present application, the method further includes:
step S410: based on RANSAC algorithm, screening a plurality of pairs of feature points in the feature point corresponding relation to obtain a plurality of pairs of feature points with accuracy greater than the preset threshold;
step S420: and performing affine transformation processing on the ghost floating image based on the change matrix and the plurality of pairs of feature points to obtain the processed ghost floating image.
Specifically, there are multiple pairs of feature points in the feature point correspondence relationship, the feature points are screened according to the RANSAC algorithm, the preset threshold is used as a screening target, and a plurality of pairs of feature points with accuracy greater than the preset threshold are collected. The change matrix is a change process matrix for adjusting the floating image, which is obtained according to the corresponding relation of the characteristic points. And carrying out affine transformation processing on the ghost floating image according to the plurality of pairs of feature points on the ghost floating image and the change matrix, adjusting the feature point positions on the ghost floating image through linear transformation, and further adjusting the ghost floating image to obtain the processed ghost floating image. The technical effect of carrying out transformation processing on the image so as to eliminate the influence of eye movement is achieved.
Step S500: performing pixel registration processing on the ghost floating image to obtain a region registration image;
further, the step S500 of the embodiment of the present application further includes performing pixel registration processing on the processed ghost floating image:
step S510: calculating pixel-by-pixel optical flow (flox, floy) of the ghost floating image and the ghost reference image based on a dense pyramid optical flow method;
step S520: and mapping the pixels in the ghost floating image onto the processed ghost floating image according to the pixel-by-pixel optical flow (flox, floy) to obtain the area registration image.
Specifically, the region registration image is an image obtained by performing pixel registration processing on a graph so as to supplement image details. The dense pyramid optical flow method is to obtain the pixel-by-pixel optical flow (flox, floy) by performing point-by-point matching on images and calculating the offset between each pixel point in the ghost floating image and the ghost reference image. The flox refers to the offset of the offset condition between the pixel points mapped on the x axis, and the floy refers to the offset of the offset condition between the pixel points mapped on the y axis. And then, moving pixel points in the ghost floating images according to the pixel-by-pixel luminous flux (flox, floy) and mapping the pixel points to the ghost floating images, so that the details are subjected to region registration to obtain the region registration images. The technical effect of high-quality and high-efficiency registration of the images is achieved.
Step S600: and carrying out alignment fusion processing on the registration images of the areas in the multiple ghost areas, and combining the reference image and the floating image to obtain a registration image.
Further, performing alignment fusion processing on the multiple region registration images in the multiple ghost regions, in step S600 in this embodiment of the present application, further includes:
step S610: carrying out alignment and fusion processing on the multiple region registration images to obtain a primary registration image;
step S620: extracting the edge missing part ROI of a plurality of region registration images in the preliminary registration image to obtain a plurality of ROIs;
step S630: performing Gaussian blur processing on the plurality of ROIs to obtain a plurality of blurred ROIs;
step S640: adjusting the gray distribution of a reference image according to the pixel distribution of the floating image, and overlapping the gray distribution of the reference image with the plurality of fuzzy ROIs according to a third preset proportion to obtain img1;
step S650: iterating the plurality of fuzzy ROIs and the floating image according to a third preset proportion to obtain img2;
step S660: and performing pixel-by-pixel superposition on the img1 and the img2 to the preliminary registration image to obtain the registration image.
Specifically, the registration image is an image obtained by performing high-precision registration on an image with low finished quality and eliminating a ghost image caused by eye movement. The alignment fusion processing refers to an operation of aligning the multiple region registration images according to a previous segmentation order and fusing image contents. The preliminary registration image is an image obtained by performing preliminary processing on a plurality of region registration images. And then, setting the missing part of the plurality of region registration images in the preliminary registration image after alignment and fusion as a region of interest (ROI). The multiple ROIs reflect the absence of the preliminary registered image. Furthermore, gaussian blur processing is carried out on the plurality of ROIs, noise and boundaries are eliminated, and the plurality of blurred ROIs are obtained. And then, adjusting the gray distribution of the reference image according to the pixel distribution of the floating image, adjusting the gray distribution of the reference image by taking the floating image as the reference, and superposing the reference image according to the plurality of fuzzy ROIs, wherein the third preset proportion is the proportion corresponding to the superposition of the reference image and the fuzzy ROIs, namely the quantity of the fuzzy ROIs superposed by one reference image. And then, iterating the floating image and the plurality of fuzzy ROIs according to a third preset proportion, and obtaining the img2. The img1 and the img2 are overlapped to the preliminary registration image pixel by pixel, and the edge missing part in the preliminary registration image is supplemented, so that the technical effect of improving the image precision is achieved.
The technical scheme provided by the invention at least has the following technical effects or advantages:
1. according to the method, a plurality of structural characteristic images in a plurality of monochromatic laser images are extracted, so that images capable of reflecting the structures of eyegrounds are obtained, including blood vessels and optic disc information structures, the structural characteristic images are deeply excavated, so that a plurality of ghost areas caused by eye movement are obtained, a down-sampling construction method is adopted, the corresponding relation of the characteristic points between the ghost floating images and the ghost reference images in each ghost area is collected, the target for providing basis for subsequent registration is achieved, then a plurality of pairs of characteristic points with the accuracy greater than a preset threshold value in the corresponding relation of the characteristic points are screened, the ghost floating images are subjected to characteristic point matching processing, processed ghost floating images are obtained, pixel registration processing is carried out on the processed ghost floating images, area registration images are obtained, then alignment and fusion processing is carried out on the registration images in the plurality of areas in the plurality of ghost areas, and the reference images and the floating images are combined to obtain the registration images. The technical effects of improving the accuracy of image registration, correcting image offset caused by eye movement, eliminating the influence of eye movement and efficiently finishing full-image high-precision real-time registration are achieved.
2. The method and the device have the advantages that the Gaussian filtering smoothing noise processing is carried out on the single-color laser images to obtain a plurality of images subjected to noise processing, the accuracy of subsequent analysis is improved, whether the gray values of the plurality of noise-reduction processed images are smaller than a preset gray value threshold value or not is judged, different processing is carried out on the images, if yes, the noise-reduction processed images are processed by adopting an adaptive histogram enhancement algorithm and a gamma curve stretching histogram, if not, the processing is not carried out to obtain a plurality of processed images, then the images are subjected to fuzzy enhancement processing to obtain a plurality of structural feature images. The technical effects of enhancing the image display effect and improving the accuracy and efficiency of subsequent image analysis are achieved.
Example two
As shown in fig. 4, in order to explain a technical solution of an image registration data processing method more clearly, an embodiment of the present application provides an image registration data processing system, which includes:
a feature image obtaining module 11, where the feature image obtaining module 11 is configured to obtain a plurality of structural feature images in a plurality of monochromatic laser images, where the plurality of monochromatic laser images include a reference image and a floating image;
a ghost region obtaining module 12, wherein the ghost region obtaining module 12 is configured to obtain a plurality of ghost regions with excessive eye movement according to the plurality of structural feature images, and each ghost region includes a corresponding ghost floating image and a ghost reference image;
a corresponding relation obtaining module 13, where the corresponding relation obtaining module 13 is configured to obtain, for each ghost region, a corresponding relation between feature points in the ghost floating image and the ghost reference image, where the corresponding relation between feature points includes multiple pairs of feature points;
the floating image obtaining module 14 is configured to obtain a plurality of pairs of feature points of which accuracy is greater than a preset threshold in the feature point correspondence relationship, perform feature point matching processing on the ghost floating image, and obtain a processed ghost floating image;
a region image obtaining module 15, where the region image obtaining module 15 is configured to perform pixel registration processing on the ghost floating image to obtain a region registration image;
a registered image obtaining module 16, where the registered image obtaining module 16 is configured to perform alignment fusion processing on the multiple region registered images in the multiple ghost regions, and obtain a registered image by combining the reference image and the floating image.
Further, the system further comprises:
a noise reduction image obtaining unit, configured to perform gaussian filtering smoothing noise processing on the plurality of monochromatic laser images to obtain a plurality of noise reduction processed images;
the grey value judging unit is used for judging whether the grey values of the plurality of noise reduction processing images are smaller than a preset grey value threshold value or not; if yes, the noise reduction processing image is processed by adopting an adaptive histogram enhancement algorithm and gamma curve stretching histogram processing, and if not, the processing is not carried out, and a plurality of processing images are obtained;
a structural feature image obtaining unit, configured to perform blur enhancement processing on the plurality of processed images to obtain the plurality of structural feature images.
Further, the system further comprises:
a blur-processed image obtaining unit configured to perform blur filter processing on the plurality of processed images to obtain a plurality of blur-processed images;
an enhanced processing image obtaining unit, configured to superimpose the plurality of blurred processing images onto the plurality of single-color laser images according to a first preset ratio to obtain a plurality of first enhanced processing images;
a second enhanced image obtaining unit, configured to superimpose the plurality of processed images and the plurality of first enhanced processed images according to a second preset ratio to obtain a plurality of second enhanced processed images;
a gray value threshold obtaining unit, configured to obtain multiple gray value thresholds according to average gray values of the multiple monochromatic laser images;
and the image extraction unit is used for extracting the plurality of second enhanced processing images according to the plurality of gray value thresholds to obtain the plurality of structural feature images.
Further, the system further comprises:
the region dividing unit is used for longitudinally dividing the structural feature images into a plurality of regions to obtain a plurality of { block1, block2, block3, … };
a difference sequence obtaining unit, configured to calculate, according to the plurality of { block1, block2, block3, … }, a mutual information value difference between each reference image and each floating image in the plurality of regions, and obtain a plurality of mutual information value difference sequences { d1, d2, d3, … };
a ghost region setting unit for, as the plurality of ghost regions, regions when a mutual information value difference value starts to be continuously smaller or larger than a preset difference value threshold.
Further, the system further comprises:
a reference feature set obtaining unit, configured to extract, for each ghost region, feature points of a ghost reference image based on a feature point extraction algorithm, and obtain a reference feature point set;
the image pyramid construction unit is used for respectively constructing a reference image pyramid and a floating image pyramid by downsampling a ghost reference image and a ghost floating image, wherein the ghost reference image and the ghost floating image are respectively positioned at the bottom layers of the reference image pyramid and the floating image pyramid, and an upper layer image is obtained by downsampling in a 1/2 ratio based on a lower layer image;
a highest-layer feature point obtaining unit, configured to correspond multiple feature points in the reference feature point set to a highest layer in the reference image pyramid, and then search for the multiple feature points in a corresponding layer of the floating image pyramid to obtain a highest-layer floating feature point set;
a corresponding relation obtaining unit, configured to perform pre-translation and residual calculation on the next-layer floating image according to the highest-layer floating feature point set, so as to obtain a first feature point corresponding relation of the next-layer floating image;
and the iterative calculation unit is used for carrying out iterative calculation on each layer of floating images based on the first feature point corresponding relation to obtain the feature point corresponding relation.
Further, the system further comprises:
the characteristic point screening unit is used for screening a plurality of pairs of characteristic points in the corresponding relation of the characteristic points based on an RANSAC algorithm to obtain the plurality of pairs of characteristic points with the accuracy greater than the preset threshold value;
an affine variation processing unit configured to perform affine transformation processing on the ghost floating image based on a variation matrix and the pairs of feature points, to obtain the processed ghost floating image.
Further, the system further comprises:
an optical flow rate calculation unit for calculating pixel-by-pixel optical flow rates (flox, floy) of the ghost floating image and the ghost reference image based on a dense pyramid optical flow method;
a region registration image obtaining unit, configured to map pixels in the ghost floating image onto the processed ghost floating image according to the pixel-by-pixel optical flow (flox, floy) to obtain the region registration image.
Further, the system further comprises:
a preliminary registration image obtaining unit, configured to perform alignment and fusion processing on the multiple region registration images to obtain a preliminary registration image;
an edge missing part obtaining unit, configured to extract edge missing part ROIs of a plurality of region registration images in the preliminary registration image, and obtain a plurality of ROIs;
the Gaussian blur processing unit is used for carrying out Gaussian blur processing on the plurality of ROIs to obtain a plurality of blurred ROIs;
the imgl acquisition unit is used for adjusting the gray distribution of the reference image according to the pixel distribution of the floating image and superposing the gray distribution of the reference image with the fuzzy ROIs according to a third preset proportion to obtain img1;
the img2 obtaining unit is used for iterating the fuzzy ROIs and the floating image according to a third preset proportion to obtain img2;
and the pixel superposition unit is used for superposing the img1 and the img2 to the preliminary registration image pixel by pixel to obtain the registration image.
EXAMPLE III
Based on the same inventive concept as the image registration data processing system in the second embodiment, the present embodiment also provides a fundus laser photocopier including the image registration data processing system as described in the second embodiment.
Any of the methods or steps described above may be stored as computer instructions or programs in various non-limiting types of computer memory that are recognized by various non-limiting types of computer processors to implement any of the methods or steps described above.
Based on the above embodiments of the present invention, those skilled in the art should make any improvements and modifications to the present invention without departing from the principle of the present invention, and therefore, the present invention should fall into the protection scope of the present invention.
Claims (10)
1. A method of image registration data processing, the method comprising:
acquiring a plurality of structural feature images in a plurality of monochromatic laser images, wherein the plurality of monochromatic laser images comprise reference images and floating images;
acquiring a plurality of ghost areas with overlarge eyeball movement according to the structural feature images, wherein each ghost area comprises a corresponding ghost floating image and a ghost reference image;
for each ghost region, acquiring a feature point correspondence relationship between the ghost floating image and a ghost reference image, wherein the feature point correspondence relationship comprises a plurality of pairs of feature points;
acquiring a plurality of pairs of feature points of which the accuracy is greater than a preset threshold value in the feature point corresponding relation, and performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image;
performing pixel registration processing on the ghost floating image to obtain a region registration image;
and carrying out alignment fusion processing on the registration images of the areas in the multiple ghost areas, and combining the reference image and the floating image to obtain a registration image.
2. The method of claim 1, wherein acquiring a plurality of structural feature images within a plurality of monochromatic laser images comprises:
performing Gaussian filtering smoothing noise processing on the plurality of monochromatic laser images to obtain a plurality of noise reduction processing images;
judging whether the gray values of the plurality of noise reduction processing images are smaller than a preset gray value threshold value or not; if yes, the noise reduction processing image is processed by adopting an adaptive histogram enhancement algorithm and gamma curve stretching histogram processing, and if not, the processing is not carried out, and a plurality of processing images are obtained;
and carrying out fuzzy enhancement processing on the plurality of processed images to obtain a plurality of structural feature images.
3. The method of claim 2, wherein performing blur enhancement processing on the plurality of processed images comprises:
performing fuzzy filter processing on the plurality of processed images to obtain a plurality of fuzzy processed images;
superposing the plurality of blurred processing images on the plurality of single-color laser images according to a first preset proportion to obtain a plurality of first enhanced processing images;
superposing the plurality of processed images and the plurality of first enhanced processed images according to a second preset proportion to obtain a plurality of second enhanced processed images;
obtaining a plurality of gray value thresholds according to the average gray value of the plurality of monochromatic laser images;
and extracting the plurality of second enhancement processing images according to the plurality of gray value thresholds to obtain the plurality of structural feature images.
4. The method of claim 1, wherein obtaining a plurality of ghost regions with excessive eye movement from the plurality of structural feature images comprises:
longitudinally blocking the structural feature images, dividing the structural feature images into a plurality of regions, and obtaining a plurality of { block1, block2, block3, … };
according to the plurality of { block1, block2, block3, … }, calculating mutual information value difference values of each reference image and each floating image in the plurality of areas, and obtaining a plurality of mutual information value difference value sequences { d1, d2, d3, … };
and taking the areas when the mutual information value difference value is continuously smaller or larger than a preset difference value threshold value as the ghost areas.
5. The method of claim 1, wherein obtaining feature point correspondences within the ghost floating images and ghost reference images for each ghost region comprises:
for each ghost area, extracting the characteristic points of a ghost reference image based on a characteristic point extraction algorithm to obtain a reference characteristic point set;
constructing a reference image pyramid and a floating image pyramid by respectively carrying out down-sampling on a ghost reference image and a ghost floating image, wherein the ghost reference image and the ghost floating image are respectively positioned at the bottom layers of the reference image pyramid and the floating image pyramid, and an upper layer image is obtained by carrying out 1/2 proportion down-sampling on the basis of a lower layer image;
corresponding the plurality of feature points in the reference feature point set to the highest layer in the reference image pyramid, and then searching the plurality of feature points in the corresponding layer of the floating image pyramid to obtain a highest-layer floating feature point set;
according to the highest layer floating feature point set, pre-translation and residual calculation are carried out on the next layer floating image, and a first feature point corresponding relation of the next layer floating image is obtained;
and performing iterative computation on each layer of floating images based on the first feature point corresponding relation to obtain the feature point corresponding relation.
6. The method according to claim 1, wherein a plurality of pairs of feature points with accuracy greater than a preset threshold in the feature point correspondence are obtained, and the performing of feature point matching on the ghost floating image comprises:
based on RANSAC algorithm, screening a plurality of pairs of feature points in the feature point corresponding relation to obtain a plurality of pairs of feature points with accuracy greater than the preset threshold;
and performing affine transformation processing on the ghost floating image based on the change matrix and the plurality of pairs of feature points to obtain the processed ghost floating image.
7. The method of claim 1, wherein performing a pixel registration process on the processed ghost floating images comprises:
calculating pixel-by-pixel optical flow (flox, floy) of the ghost floating image and the ghost reference image based on a dense pyramid optical flow method;
and mapping the pixels in the ghost floating image onto the processed ghost floating image according to the pixel-by-pixel optical flow (flox, floy) to obtain the area registration image.
8. The method according to claim 1, wherein performing an alignment fusion process on the plurality of region registration images within the plurality of ghost regions comprises:
carrying out alignment fusion processing on the multiple region registration images to obtain a primary registration image;
extracting the edge missing part ROI of a plurality of region registration images in the preliminary registration image to obtain a plurality of ROIs;
performing Gaussian blur processing on the plurality of ROIs to obtain a plurality of blurred ROIs;
adjusting the gray distribution of a reference image according to the pixel distribution of the floating image, and overlapping the gray distribution of the reference image with the plurality of fuzzy ROIs according to a third preset proportion to obtain img1;
iterating the plurality of fuzzy ROIs and the floating image according to a third preset proportion to obtain img2;
and performing pixel-by-pixel superposition on the img1 and the img2 to the preliminary registration image to obtain the registration image.
9. An image registration data processing system, characterized in that the system comprises:
a feature image obtaining module, configured to obtain a plurality of structural feature images in a plurality of monochromatic laser images, where the plurality of monochromatic laser images include a reference image and a floating image;
a ghost region obtaining module, configured to obtain a plurality of ghost regions with excessive eye movement according to the plurality of structural feature images, where each ghost region includes a corresponding ghost floating image and a ghost reference image;
the corresponding relation obtaining module is used for obtaining the corresponding relation of the characteristic points in the ghost floating image and the ghost reference image for each ghost area, wherein the corresponding relation of the characteristic points comprises a plurality of pairs of characteristic points;
the floating image obtaining module is used for obtaining a plurality of pairs of feature points of which the accuracy is greater than a preset threshold value in the feature point corresponding relation, and performing feature point matching processing on the ghost floating image to obtain a processed ghost floating image;
the area image obtaining module is used for carrying out pixel registration processing on the ghost floating image to obtain an area registration image;
a registered image obtaining module, configured to perform alignment fusion processing on the registered images of the multiple regions in the multiple ghost regions, and obtain a registered image by combining the reference image and the floating image.
10. A fundus laser camera comprising the image registration data processing system of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211472016.3A CN115690183A (en) | 2022-11-23 | 2022-11-23 | Image registration data processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211472016.3A CN115690183A (en) | 2022-11-23 | 2022-11-23 | Image registration data processing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115690183A true CN115690183A (en) | 2023-02-03 |
Family
ID=85055733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211472016.3A Pending CN115690183A (en) | 2022-11-23 | 2022-11-23 | Image registration data processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115690183A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315032A (en) * | 2023-11-28 | 2023-12-29 | 北京智愈医疗科技有限公司 | Tissue offset monitoring method |
-
2022
- 2022-11-23 CN CN202211472016.3A patent/CN115690183A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117315032A (en) * | 2023-11-28 | 2023-12-29 | 北京智愈医疗科技有限公司 | Tissue offset monitoring method |
| CN117315032B (en) * | 2023-11-28 | 2024-03-08 | 北京智愈医疗科技有限公司 | Tissue offset monitoring method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abràmoff et al. | Retinal imaging and image analysis | |
| CN102737395B (en) | Image processing method and device in a kind of medical X-ray system | |
| WO2020183799A1 (en) | Medical image processing device, medical image processing method, and program | |
| US20230120295A1 (en) | Methods and systems for ocular imaging, diagnosis and prognosis | |
| CN111710012B (en) | OCTA imaging method and device based on two-dimensional composite registration | |
| WO2011059655A1 (en) | Enhanced imaging for optical coherence tomography | |
| CN106846346B (en) | Method for rapidly extracting pelvis outline of sequence CT image based on key frame mark | |
| CN106558031B (en) | A kind of image enchancing method of the colored eyeground figure based on imaging model | |
| CN113557714A (en) | Medical image processing apparatus, medical image processing method, and program | |
| Manakov et al. | Noise as domain shift: Denoising medical images by unpaired image translation | |
| CN112330638B (en) | Retina OCT (optical coherence tomography) image horizontal registration and image enhancement method | |
| CN112541923B (en) | Cup optic disk segmentation method based on fundus image data set migration learning | |
| Adal et al. | A hierarchical coarse-to-fine approach for fundus image registration | |
| CN106780491B (en) | Initial contour generation method adopted in segmentation of CT pelvic image by GVF method | |
| JP4274400B2 (en) | Image registration method and apparatus | |
| JP2023505268A (en) | Neural network processing of OCT data to generate predictions of geographic atrophy growth rates | |
| CN115690183A (en) | Image registration data processing method and system | |
| CN115775210A (en) | Wide-area fundus camera image fusion method, system and storage medium | |
| CN109919098B (en) | Target object identification method and device | |
| CN106780492B (en) | Method for extracting key frame of CT pelvic image | |
| WO2021009804A1 (en) | Method for learning threshold value | |
| CN114092405A (en) | Retina layer automatic segmentation method for macular edema OCT image | |
| CN110033496B (en) | Motion artifact correction method for time sequence three-dimensional retina SD-OCT image | |
| Niemeijer et al. | Automated localization of the optic disc and the fovea | |
| Bontala et al. | Image mosaicing of low quality neonatal retinal images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |