CN117541633A - Confocal endoscope image alignment parameter calculation method and related equipment - Google Patents
Confocal endoscope image alignment parameter calculation method and related equipment Download PDFInfo
- Publication number
- CN117541633A CN117541633A CN202311600781.3A CN202311600781A CN117541633A CN 117541633 A CN117541633 A CN 117541633A CN 202311600781 A CN202311600781 A CN 202311600781A CN 117541633 A CN117541633 A CN 117541633A
- Authority
- CN
- China
- Prior art keywords
- data set
- data
- line number
- alignment
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 19
- 238000003860 storage Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 22
- 238000004458 analytical method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000001914 filtration Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000003745 diagnosis Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000010191 image analysis Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000001839 endoscopy Methods 0.000 description 1
- 230000002496 gastric effect Effects 0.000 description 1
- 231100001014 gastrointestinal tract lesion Toxicity 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Endoscopes (AREA)
Abstract
The application discloses a confocal endoscope image alignment parameter calculation method and related equipment, and relates to the field of image processing, wherein the method comprises the following steps: acquiring a data set to be processed; carrying out data denoising operation on the data set to be processed so as to obtain the data set to be calculated; selecting a first data set and two second data sets from the data set to be calculated, wherein each data set in the second data set is adjacent to the data in the first data set in the calculation data set, and the second data sets with different sequences in the second data set and the first data set are positioned on two sides of the first data in the first data set in the calculation data set; performing offset operation on the second data set by taking the first data set as a reference, and calculating an alignment cost value; and determining a target alignment parameter based on the alignment cost value.
Description
Technical Field
The present disclosure relates to the field of image processing, and more particularly, to a method and apparatus for computing image alignment parameters for a confocal endoscope.
Background
The confocal endoscope is medical equipment which can extend into a human body by means of channels such as a gastroscope, a colonoscope and the like to acquire local histological images so as to realize accurate diagnosis of micro focus, gastrointestinal lesions and early gastrointestinal canceration. There are two important components in the scan control module of a confocal endoscope: a resonant mirror and a galvanometer vibrating mirror. The resonant mirror acts to rapidly scan light in a horizontal direction (also referred to as an X-galvanometer mirror), and the galvanometer mirror acts to scan light in a vertical direction (also referred to as a Y-galvanometer mirror), which cooperate to obtain a two-dimensional planar image.
The resonant mirror operates on the principle of reciprocating rotation through a certain angle along a rotation axis, steering when reaching the edge of the scanning range, and rotating in the opposite direction. And the angular velocity during scanning varies sinusoidally with the spatial position. Because of the high scanning speed, the scanning start points of two adjacent lines are difficult to be consistent, and the two adjacent lines are shifted. In the related art, an accurate alignment parameter calculation method is lacking.
Disclosure of Invention
In the summary, a series of concepts in a simplified form are introduced, which will be further described in detail in the detailed description. The summary of the present application is not intended to define the key features and essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.
In a first aspect, the present application proposes a method for calculating an alignment parameter of a confocal endoscope image, the method comprising:
acquiring a data set to be processed;
carrying out data denoising operation on the data set to be processed so as to obtain the data set to be calculated;
selecting a first data set and two second data sets from the data set to be calculated, wherein each data set in the second data set is adjacent to the data in the first data set in the calculation data set, and the second data sets with different sequences in the second data set and the first data set are positioned on two sides of the first data in the first data set in the calculation data set;
performing offset operation on the second data set by taking the first data set as a reference, and calculating an alignment cost value;
and determining a target alignment parameter based on the alignment cost value.
Optionally, in the case that the first data set is an odd data set, the second data set is an even data set.
Optionally, in the case that the first data set is an even data set, the second data set is an odd data set.
Optionally, the selecting a first data set and two second data sets from the calculated data sets includes:
determining a start line number and an end line number of a target area in the image in the calculation dataset;
performing segmentation processing based on the starting line number, the ending line number and a preset segment number to obtain the nearest odd line or even line at the segmentation position so as to form the first data set;
second data adjacent to the first data in the first data set is acquired based on the first data set to form two second data sets.
Optionally, the determining the start line number and the end line number of the target area in the image in the calculation dataset includes:
acquiring a pixel value histogram of the image data in the calculation dataset;
acquiring threshold information by using an OTSU algorithm based on the pixel value histogram;
and determining the starting line number and the ending line number based on the threshold information.
Optionally, the method further comprises:
the alignment cost value cost is calculated by:
wherein P represents the number of lines used in the alignment operation, eid and sil represent the end line number and the start line number, od, respectively p (id) represents the id-th data point, ed in even row p at offset D 1 p (id) and ed 2 p (id) respectively represent the corresponding points of the id-th data point in the odd-numbered row p in two different even-numbered row data sets,for a double summation symbol, it is indicated that all selected odd rows and each data point in these rows are to be iterated, +.>Andis the absolute value difference, representing the difference between the parity rows of a given data point at the D offset,is a normalization factor.
Optionally, the method further comprises:
switching the confocal endoscope to a preprocessing mode to acquire a preprocessing image set;
and performing overturn operation on the preprocessed images in the preprocessed image set to obtain a data set to be processed, so that all images in the data set to be processed have the same acquisition direction.
In a second aspect, the present application also proposes a confocal endoscopic image alignment parameter calculation apparatus including:
the first acquisition unit is used for acquiring a data set to be processed;
the second acquisition unit is used for carrying out data denoising operation on the data set to be processed so as to acquire the data set to be calculated;
a selecting unit, configured to select a first data set and two second data sets in the computing data set, where each data set in the second data set is adjacent to the data set in the first data set, and different second data sets in the same order as the second data sets in the first data set are located on both sides of the first data set in the computing data set;
an offset unit, configured to perform an offset operation on the second data set with the first data set as a reference, and calculate an alignment cost value;
and the determining unit is used for determining the target alignment parameter based on the alignment cost value.
In a third aspect, an electronic device, comprising: a memory, a processor and a computer program stored in and executable on the processor for performing the steps of the method for computing the parameters of the image alignment of a confocal endoscope according to any one of the first aspects described above when the computer program stored in the memory is executed.
In a fourth aspect, the present application also proposes a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the confocal endoscopic image alignment parameter calculation method of any one of the above aspects.
In summary, the confocal endoscope image alignment parameter calculation method of the embodiment of the present application includes: acquiring a data set to be processed; carrying out data denoising operation on the data set to be processed so as to obtain the data set to be calculated; selecting a first data set and two second data sets from the data set to be calculated, wherein each data set in the second data set is adjacent to the data in the first data set in the calculation data set, and the second data sets with different sequences in the second data set and the first data set are positioned on two sides of the first data in the first data set in the calculation data set; performing offset operation on the second data set by taking the first data set as a reference, and calculating an alignment cost value; and determining a target alignment parameter based on the alignment cost value. According to the confocal endoscope image alignment parameter calculation method, the original image is turned over to unify the acquisition direction, and a denoising technology such as median filtering or Gaussian filtering is adopted, so that the quality of image data is improved, noise and errors are reduced, and subsequent data processing and analysis are more accurate. And proper data sets are selected and spatial adjacency analysis is performed, so that the refinement and high efficiency of data processing are ensured. The data sets can be precisely aligned by calculating the alignment cost values and adjusting the alignment parameters based on these values, thereby ensuring consistency and comparability of the data. The precisely aligned and optimized data set provides a solid basis for subsequent image analysis and interpretation, thereby enhancing the reliability and effectiveness of the analysis results. The method can at least partially eliminate the problems caused by the scanning characteristics of the resonant mirror, provide the user with the correct image with the same actual shape, and further provide the clinic with accurate diagnosis information.
Additional advantages, objects, and features of the present application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the present application.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the specification. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 is a schematic flow chart of a method for calculating an image alignment parameter of a confocal endoscope according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a spatial position of a resonant mirror during scanning according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a resonant mirror reciprocating scanning sampling principle provided in an embodiment of the present application;
FIG. 4 is a schematic diagram showing the relationship between angular velocity and spatial position of a resonant mirror during scanning according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of an original image scanned by a resonant mirror according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an original diagram inversion principle provided in an embodiment of the present application;
fig. 7 is a schematic diagram of an original image after being flipped according to an embodiment of the present application;
fig. 8 is a schematic diagram of determining a start line and an end line of an effective area of an image according to an embodiment of the present application;
fig. 9 is a schematic diagram of a pixel histogram according to an embodiment of the present application;
fig. 10 is a schematic diagram of an alignment scenario at different offset values according to an embodiment of the present application;
FIG. 11 is a schematic structural diagram of a confocal endoscopic image alignment parameter calculation apparatus provided in an embodiment of the present application;
fig. 12 is a schematic structural diagram of a confocal endoscope image alignment parameter calculation electronic device according to an embodiment of the present application.
Detailed Description
The terms "first," "second," "third," "fourth" and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application.
FIG. 2 is a schematic diagram of the spatial position of a resonant mirror during scanning in the related art; FIG. 3 is a schematic diagram of a resonant mirror reciprocation scanning sampling principle in the related art; fig. 4 is a schematic diagram showing the relationship between angular velocity and spatial position during scanning of a resonant mirror in the related art. As shown in fig. 2 to 4, the resonant mirror operates on a principle of reciprocating rotation by a certain angle along a rotation axis, turning when reaching the edge of the scanning range, and rotating in the opposite direction. And the angular velocity during scanning varies sinusoidally with the spatial position.
Confocal endoscopy is typically performed using equally spaced samples. The original image obtained by sampling has the following problems due to the characteristics of reciprocation and sine in the scanning process of the resonant mirror: (1) the scanning directions of two adjacent rows are reversed; (2) Because the scanning speed is high, the scanning starting points of two adjacent lines are difficult to be consistent, so that the two adjacent lines are shifted; (3) The angular velocity of the sinusoidal features during scanning is slow at the edges and fast in the middle, resulting in distortion of the whole with stretching at both ends and compression in the middle.
Inversion, shifting and distortion result in the obtained image not conforming to the actual shape of the object. Such images, if applied clinically, may provide the user with erroneous information, which in turn may lead to the user making erroneous diagnostic results, which is not acceptable. Therefore, the confocal endoscope is used for aligning displacement and correcting distortion so as to eliminate the problems caused by the scanning characteristics of the resonant mirror, provide the user with an image which is correct and has the same shape as the actual shape, and further provide accurate diagnosis information for clinic. In order to solve at least some of the above problems, the present application proposes a confocal endoscope image alignment parameter calculation method for performing an alignment operation on an image.
Referring to fig. 1, a flowchart of a method for calculating an image alignment parameter of a confocal endoscope according to an embodiment of the present application may specifically include:
s110, acquiring a data set to be processed;
illustratively, the data set to be processed is obtained by performing a flipping operation on an original image acquired by the confocal endoscope in the preprocessing mode, and all images in the data set to be processed have a uniform acquisition direction.
S120, performing data denoising operation on the data set to be processed to obtain the data set to be calculated;
illustratively, the data set to be processed is denoised to improve image quality and reduce errors in the data. The processing can be performed by adopting a method of median filtering, gaussian filtering or waveform threshold value and the like. The denoised dataset is called the dataset to be calculated, which will be used for more complex image processing and analysis.
S130, selecting a first data set and two second data sets from the data set to be calculated, wherein each data set in the second data sets is adjacent to the data in the first data set in the calculation data set, and the second data sets with different orders in the second data sets and the first data sets are positioned on two sides of the first data set in the calculation data set;
illustratively, one first data set and two second data sets are selected from the data sets to be calculated. Each data point in the second data set is spatially adjacent to a data point in the first data set, and the same order of data points in each second data set flank a particular data point in the first data set.
S140, performing offset operation on the second data set by taking the first data set as a reference, and calculating an alignment cost value;
illustratively, the second data set is offset with respect to the first data set to achieve optimal alignment. By calculating the alignment cost value, it is possible to evaluate the similarity or difference between the data sets by cross-correlation, euclidean distance, etc. This alignment cost value reflects the accuracy and effect of the alignment.
S150, determining a target alignment parameter based on the alignment cost value.
Illustratively, an optimal alignment parameter, such as an offset, rotation angle, or scaling factor, is determined based on the calculated alignment cost value. And (3) finding an alignment parameter capable of minimizing the difference between the first data set and the second data set to ensure that the data sets have optimal consistency and corresponding relation.
In summary, according to the confocal endoscope image alignment parameter calculation method provided by the embodiment of the application, the original image is turned over to unify the acquisition direction, and a denoising technology such as median filtering or Gaussian filtering is adopted, so that the quality of image data is improved, noise and errors are reduced, and subsequent data processing and analysis are more accurate. And proper data sets are selected and spatial adjacency analysis is performed, so that the refinement and high efficiency of data processing are ensured. The data sets can be precisely aligned by calculating the alignment cost values and adjusting the alignment parameters based on these values, thereby ensuring consistency and comparability of the data. The precisely aligned and optimized data set provides a solid basis for subsequent image analysis and interpretation, thereby enhancing the reliability and effectiveness of the analysis results. The method can at least partially eliminate the problems caused by the scanning characteristics of the resonant mirror, provide the user with the correct image with the same actual shape, and further provide the clinic with accurate diagnosis information.
In some examples, where the first data set is an odd data set, the second data set is an even data set.
Illustratively, the first data set and the second data set may be distinguished by parity of sequence numbers at the time of sampling, and when the first data set is an odd data set, the second data set is an even data set, if the first data set contains data of an odd row (e.g., row 1, row 3, etc.), then the second data set will contain data of an even row (e.g., row 2, row 4, etc.).
In some examples, where the first data set is an even data set, the second data set is an odd data set.
Illustratively, when the first data set is an even data set, the second data set is an odd data set. Conversely, if the first data set contains even rows of data, then the second data set contains odd rows of data
In some examples, the selecting one first data set and two second data sets from the calculated data sets includes:
determining a start line number and an end line number of a target area in the image in the calculation dataset;
performing segmentation processing based on the starting line number, the ending line number and a preset segment number to obtain the nearest odd line or even line at the segmentation position so as to form the first data set;
second data adjacent to the first data in the first data set is acquired based on the first data set to form two second data sets.
The specific method for selecting one first data set and two second data sets in the calculation data set is specifically described by taking the first data set as an odd-line number data set and the second data set as an even-line number data set as an example. The data set to be processed is accumulated K 1 (K 1 1) frame original image, and the data to be processed is D 1 ＝{D k },k＝0,1,L,K 1 -1, data denoising the data to be processed, each pixel position to be K 1 And obtaining 1 frame of image by taking the average value of the values. Recording the denoised image as And further a calculated dataset consisting of denoised images is obtained. The mean value can also be obtained by adopting the method of taking the mean value: />The averaging/median function of the plurality of data is to cancel the effect of noise. A first dataset and two second datasets are selected within a target region of a computed dataset image.
Selecting a first data set SOL consisting of P (1 < = P < H/2) odd line numbers:
SOL＝{sol p }
p＝0,1,L,P-1
in the above formula, the round (… …/2) 2-1 operation is guaranteed to be odd.
The specific method for selecting P odd line numbers is to divide the image between the starting line (sl) and the ending line (el) into P+1 segments in the vertical direction, and P segmentation intervals are all used for selecting the odd line number nearest to each interval. As shown in fig. 8, the scanning range is equally divided into 4 segments, and the selected 3 odd-numbered rows are the intervals of segment 1 and segment 2, segment 2 and segment 3, and segment 3 and segment 4, respectively.
Correspondingly, two even-numbered line number sets SEL are selected 1 And SEL (presentation of) 2 Each set contains P elements. SEL (SEL) 1 The P even line numbers in the set are the corresponding P odd line numbers minus 1, SEL in SOL 2 The P even line numbers in the set are the corresponding P odd line numbers in the SOL plus 1. I.e.
p＝0,1,L,P-1
Record P odd lines in SOL setThe data in (a) is
OD＝{od p },p＝0,1,L,P-1
Recording SEL 1 P even lines in the setThe data in (a) is
Recording SEL 2 P even lines in the setThe data in (a) is
In conclusion, the scanning range is equally divided, the nearest odd lines in each interval are selected, the uniformity of data sampling in space is ensured, the acquisition of representative strong data is facilitated, and sampling deviation is avoided. By selecting the adjacent odd and even rows, data comparison can be conveniently performed. The method simplifies the data processing flow by systematically selecting the data rows. The selection rule is clear and easy to implement, which is particularly important when processing a large amount of data. Since adjacent parity rows are selected, they are closely related in space. This close spatial correlation facilitates subsequent data alignment work, especially in applications requiring accurate alignment of image data.
In some examples, determining the start line number and the end line number of the target region in the image in the computing dataset includes:
acquiring a pixel value histogram of the image data in the calculation dataset;
acquiring threshold information by using an OTSU algorithm based on the pixel value histogram;
and determining the starting line number and the ending line number based on the threshold information.
Illustratively, the image data in the calculation data set is analyzed to obtain a frequency distribution of each pixel value, i.e., a pixel value histogram. The OTSU algorithm is an automatic thresholding method that selects the best threshold by maximizing the inter-class variance, which can divide the image into two parts, the active area and the inactive area. With the threshold value obtained by the OTSU algorithm, it is possible to determine which line numbers of image data are important. Determining the start line number and the end line number is important for focusing on a specific part of the image. In medical imaging, only a certain region of the image may be of interest, which can be determined by analyzing the pixel value histogram and applying the OTSU algorithm.
Specifically, the method for determining the start line sl and the end line el comprises the following steps: for imagesStatistics of pixel value histograms, results as described in fig. 9, the threshold T is obtained using the OTSU algorithm for the above histograms, and the example histogram of fig. 9 uses the OTSU algorithm to obtain the threshold t=88.
The sl is found by the following pseudocode:
the pseudo code traverses from the first line of the image to the last line. H is the total number of lines of the image. Within each row, the traversal starts from the second pixel until the penultimate pixel. N is the total number of columns of the image. For each pixel in the currently traversed row it is checked whether its value and its left and right neighboring pixels are both greater than a certain preset threshold T. This check is to identify a continuous change in luminance in the horizontal direction, which may represent an edge or a feature line. If a sequence of pixels satisfying the condition is found in a certain line, this line is recorded as the starting line S. After the initial line is recorded, the whole traversal process is finished, and the initial line number is returned.
The el is found by the following pseudocode:
traversing up from the last row (H-1) of the image to the first row, in each row, traversing from the second column to the penultimate column. For each pixel point D (i, j), it and the pixel values D (i, j-1) and D (i, j+1) on its left and right sides are checked whether both are greater than a certain threshold T. If the above condition is satisfied, the line number i of the line is recorded. Once a line number satisfying the condition is found, this line number is returned and the program is ended.
In summary, the method provided by the embodiment of the application determines the target area in the image by an automatic method, so that subjectivity and time cost of manual selection are reduced. The OTSU algorithm automatically determines the threshold by calculation, which is more efficient than manually adjusting the threshold. The OTSU algorithm provides an objective method to determine the optimal threshold based on statistical principles, thus maintaining consistency across different image sets.
In some examples, the above method further comprises:
the alignment cost value cost is calculated by:
wherein P represents the number of lines used in the alignment operation, eid and sil represent the end line number and the start line number, od, respectively p (id) represents the id-th data point, ed in even row p at offset D 1 p (id) and ed 2 p (id) respectively represent the corresponding points of the id-th data point in the odd-numbered row p in two different even-numbered row data sets,for a double summation symbol, it is indicated that all selected odd rows and each data point in these rows are to be iterated, +.>Andis the absolute value difference, representing the difference between the parity rows of a given data point at the D offset,is a normalization factor.
Illustratively, the offset is an integer, denoted by a. cost represents the alignment cost at offset a. When a is sequentially taken from [1-N 1 ,N 1 -1]When the integer value is within the range, the cost is calculated. The even-numbered data is offset by a with reference to the odd-numbered data. The different offset values are shown in schematic diagram 10, and are respectively the offset values of-4,0,3. od denotes odd line data, and ed denotes even line data. The numbers in the boxes represent the data element indices in each row of data.
And finding out the overlapping part of the offset odd-line data and the even-line data. The overlap is indicated by an odd line subscript. The beginning subscript and the ending subscript of the overlap are noted as sed and eid, respectively:
sid＝max(a,0)
eid＝min(N 1 -1,N 1 -1+a)
the alignment cost value cost formula provided by the formula provides a quantization method to evaluate the data alignment effect, and the best alignment parameter can be found by minimizing the cost function, so that the accuracy and the reliability of the image processing task are improved.
In some examples, the above method further comprises:
switching the confocal endoscope to a preprocessing mode to acquire a preprocessing image set;
and performing overturn operation on the preprocessed images in the preprocessed image set to obtain a data set to be processed, so that all images in the data set to be processed have the same acquisition direction.
Illustratively, the number of lines of the original image generated upon confocal endoscopic scanning is denoted as H. The "line number" indicates what number of lines this line of data is located in the image. Each row of data contains N 1 Data elements, subscripts from 0 to N 1 -1. Confocal endoscope-to-fiber bundle probeImaging, the confocal endoscope is operated in a pretreatment mode. At this point, the imaging example is shown at 5, where raw image data is continuously acquired to form a preprocessed image set. Since the resonant mirror is reciprocally scanned, the data directions of the adjacent two rows of samples are opposite. All even rows are flipped so that the data direction of the even rows becomes identical to the odd rows. And performing a flipping operation on the preprocessed images in the preprocessed image set.
The data before and after the overturn of a certain line are recorded as { A }, respectively n Sum { B } n N=0, 1,2, l, n-1, then B n ＝A N-1-n The physical meaning of n=0, 1,2, l, n-1 inversion is shown in fig. 6, the inverted image is shown in fig. 7, and all images in the data set to be processed have the same acquisition direction.
Referring to fig. 11, an embodiment of the confocal endoscopic image alignment parameter calculation apparatus in the embodiment of the present application may include:
a first acquisition unit 21 for acquiring a data set to be processed;
a second obtaining unit 22, configured to perform a data denoising operation on the data set to be processed, so as to obtain a data set to be calculated;
a selecting unit 23, configured to select a first data set and two second data sets from the calculation data sets, where each data set in the second data sets is adjacent to the data set in the first data set, and different second data sets in the same order as the second data sets in the first data sets are located on both sides of the first data set in the first data sets;
an offset unit 24 for performing an offset operation on the second data set with reference to the first data set, and calculating an alignment cost value;
a determining unit 25 for determining a target alignment parameter based on the alignment cost value.
As shown in fig. 12, the embodiment of the present application further provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor, where the processor 320 implements the steps of any of the above methods for calculating the image alignment parameters of the confocal endoscope when executing the computer program 311.
Since the electronic device described in this embodiment is a device for implementing the confocal endoscope image alignment parameter calculating apparatus in this embodiment, based on the method described in this embodiment, those skilled in the art can understand the specific implementation of the electronic device in this embodiment and various modifications thereof, so how the electronic device implements the method in this embodiment will not be described in detail herein, and those skilled in the art will be in the scope of protection of this application as long as those skilled in the art implement the device for implementing the method in this embodiment.
In a specific implementation, the computer program 311 may implement any of the embodiments corresponding to fig. 1 when executed by a processor.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Embodiments also provide a computer program product comprising computer software instructions that, when run on a processing device, cause the processing device to perform the confocal endoscopic image alignment parameter calculation procedure in the corresponding embodiment
The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be stored by a computer or data storage devices such as servers, data centers, etc. that contain an integration of one or more available media. Usable media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid State Disks (SSDs)), among others.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (10)
1. A confocal endoscopic image alignment parameter calculation method, comprising:
acquiring a data set to be processed;
performing data denoising operation on the data set to be processed to obtain the data set to be calculated;
selecting a first data set and two second data sets from the data set to be calculated, wherein each data in the second data set is adjacent to the data in the first data set in the calculation data set, and the second data in the same order of the second data sets and the first data sets are positioned on two sides of the first data in the first data set in the calculation data set;
performing offset operation on the second data set by taking the first data set as a reference, and calculating an alignment cost value;
and determining a target alignment parameter based on the alignment cost value.
2. The confocal endoscopic image alignment parameter calculation method of claim 1, wherein in the case where the first data set is an odd data set, the second data set is an even data set.
3. The confocal endoscopic image alignment parameter calculation method of claim 1, wherein in the case where the first data set is an even data set, the second data set is an odd data set.
4. The method of claim 1, wherein selecting one first data set and two second data sets from the calculated data sets comprises:
determining a start line number and an end line number of a target area in the image in the calculation dataset;
performing segmentation processing based on the starting line number, the ending line number and a preset segment number to acquire the nearest odd line or even line at the segmentation position so as to form the first data set;
second data adjacent to first data in the first data set is acquired based on the first data set to form two second data sets.
5. The method of claim 4, wherein determining a start line number and an end line number of a target region in an image in the calculation dataset comprises:
acquiring a pixel value histogram of image data in the calculation dataset;
acquiring threshold information by using an OTSU algorithm based on the pixel value histogram;
the starting line number and the ending line number are determined based on the threshold information.
6. The confocal endoscopic image alignment parameter calculation method of claim 1, further comprising:
the alignment cost value cost is calculated by:
wherein P represents the number of lines used in the alignment operation, eid and sil represent the end line number and the start line number, od, respectively p (id) represents the id-th data point, ed in even row p at offset D 1 p (id) and ed 2 p (id) respectively represent the corresponding points of the id-th data point in the odd-numbered row p in two different even-numbered row data sets,for a double summation symbol, it is indicated that all selected odd rows and each data point in these rows are to be iterated, +.>Andis the absolute value difference, representing the difference between the parity rows of a given data point at the D offset,is a normalization factor.
7. The confocal endoscopic image alignment parameter calculation method of claim 1, further comprising:
switching the confocal endoscope to a preprocessing mode to acquire a preprocessed image set;
and performing overturn operation on the preprocessed images in the preprocessed image set to obtain a data set to be processed, so that all images in the data set to be processed have the same acquisition direction.
8. A confocal endoscopic image alignment parameter calculation apparatus comprising:
the first acquisition unit is used for acquiring a data set to be processed;
the second acquisition unit is used for carrying out data denoising operation on the data set to be processed so as to acquire the data set to be calculated;
a selecting unit, configured to select a first data set and two second data sets in the computing data set, where each data in the second data set is adjacent to the data in the first data set in the computing data set, and different second data sets in the same order are located on two sides of the first data in the computing data set in the first data set with the first data set;
the offset unit is used for performing offset operation on the second data set by taking the first data set as a reference and calculating an alignment cost value;
and the determining unit is used for determining a target alignment parameter based on the alignment cost value.
9. An electronic device, comprising: memory and processor, characterized in that the processor is adapted to carry out the steps of the method of confocal endoscopic image alignment parameter calculation according to any one of claims 1-7 when executing a computer program stored in the memory.
10. A computer-readable storage medium having stored thereon a computer program, characterized by: the computer program, when executed by a processor, implements a method of confocal endoscopic image alignment parameter calculation according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311600781.3A CN117541633A (en) | 2023-11-28 | 2023-11-28 | Confocal endoscope image alignment parameter calculation method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311600781.3A CN117541633A (en) | 2023-11-28 | 2023-11-28 | Confocal endoscope image alignment parameter calculation method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117541633A true CN117541633A (en) | 2024-02-09 |
Family
ID=89795567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311600781.3A Pending CN117541633A (en) | 2023-11-28 | 2023-11-28 | Confocal endoscope image alignment parameter calculation method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117541633A (en) |
-
2023
- 2023-11-28 CN CN202311600781.3A patent/CN117541633A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6150583B2 (en) | Image processing apparatus, endoscope apparatus, program, and operation method of image processing apparatus | |
| US9576362B2 (en) | Image processing device, information storage device, and processing method to acquire a summary image sequence | |
| US9740939B2 (en) | Image processing device, information storage device, and image processing method | |
| CN113383225A (en) | System and method for propagating two-dimensional fluorescence waves onto a surface using deep learning | |
| US10366504B2 (en) | Image processing apparatus and image processing method for performing three-dimensional reconstruction of plurality of images | |
| KR101468418B1 (en) | Method and apparatus for processing ultrasound images | |
| US9569846B2 (en) | Ophthalmic reference image selection | |
| EP2153407A1 (en) | Motion compensated image averaging | |
| RU2747946C1 (en) | Method and device for processing an image obtained using a fiber-optic bundle | |
| US8582839B2 (en) | Ultrasound system and method of forming elastic images capable of preventing distortion | |
| CN117541633A (en) | Confocal endoscope image alignment parameter calculation method and related equipment | |
| CN107529962B (en) | Image processing apparatus, image processing method, and recording medium | |
| CN111967462A (en) | Method and device for acquiring region of interest | |
| CN117593228A (en) | Confocal endoscope image optimization method and related equipment | |
| US20180025211A1 (en) | Cell tracking correction method, cell tracking correction device, and storage medium which stores non-transitory computer-readable cell tracking correction program | |
| EP1877954B1 (en) | Four-dimensional imaging of periodically moving objects via post-acquisition synchronization of nongated slice-sequences | |
| EP3716199A1 (en) | Signal processing apparatus and method using local length scales for deblurring | |
| WO2024117192A1 (en) | Image processing for improving resolution of image of specimen | |
| JP2020168371A (en) | Medical imaging apparatus | |
| CN113298711B (en) | Optical fiber bundle multi-frame image super-resolution reconstruction method and device | |
| CN117541569A (en) | Confocal endoscope invalid image screening method and related equipment | |
| CN117635494A (en) | Multi-reference-value confocal endoscope image correction method and related equipment | |
| CN117635495A (en) | Multi-index-value confocal endoscope image correction method and related equipment | |
| EP4000502B1 (en) | Volumetric oct image data processing | |
| JP5463974B2 (en) | Information processing apparatus, program, and information processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |