WO2020037570A1 - 图像配准方法、装置和计算机程序产品 - Google Patents

图像配准方法、装置和计算机程序产品 Download PDF

Info

Publication number
WO2020037570A1
WO2020037570A1 PCT/CN2018/101815 CN2018101815W WO2020037570A1 WO 2020037570 A1 WO2020037570 A1 WO 2020037570A1 CN 2018101815 W CN2018101815 W CN 2018101815W WO 2020037570 A1 WO2020037570 A1 WO 2020037570A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
bright spot
pixel
candidate
matrix
Prior art date
Application number
PCT/CN2018/101815
Other languages
English (en)
French (fr)
Inventor
李林森
徐伟彬
金欢
姜泽飞
周志良
颜钦
Original Assignee
深圳市真迈生物科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市真迈生物科技有限公司 filed Critical 深圳市真迈生物科技有限公司
Priority to US17/270,423 priority Critical patent/US12008775B2/en
Priority to PCT/CN2018/101815 priority patent/WO2020037570A1/zh
Priority to EP18930564.2A priority patent/EP3843032A4/en
Publication of WO2020037570A1 publication Critical patent/WO2020037570A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/37—Determination of transform parameters for the alignment of images, i.e. image registration using transform domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20048—Transform domain processing
    • G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30072—Microarray; Biochip, DNA array; Well plate

Definitions

  • the present invention relates to the field of image processing, and in particular, to an image registration method, an image registration device, and a computer program product having an image registration function.
  • the hardware movement has a certain accuracy, that is, there is a certain error between the specified movement amount and the actual movement amount, and / or changes in the shape of the nucleic acid molecule due to changes in the environment / system in which the nucleic acid molecule is located
  • the position information of the fixed nucleic acid molecule in the image of the field of view obtained at multiple times may be different, making it difficult to directly identify and determine the sequence of the nucleic acid molecule by using the obtained image information directly.
  • the embodiments of the present invention aim to solve at least one of the technical problems in the related art or provide at least one optional practical solution.
  • an image registration method includes: performing a first registration on an image to be registered based on a reference image, the so-called reference image and the image to be registered correspond to the same object, the reference image and The images to be registered each include multiple bright spots, including determining a first offset of a predetermined region on the image to be registered and a corresponding predetermined region on a so-called reference image, and moving the image to be registered based on the first offset.
  • All the bright spots on the image to obtain the image to be registered after the first registration including merging the to-be-registered images after the first registration Quasi-image and reference image, obtain the merged image, calculate the offset of all overlapping bright spots in a predetermined area on the merged image to determine the second offset, and two or more bright spots with a distance less than the predetermined pixel are the same
  • all bright spots on the image to be registered after the first registration are moved based on the second offset to achieve registration of the image to be registered.
  • an image registration device is provided.
  • the device is used to implement the image registration method in the foregoing embodiment of the present invention.
  • the device includes: a first registration module, configured to treat an image based on a reference image.
  • the first registration of the registration image includes determining a first offset of a predetermined region on the image to be registered and a corresponding predetermined region on the reference image, and moving all bright spots on the image to be registered based on the first offset.
  • the so-called reference image and the image to be registered correspond to the same object, and both the reference image and the image to be registered contain multiple bright spots; a second registration module is used to The reference image performs second registration on the first to-be-registered image from the first registration module, including merging the to-be-registered image and the reference image after the first registration, obtaining a merged image, and calculating the merged image.
  • a computer-readable storage medium for storing a program for execution by a computer, and executing the program includes performing an image registration method in any one of the foregoing embodiments.
  • the computer-readable storage medium includes, but is not limited to, read-only memory, random access memory, magnetic disks, or optical disks.
  • a terminal and a computer program product including instructions that, when the computer executes the so-called program, causes the computer to execute all of the image registration methods in the embodiment of the present invention described above. Or some steps.
  • Utilizing the image registration method and device and / or the terminal / computer program product for realizing image registration in the foregoing embodiments of the present invention can realize high-precision image correction, and is particularly suitable for scenes requiring high-resolution image correction.
  • FIG. 1 is a schematic flowchart of an image registration method in a specific embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an image correction process and a correction result in a specific embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of an image registration method in a specific embodiment of the present invention.
  • FIG. 4 is a schematic diagram of corresponding matrices of candidate bright spots and pixels together in a specific embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of an image registration method in a specific embodiment of the present invention.
  • FIG. 6 is a schematic diagram of pixel values in a range of m1 * m2 centered on a central pixel of a pixel matrix in a specific embodiment of the present invention.
  • FIG. 7 is a schematic diagram of comparison of bright spot detection results before and after determination according to a second bright spot detection threshold in a specific embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an image registration device in a specific embodiment of the present invention.
  • first and second are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number or order of the technical features indicated.
  • meaning of “plurality” is two or more, unless specifically defined otherwise.
  • An embodiment of the present invention provides an image registration method, that is, an image correction method, as shown in FIG. 1, including: S10 performing a first registration based on a reference image to be registered, and the reference image corresponds to the image to be registered For the same object, the reference image and the image to be registered each include multiple bright spots, including determining a first offset of a predetermined region on the image to be registered and a corresponding predetermined region on the reference image, and moving the target based on the first offset.
  • All bright spots on the registration image to obtain the image to be registered after the first registration S20 performs second registration on the image to be registered after the first registration based on the reference image, including merging the first registration An image to be registered and a reference image are obtained, a merged image is obtained, and an offset of all coincident bright spots in a predetermined area on the merged image is calculated to determine a second offset.
  • Two or more bright spots with a distance less than a predetermined pixel are An overlapping bright spot, based on the second offset, all bright spots on the image to be registered after the first registration are moved to achieve registration of the image to be registered.
  • This method uses two association registrations, which can be relatively referred to as coarse registration and fine registration, including fine registration using bright spots on the image.
  • single-molecule-level image detection such as images of sequencing reactions from third-generation sequencing platforms.
  • the so-called single molecule level refers to the size of a single or a few molecules, such as 10, 8, 5, 4, or less molecules.
  • the images to be registered are from a sequencing platform that uses optical imaging principles for sequence determination.
  • sequencing also called sequence determination, refers to nucleic acid sequence determination, including DNA sequencing and / or RNA sequencing, including long-sequence sequencing and / or short-sequence sequencing, and sequencing biochemical reactions include base extension. Sequencing can be performed through a sequencing platform.
  • the sequencing platform can be selected but not limited to Hisq / Miseq / Nextseq sequencing platform from Illumina, IonTorrent platform from Thermo Fisher / Life Technologies, BGISEQ platform and single molecule sequencing platform from BGI; sequencing method You can choose single-end sequencing or double-end sequencing; the obtained sequencing results / data are the read out fragments, which are called reads, and the length of the reads is called read length.
  • the so-called "bright spots" correspond to the optical signals of extended bases or clusters of bases.
  • the predetermined area on the so-called image may be the entire image or a part of the image.
  • the predetermined area on the image is a part of the image, such as a 512 * 512 area in the center of the image.
  • the so-called image center is the center of the field of view.
  • the intersection between the optical axis of the imaging system and the imaging plane can be referred to as the image center point, and the area centered on the center point can be regarded as the image center area.
  • the image to be registered comes from a nucleic acid sequencing platform
  • the platform includes an imaging system and a nucleic acid sample carrying system
  • the nucleic acid molecule to be tested with an optical detection label is fixed in a reactor, which is also called
  • the chip is mounted on a movable table, and the moving table drives the chip to realize image acquisition of the nucleic acid molecules to be tested located at different positions (different fields of view) of the chip.
  • there is a limit on the accuracy of the movement of the optical system and / or the mobile stage For example, there is a deviation between the specified movement to a certain position and the position reached by the actual movement of the mechanical structure, especially in application scenarios that require high accuracy.
  • the so-called reference image is obtained through construction, and the reference image can be constructed when the image to be registered is registered, or it can be constructed in advance to be saved when needed.
  • constructing the reference image includes: acquiring a first image and a second image, the first image and the second image corresponding to the same object as the image to be registered; and performing coarse registration of the second image based on the first image, including determining The offset of the second image and the first image, and based on the offset, the second image is moved to obtain a second image after coarse registration; the first image and the second image after coarse registration are merged to obtain a reference image ,
  • the first image and the second image each include multiple bright spots.
  • the use of multiple images to construct a reference image facilitates the reference image to obtain complete bright spot information of the corresponding nucleic acid molecule and facilitates correction of the bright spot-based image.
  • the first image and the second image are from the same field of view at different times of the nucleic acid sequence determination reaction (sequencing reaction), respectively.
  • a round of sequencing reactions includes multiple base extension reactions, such as monochrome sequencing, and the reaction substrates (nucleotide analogs) corresponding to the four types of bases all carry the same fluorescent dye.
  • the round sequencing reaction includes four base extension reactions.
  • one base extension reaction includes one image acquisition, and the first image and the second image are the same field of view for different base extension reactions. In this way, the reference image obtained by processing and collecting the information of the first image and the second image is used as a reference for correction, which facilitates more accurate image correction.
  • a single-molecule, two-color sequencing reaction uses two types of bases (nucleotide analogs) with one fluorescent dye and two with a different Excitation wavelength fluorescent dye.
  • One round of sequencing reaction includes two base extension reactions. Two types of base reaction substrates with different dyes are combined in one base extension reaction. For one field of view, one base extension The reaction includes two image acquisitions at different excitation wavelengths. The first image and the second image are from different base extension reactions or the same field of view at different excitation wavelengths in the same base extension reaction. In this way, the reference image obtained by processing and collecting the information of the first image and the second image is used as a reference for correction, which facilitates more accurate image correction.
  • a round of sequencing reactions includes a base extension reaction, such as a two-color sequencing reaction of a second-generation sequencing platform, and the four types of base reaction substrates (such as nucleotide analogs) have dyes a, With dye b, with dye a and dye b, and without any dye, the excitation wavelengths of dye a and dye b are different; four types of reaction substrates achieve a round of sequencing reactions in the same base extension reaction, the first The image and the second image are from the same field of view in different rounds of sequencing reactions or different excitation wavelengths in the same round of sequencing reactions, respectively.
  • the reference image obtained by processing and collecting the information of the first image and the second image is used as a reference for correction, which facilitates more accurate image correction.
  • the first image and / or the second image may be one image or multiple images. Further, in some specific embodiments, the method further includes constructing a so-called reference image using the third image and the fourth image, and the image to be registered, the first image, the second image, the third image, and the fourth image are from a sequencing reaction.
  • the same field of view, the first image, the second image, the third image, and the fourth image respectively correspond to the field of view of the four types of base extension reactions of A / U, T, G, and C, and the field of view during the base extension reaction exists
  • a plurality of nucleic acid molecules with optically detectable labels, at least a portion of the nucleic acid molecules appear as bright spots on the image, and constructing the reference image further includes: coarsely registering the third image based on the first image, including determining the third image and the first image.
  • An image offset based on which the third image is moved to obtain a third image after coarse registration; performing a coarse registration on the fourth image based on the first image, including determining the fourth image and the first image An offset, based on which the fourth image is moved to obtain a fourth image after coarse registration; the first image is merged with the second image after coarse registration, the third image after coarse registration, and the coarse registration After the first Image, to obtain a reference image.
  • a Fourier transform can be used to determine the first offset by using frequency domain registration.
  • a Fourier transform can be used to determine the first offset by using frequency domain registration.
  • the first registration / coarse registration can achieve an accuracy of 1 pixel. In this way, the first offset can be determined quickly and accurately, and / or a reference image that facilitates accurate correction can be constructed.
  • the reference image and the image to be registered are binarized images. In this way, it is beneficial to reduce the amount of calculation and quickly correct the deviation.
  • the image to be corrected and the reference image are both binary images, that is, each pixel in the image is not a or b, for example, a is 1, b is 0, and a pixel mark of 1 is brighter than a pixel mark of 0.
  • the intensity is strong; here, the nucleic acid molecule to be tested is extended by one base or one base during the nucleic acid sequencing process, which is called a cycle.
  • the reference image is constructed using the images of cycle 1-4 of cycle 1-4.
  • the first image and the second image are selected from any one, two, or three of the images cycle1-cycle4.
  • the first image is the image cycle1
  • the image cycle2-4 is the second image.
  • the images cycle2-4 are sequentially coarsely registered to obtain the coarsely registered images cycle2-4, respectively; the merged images
  • a reference image was obtained from cycle1 and the coarsely registered image cycle2-4.
  • the so-called merged image is an overlapping bright spot in the merged image. It is mainly based on the size of the bright spot of the corresponding nucleic acid molecule and the resolution of the imaging system. In one example, two bright spots with a distance of no more than 1.5 pixels on the two images are set as coincident bright spots.
  • the composite image center area of 4 cycles is used as the reference image, which is helpful to make the reference image have a sufficient amount of bright spots and facilitate subsequent registration, and secondly, to detect and locate the bright spots in the central area of the image.
  • the speckle information is relatively more accurate and facilitates accurate registration.
  • the following steps are performed to correct the image: 1) Rough correction is performed on the cycle5 image of a certain field of vision collected from the fifth round of response.
  • Cycle5 is a binary image, and the center of the image is taken as 512 * 512.
  • two bright spots with a distance of no more than 1.5 pixels on the two images are set as coincident bright spots; 3)
  • a field-of-view image (fov) with offsets (x0, y0) of different cycles is obtained.
  • -(x1, y1), for a bright spot (peak) can be expressed as: curCyclePoints + (x0, y0)-(x1, y1)
  • curCyclePoints represents the original coordinates of the bright spot, that is, the coordinates in the image before correction.
  • the correction result obtained by the above image correction has higher accuracy, and the correction accuracy is less than or equal to 0.1 pixels.
  • Figure 2 shows the correction process and results.
  • image C is corrected based on image A.
  • the circles in image A and image C indicate bright spots.
  • Bright spots with the same digital mark are coincident bright spots.
  • Image C-> A indicates The correction result, that is, the image C is aligned to the image A.
  • the image registration method further includes S01 to identify bright spots, including detecting bright spots on the image by using a k1 * k2 matrix, and determining whether the central pixel value of the matrix is not less than any non-matrix of the matrix.
  • the matrix of pixel values corresponds to a candidate bright spot, and it is determined whether the candidate bright spot is a bright spot.
  • Both k1 and k2 are odd numbers greater than 1, and the k1 * k2 matrix contains k1 * k2 pixels.
  • the so-called image is selected from at least one of an image to be registered and an image constructing a reference image.
  • this method to detect bright spots on an image can quickly and effectively detect bright spots (spots or peaks) on an image, especially for images collected from a nucleic acid sequence determination reaction.
  • the method has no special restrictions on the detection images, ie, the original input data, and is applicable to the processing and analysis of images generated by any platform that uses the principle of optical detection for nucleic acid sequence determination, including but not limited to second- and third-generation sequencing. Efficient feature, can get more representative sequence information from the image. It is especially advantageous for signal recognition with random images and high accuracy requirements.
  • the image is derived from a nucleic acid sequence determination reaction.
  • the nucleic acid molecule is provided with an optically detectable label, such as a fluorescent label.
  • the fluorescent molecule can be excited to emit fluorescence under laser irradiation at a specific wavelength, and the image is acquired by an imaging system.
  • the acquired images include light spots / bright spots that may correspond to the location of the fluorescent molecules. Understandably, when in the focal position, the size of the bright spot corresponding to the position of the fluorescent molecule in the collected image is small and the brightness is high; when it is in the non-focus position, the collected image The size of the bright spot corresponding to the position of the fluorescent molecule is larger and the brightness is lower.
  • the so-called single molecule is a few molecules, for example, the number of molecules is not more than 10, for example, one, two, three, four, five, six, eight or ten.
  • the central pixel value of the matrix is greater than the first preset value
  • any non-central matrix pixel value is greater than the second preset value
  • the first preset value and the second preset value are related to the average pixel value of the image.
  • a k1 * k2 matrix may be used to perform ergodic detection on the image, and the setting of the so-called first preset value and / or the second preset value is related to the average pixel value of the image.
  • the pixel values are the same as the grayscale values.
  • k1 * k2 matrix, k1 and k2 may be equal or unequal.
  • the relevant parameters of the imaging system are: the objective lens is 60 times, the size of the electronic sensor is 6.5 ⁇ m, and the image formed by the microscope and then passed through the electronic sensor, the minimum size that can be seen is 0.1 ⁇ m.
  • It can be a 16-bit grayscale or color image of 512 * 512, 1024 * 1024, or 2048 * 2048.
  • the inventor has performed a large number of image processing statistics, taking the first preset value as 1.4 times the average pixel value of the image, and taking the second preset value as 1.1 times the average pixel value of the image, which can eliminate interference, Obtain bright spot detection results from the optical detection mark.
  • the size, similarity and / or intensity of the ideal bright spot can be used to further screen and judge candidate bright spots.
  • the size of the candidate bright spots on the comparison image is quantitatively reflected by using the size of the connected domain corresponding to the candidate bright spots, so as to filter and determine whether the candidate bright spots are the desired bright spots.
  • the size defines the connected pixels in a k1 * k2 matrix that are larger than the average pixel value as a connected domain corresponding to a so-called candidate bright spot. In this way, it is possible to effectively obtain bright spots corresponding to the labeled molecules and conforming to subsequent sequence recognition, and obtain nucleic acid sequence information.
  • the average pixel value of the image is used as a reference, and two or more adjacent pixels that are not less than the average pixel value are called connected pixels / connectivity, as shown in FIG. 4, Bold and enlarged represents the center of the matrix corresponding to the candidate bright spot, and the thick line frame represents the 3 * 3 matrix corresponding to the candidate bright spot.
  • the so-called third preset value may be determined according to the information of the size of the connected domain corresponding to all candidate bright spots on the image. For example, by calculating the size of the connected domain corresponding to each candidate bright spot on the graph, taking the average value of the size of the connected domain of the bright spots represents a characteristic of the image as a third preset value; for example, each candidate in the image may be The size of the connected domain corresponding to the bright spot is sorted from small to large, and the size of the connected domain at the 50th, 60th, 70th, 80th, or 90th quantile is taken as the third preset value. In this way, the bright spot information can be effectively obtained, which is beneficial for subsequent recognition of the nucleic acid sequence.
  • the candidate bright spots are screened by statistically setting parameters to quantitatively reflect the intensity characteristics of the comparison candidate bright spots.
  • the so-called fourth preset value may be determined according to the information of the magnitudes of the scores of all candidate bright spots on the image. For example, when the number of candidate bright spots on the image is greater than a certain number, which meets the statistical requirements, for example, the number of candidate bright spots on the image is greater than 30, the score values of all candidate bright spots on the image can be calculated and Ascending order, the fourth preset value can be set to the 50th, 60th, 70th, 80th, or 90th quantile Score value, so that less than 50th, 60th, 70th, 80th, or 90th can be excluded.
  • the candidate bright spots of the quantile Score value are helpful for effectively obtaining the target bright spots and accurate subsequent recognition of the base sequence.
  • the basis for performing this processing or the screening setting is that, generally, it is considered that the bright spots that have a large difference in intensity and pixel value between the center and the edge and are converged are bright spots corresponding to the location of the molecule to be detected.
  • the number of candidate bright spots on the image is greater than 50, greater than 100, or greater than 1,000.
  • candidate bright spots are screened for morphology and intensity / brightness.
  • a connected pixel that is larger than the average pixel value in a k1 * k2 matrix as a connected field corresponding to a so-called candidate bright spot.
  • CV represents the candidate bright spot.
  • the center pixel value of the corresponding matrix, EV represents the sum of the non-center pixel values of the matrix corresponding to the bright spot; the candidate bright spots whose size of the corresponding connected domain is greater than the third preset value and the score is greater than the fourth preset value are A bright spot.
  • the so-called third preset value and / or fourth preset value may be considered and set with reference to the foregoing specific implementation manner.
  • the image registration method further includes S03 to identify bright spots.
  • the image to be registered and / or the reference image is from a field where the base extension reaction occurs, and the field where the base extension reaction occurs.
  • S03 includes: pre-processing the image to obtain a pre-processed image. The so-called image is selected from the images to be registered.
  • determining a critical value to simplify the preprocessed image including assigning a pixel value of a pixel point on the preprocessed image that is less than the critical value to a first preset value, The pixel value of the pixels on the preprocessed image that is less than the critical value is assigned a second preset value to obtain a simplified image; the first bright spot detection threshold c1 is determined based on the preprocessed image; based on the preprocessed image And simplified image recognition of candidate bright spots, including determining a pixel matrix that satisfies at least two of the following a) -c) as a candidate bright spot, a) the image after preprocessing , The pixel value of the central pixel point of the pixel point matrix is the largest, the pixel point matrix can be expressed as r1 * r2, r1 and r2 are both odd numbers greater than 1, the r1 * r2 pixel point matrix contains r1 * r2
  • G1 is the correlation coefficient of the two-dimensional Gaussian distribution in the range of m1 * m2 centered on the central pixel point of the pixel matrix.
  • G2 For the pixels in the m1 * m2 range, m1 and m2 are both odd numbers greater than 1, and the m1 * m2 range includes m1 * m2 pixels; and determining whether the candidate bright spot is a bright spot.
  • the detection of bright spots on an image using this method includes the use of judgment conditions or a combination of judgment conditions determined by the inventor through a large amount of data training, which can quickly and effectively detect the bright spots on the image, especially in response to the determination of the nucleic acid sequence collected Image.
  • the method has no special restrictions on the detection images, ie, the original input data, and is applicable to the processing and analysis of images generated by any platform that uses the principle of optical detection for nucleic acid sequence determination, including but not limited to second- and third-generation sequencing. Efficient feature, can get more representative sequence information from the image. It is especially advantageous for signal recognition with random images and high accuracy requirements.
  • the pixel values are the same as the grayscale values. If the image is a color image and one pixel of the color image has three pixel values, the color image can be converted into a grayscale image and then bright spot detection can be performed to reduce the calculation amount and complexity of the image detection process. You can choose, but are not limited to, converting non-grayscale images to grayscale images using floating-point algorithms, integer methods, shifting methods, or average methods.
  • preprocessing the image includes: determining the background of the image using an open operation; converting the image into a first image using a top hat operation based on the background; performing Gaussian blur processing on the first image to obtain a second image; The two images are sharpened to obtain the so-called pre-processed image.
  • the image can be effectively reduced in noise or the signal-to-noise ratio of the image can be improved, which is beneficial to the accurate detection of bright spots.
  • the open operation is a morphological process, that is, the process of expanding and then corroding.
  • the etching operation will make the foreground (the part of interest) smaller, and the expansion will make the foreground larger.
  • the open operation can be used to eliminate small objects. Separates objects at points and does not significantly change their area while smoothing the boundaries of larger objects.
  • the size of the structural elements p1 * p2 (the basic template used to process the image) for the image open operation is not particularly limited, and p1 and p2 are odd numbers.
  • the structural elements p1 * p2 may be 15 * 15, 31 * 31, and the like, and finally, a pre-processed image that is favorable for subsequent processing and analysis can be obtained.
  • the top hat operation is often used to separate plaques that are brighter than neighboring points (bright spots / bright spots). In an image with a large background and small objects are more regular, the top hat operation can be used for background extraction.
  • performing a top hat transformation on an image includes first performing an open operation on the image, and then subtracting the result of the open operation from the original image to obtain a first image, which is the top hat transformed image.
  • the inventor believes that the result of the open operation enlarges the crack or local low-luminance area, so the image obtained after subtracting the open operation from the original image highlights a brighter area than the area around the outline of the original image.
  • the operation is related to the size of the selected kernel. It can be considered to be related to the expected size of the bright spots / bright spots. If the bright spots are not the expected size, the processed effect will cause a lot of small bumps in the whole picture. For details, refer to the virtual focus picture, that is, Bright spots / bright spots halo. In one example, the expected size of the bright spot, that is, the size of the selected kernel is 3 * 3, and the obtained top-hat transformed image is beneficial for subsequent further denoising processing.
  • Gaussian Blur also known as Gaussian filtering
  • Gaussian filtering is a linear smoothing filter that is suitable for eliminating Gaussian noise and is widely used in image reduction noise reduction processes.
  • Gaussian filtering is a process of weighted average of the entire image. The value of each pixel is obtained by weighted average of itself and other pixel values in the neighborhood.
  • the specific operation of Gaussian filtering is: use a template (or convolution, mask) to scan each pixel in the image, and use the weighted average gray value of the pixels in the neighborhood determined by the template to replace the value of the central pixel of the template.
  • Gaussian blur processing is performed on the first image, and the Gaussian Blur function is used in OpenCV.
  • the Gaussian distribution parameter Sigma is 0.9.
  • the two-dimensional filter matrix (convolution kernel) used is 3 * 3.
  • the Gaussian blur processing is performed on the image angle, the small protrusions on the first image are smoothed, and the edges of the image are smooth.
  • the second image that is, the Gaussian filtered image is sharpened, for example, two-dimensional Laplacian sharpening is performed. After processing from an image perspective, the edges are sharpened, and the Gaussian blurred image is restored.
  • simplifying the pre-processed image includes: determining a critical value based on the background and the pre-processed image; comparing a pixel value of a pixel point on the pre-processed image with a critical value, The pixel value of the pixel point on the processed image is assigned a first preset value, and the pixel value of the pixel point on the preprocessed image not less than a critical value is assigned a second preset value to obtain a simplified image.
  • the pre-processed image is simplified, such as binarization, which is conducive to accurate detection of subsequent bright spots, accurate identification of subsequent bases, Get high-quality data and more.
  • obtaining a simplified image includes: dividing the sharpened result obtained after preprocessing by the result of an on operation to obtain a set of values corresponding to the image pixels; and determining the binarization through the set of values The critical value of the preprocessed image.
  • the set of values can be sorted in ascending order, and the value corresponding to the 20th, 30th, or 40th percentile of the set of values is taken as the binarization threshold / threshold. In this way, the obtained binarized image facilitates accurate detection and recognition of subsequent bright spots.
  • the structure element of the open operation during image preprocessing is p1 * p2, which is called dividing the preprocessed image (the sharpened result) by the result of the operation to obtain a set of the same size as the structure element.
  • Array / matrix p1 * p2 in each array, the p1 * p2 values contained in the array are sorted in ascending order, and the value corresponding to the thirtieth percentile in the array is taken as the area (numerical matrix)
  • the threshold / threshold of the binarization of the image In this way, the thresholds are determined to binarize each area on the image.
  • the resulting binarization result highlights the required information while denoising, which is conducive to the accurate detection of subsequent bright spots. .
  • the determination of the first bright spot detection threshold is performed using the Otsu method.
  • the Otsu method can also be called the maximum inter-class variance method.
  • the Otsu method uses the largest inter-class variance to segment the image, which means that the probability of misclassification is small and the accuracy is high.
  • T (c1) the foreground and background segmentation threshold of the preprocessed image
  • the proportion of pixels belonging to the foreground to the entire image is w0
  • the average gray scale is ⁇ 0
  • the ratio is w1, and its average gray scale is ⁇ 1.
  • the traversal method is used to obtain the segmentation threshold T that maximizes the variance between classes, that is, the first bright spot detection threshold c1 obtained.
  • identifying candidate bright spots on the image based on the pre-processed image and the simplified image includes determining that a pixel point matrix that simultaneously meets a) -c) three conditions is a candidate bright spot. In this way, the accuracy of subsequent determination of the nucleic acid sequence based on the bright spot information and the quality of the offline data can be effectively improved.
  • the conditions that need to be satisfied for determining the candidate bright spots include a), k1 and k2 may be equal or unequal.
  • the relevant parameters of the imaging system are: the objective lens is 60 times, the size of the electronic sensor is 6.5 ⁇ m, and the image formed by the microscope and then passed through the electronic sensor, the minimum size that can be seen is 0.1 ⁇ m. It can be a 16-bit grayscale or color image of 512 * 512, 1024 * 1024, or 2048 * 2048.
  • the values of k1 and k2 are both greater than 1 and less than 10.
  • the conditions that need to be met for determining the candidate bright spots include b).
  • the pixel value of the central pixel of the pixel matrix is a second preset value
  • the connected pixels of the pixel matrix are greater than 2 / 3 * k1 * k2, that is, the pixel value of the central pixel is larger than the critical value and the connected pixels are larger than two thirds of the matrix.
  • two or more pixels whose adjacent pixel values are the second preset value are called connected pixels / connectivity.
  • a simplified image is a binary image, and the first preset value is It is 0, and the second preset value is 1. As shown in FIG.
  • the pixel point matrix does not satisfy the condition b), and is not a candidate bright spot.
  • the conditions that need to be met for determining the candidate bright spot include c).
  • g2 is a pixel in the range of m1 * m2 after correction, that is, the sum of pixels in the range of m1 * m2 after correction.
  • the so-called determining whether the candidate bright spot is a bright spot further includes: determining a second bright spot detection threshold based on the pre-processed image, and determining that the candidate bright spot whose pixel value is not less than the second bright spot detection threshold is Bright spots.
  • the pixel value of the pixel point at which the coordinates of the candidate bright spot are located is used as the pixel value of the candidate bright spot.
  • the center of gravity method can be used to obtain the coordinates of candidate bright spots, including sub-pixel-level coordinates.
  • the gray value of the coordinate position of the candidate bright spot is calculated by a bilinear interpolation method.
  • determining whether the candidate bright spot is a bright spot includes: dividing the pre-processed image into a set of blocks of a predetermined size, and sorting the pixel values of the pixels in the region to determine The second bright spot detection threshold corresponding to the region; for a candidate bright spot located in the region, it is determined that the candidate bright spot whose pixel value is not less than the second bright spot detection threshold corresponding to the region is a bright spot. In this way, distinguishing the differences in different areas of the image, such as the overall drop in light intensity, and further detecting and identifying bright spots, is conducive to accurately identifying bright spots and obtaining more bright spots.
  • the so-called pre-processed image is divided into a set of blocks of a predetermined size, and there may or may not be overlap between the blocks. In one example, there is no overlap between blocks.
  • the size of the pre-processed image is not less than 512 * 512, such as 512 * 512, 1024 * 1024, 1800 * 1800, or 2056 * 2056, etc., and the area of the predetermined size may be set to 200 * 200. In this way, it is beneficial to quickly calculate and identify bright spots.
  • the pixel values of the pixels in each block are arranged in ascending order by size, and p10 + (p10-p1) * 4.1 is taken as the corresponding value of the block.
  • the second bright spot detection threshold that is, the background of the block
  • p1 represents the pixel value of the hundredth percentile
  • p10 represents the pixel value of the tenth percentile.
  • the threshold is a relatively stable threshold obtained by the inventor through a large amount of data training tests, and can eliminate bright spots on a large number of backgrounds. Understandably, when the optical system is adjusted and the overall pixel distribution of the image is changed, this threshold may need to be adjusted appropriately.
  • FIG. 7 is a comparison diagram of the bright spot detection results before and after the process, that is, the bright spot detection results before and after the area background is excluded.
  • the upper half of FIG. 7 is the bright spot detection results after the processing, and the lower half.
  • the cross marks are candidate bright spots or bright spots.
  • a "computer-readable storage medium” may be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. .
  • computer-readable storage media include the following: electrical connections (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM) , Read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disk read-only memory (CDROM).
  • the computer-readable storage medium may even be paper or other suitable media on which the program can be printed, because, for example, by optically scanning the paper or other media and then editing, interpreting, or otherwise Processing is performed in a suitable manner to obtain the program electronically and then store it in a computer memory.
  • an embodiment of the present invention provides an image registration device 100, which is used to implement all or part of the steps of the image registration method in any of the above embodiments.
  • the device 100 includes: a first registration module 110, configured to perform first registration on the image to be registered based on the reference image, including determining a first offset of a predetermined region on the image to be registered and a corresponding predetermined region on the reference image, and moving the target based on the first offset.
  • the second registration Module 120 configured to perform second registration on the first to-be-registered image from the first registration module based on the reference image, including merging the to-be-registered image and the reference image after the first registration to obtain a merge Image, calculate the offset of all overlapping bright spots in a predetermined area on the merged image to determine a second offset, and two or more bright spots with a distance less than a predetermined pixel are one overlapping bright spot, based on the second offset shift All bright spots with movement of the first registration image to be equipped on the registration, registration is done to achieve image registration.
  • the image registration device 100 further includes a reference image construction module 109.
  • the reference image construction module is configured to construct a reference image, including: acquiring a first image and a second image, the first image and the second image being The registration image corresponds to the same object; performing coarse registration on the second image based on the first image includes determining an offset between the second image and the first image, and moving the second image based on the offset to obtain the coarse registration.
  • the second image; the first image and the second image after coarse registration are combined to obtain a reference image, and the first image and the second image each include multiple bright spots.
  • constructing the reference image in the reference image construction module 109 further includes using a third image and a fourth image, the image to be registered, the first image, the second image, the third image, and the fourth image from the sequencing reaction.
  • the same field of view, the first image, the second image, the third image, and the fourth image correspond to the field of view of the four types of base extension reactions of A / U, T, G, and C, respectively.
  • Nucleic acid molecules with optically detectable labels at least a portion of the nucleic acid molecules appear as bright spots on the image
  • constructing a reference image further includes: coarsely registering the third image based on the first image, including determining the third image and the first Image offset, based on which the third image is moved to obtain a third image after coarse registration; coarse registration of the fourth image based on the first image includes determining the offset of the fourth image and the first image A shift amount, based on which the fourth image is moved to obtain a fourth image after coarse registration; the first image is merged with the second image after coarse registration, the third image after coarse registration, and after coarse registration Fourth Like, to obtain the reference image.
  • the reference image and the image to be registered are binarized images.
  • a two-dimensional discrete Fourier transform is used to determine the first offset, the second image and the first image offset, the third image and the first image offset, and / or the fourth image. Offset from the first image.
  • the image registration device 100 further includes a first bright spot detection module 107.
  • the first bright spot detection module is configured to use the k1 * k2 matrix to perform bright spot detection on the image, and determine that the center pixel value of the matrix is not less than the matrix
  • the matrix of any pixel value in the center corresponds to a candidate bright spot, and it is determined whether the candidate bright spot is a bright spot.
  • Both k1 and k2 are odd numbers greater than 1, and the k1 * k2 matrix contains k1 * k2 pixels.
  • the central pixel value of the so-called matrix is greater than the first preset value
  • any pixel value of the non-matrix of the matrix is greater than the second preset value
  • the first preset value and / or the second preset value and the The average pixel value is correlated.
  • the candidate bright spot of the third preset value is one bright spot
  • A represents the connected size of the row where the center of the matrix corresponding to the candidate bright spot is
  • B represents the connected size of the column where the center of the matrix corresponding to the candidate bright spot
  • Define connected pixels that are larger than the average pixel value as a connected domain, and / or calculate the score of a candidate bright spot Score ((k1 * k2-1) CV-EV) / ((CV + EV) / (k1 * k2)), it is determined that the candidate bright spot with a score greater than the fourth preset value is a bright spot
  • CV represents the central pixel value of the matrix corresponding to the candidate bright spot
  • EV represents the sum of the non-center pixel values of the matrix corresponding to
  • the image registration device 100 further includes a second bright spot detection module 105.
  • the image to be registered or any image constituting the reference image is collected from a field of view where the base extension reaction occurs.
  • the second bright spot detection module 105 is used to: preprocess the image to obtain a preprocessed image; determine the criticality Value to simplify the pre-processed image, including assigning a pixel value of a pixel value on a pre-processed image that is less than a critical value to a first preset value, and a pixel point on a pre-processed image that is not less than a critical value Is assigned a second preset value to obtain a simplified image; the first bright spot detection threshold c1 is determined based on the pre-processed image; candidate bright spots on the image are identified based on the pre-processed image and the simplified image, including determination A pixel matrix that meets at least two of the following conditions a) -c) is a candidate bright spot.
  • the image of the center pixel of the pixel matrix The prime value is the largest, the pixel matrix can be expressed as r1 * r2, r1 and r2 are both odd numbers greater than 1, r1 * r2 pixel matrix contains r1 * r2 pixels, b) in the simplified image, the pixel matrix The pixel value of the central pixel is a second preset value and the connected pixels of the pixel matrix are greater than 2/3 * r1 * r2, and c) the pixel value of the central pixel of the pixel matrix in the preprocessed image is greater than The third preset value satisfies g1 * g2> c1, g1 is a correlation coefficient of a two-dimensional Gaussian distribution in a range of m1 * m2 centered on a central pixel point of the pixel matrix, and g2 is a pixel in the range m1 * m2, m1 and m2 are both odd numbers greater than
  • determining whether the candidate bright spot is a bright spot includes: determining a second bright spot detection threshold based on the pre-processed image, and determining that the pixel value is not less than the second bright spot detection.
  • the candidate bright spots of the threshold are bright spots.
  • the pixel value of the candidate bright spot is the pixel value of the pixel point where the coordinates of the candidate bright spot are located.
  • the second bright spot detection module 105 determines a second bright spot detection threshold based on the pre-processed image, and determines that the candidate bright spot having a pixel value not less than the second bright spot detection threshold is a bright spot, including:
  • the pre-processed image is divided into a set of regions of a predetermined size, and the pixel values of the pixels in the region are sorted to determine the second bright spot detection threshold corresponding to the region.
  • a candidate bright spot whose pixel value is not less than the second bright spot detection threshold corresponding to the area is a bright spot.
  • preprocessing the image includes: determining the background of the image using an open operation, converting the image to a first image using a top hat operation based on the background, and performing Gaussian blur on the first image Process to obtain a second image, sharpen the second image, and obtain a pre-processed image.
  • determining a threshold value to simplify the pre-processed image, and obtaining a simplified image includes: determining the threshold value based on the background and the pre-processed image, and comparing the pre-processed image The pixel value and the critical value of the pixel on the pixel to obtain a simplified image.
  • g2 is a pixel in the corrected m1 * m2 range, and according to the corresponding pixel in the simplified m1 * m2 range, the pixel value is the second preset value.
  • the ratio is corrected.
  • determining whether the candidate bright spot is a bright spot further includes: if it is determined that the candidate bright spot is a bright spot, calculating a sub-bright spot. The intensity value of the pixel center coordinate and / or the sub-pixel center coordinate. If it is determined that the candidate bright spot is not a bright spot, the candidate bright spot is discarded.
  • An embodiment of the present invention also provides a computer program product.
  • the product includes instructions.
  • the instructions When the computer executes the program, the instructions cause the computer to perform all or part of the steps of the image registration method in any of the foregoing embodiments.
  • controller in addition to implementing the controller / processor in a pure computer-readable program code manner, the controller can be controlled by logic gates, switches, ASICs, and editable logic by logically changing the method steps. Controller and embedded microcontroller to achieve the same function. Therefore, such a controller / processor can be considered as a hardware component, and a device included therein for implementing various functions can also be considered as a structure within the hardware component. Or even, the means for implementing various functions can be regarded as a structure that can be both a software module implementing the method and a hardware component.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种图像配准方法、装置和计算机程序产品。所述的图像配准方法包括：基于参考图像对待配准图像进行第一配准，包括确定待配准图像上的预定区域和参考图像上的相应预定区域的第一偏移量，基于第一偏移量移动待配准图像上的所有亮斑；基于参考图像对第一配准后的待配准图像进行第二配准，包括合并第一配准后的待配准图像和参考图像，计算合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个重合亮斑，基于该第二偏移量移动第一配准后的待配准图像上的所有亮斑，以实现对待配准图像的配准。该方法能够实现高精度图像配准，满足对图像纠偏高精度要求的情景。

Description

图像配准方法、装置和计算机程序产品 技术领域
本发明涉及图像处理领域，具体地，涉及一种图像配准方法、一种图像配准装置和一种具有图像配准功能的计算机程序产品。
背景技术
在包含了于不同时间多次采集相同一个或多个对象的图像的应用中，一般需要对获得的多个图像进行纠偏/配准，使能够基于纠偏后的图像准确获取该对象的变化信息。
在包含利用获取核酸分子的图像进行核酸序列测定的平台中，一般需要在移动硬件、在不同时刻对同一个视野中的核酸分子进行图像采集，根据不同时刻拍摄得的多个图像包括识别图像中的信息，可确定该视野中的核酸分子的序列信息。在实际的图像采集中，由于硬件移动具有一定精度，即指定移动量和实际移动量具有一定误差，和/或由于该核酸分子所处的环境/体系的变化造成的核酸分子形态等的变化，会使得获得的多个时刻的该视野的图像中的固定的核酸分子的位置信息不同，使得难以直接利用获得的图像信息准确识别确定该核酸分子的序列。
由此，对不同时间获取的相同的一个或多个对象的多个图像进行纠偏的方法，有待进一步开发或者改进。
发明内容
本发明实施方式旨在至少解决相关技术中存在的技术问题之一或者至少提供一种可选择的实用方案。
依据本发明的一个实施方式，提供一种图像配准方法，该方法包括：基于参考图像对待配准图像进行第一配准，所称的参考图像和待配准图像对应相同对象，参考图像和待配准图像均包含多个亮斑，包括确定待配准图像上的预定区域和所称参考图像上的相应预定区域的第一偏移量，基于该第一偏移量移动待配准图像上的所有亮斑，获得第一配准后的待配准图像；基于参考图像对第一配准后的待配准图像进行第二配准，包括合并所述第一配准后的待配准图像和参考图像，获得合并图像，计算合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个所述重合亮斑，基于该第二偏移量移动第一配准后的待配准图像上的所有亮斑，以实现对待配准图像的配准。
依据本发明的另一个实施方式，提供一种图像配准装置，该装置用以实施上述本发明实施方式中的图像配准方法，该装置包括：第一配准模块，用于基于参考图像对待配准图像进行第一配准，包括确定待配准图像上的预定区域和参考图像上的相应预定区域的第一偏移量，基于第一偏移量移动待配准图像上的所有亮斑，获得第一配准后的待配准图像，所称的参考图像和待配准图像对应相同对象，参考图像和待配准图像均包含多个亮斑；第二配准模块，用于基于参考图像对来自第一配准模块的第一配准后的待配准图像进行第二配准，包括合并第一配准后的待配准图像和参考图像，获得合并图像，计算合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，定义距离小于预定像素的两个或多个亮斑为一个重合亮斑，基于该第二偏移量移动第一配准后的待配准图像上的所有亮斑，以实现对待配准图像的配准。
依据本发明的又一个实施方式，提供一种计算机可读储存介质，用于存储供计算机执行的程序，执行所述程序包括完成上述任一实施方式中的图像配准方法。计算机可读存储介质包括但不限于只读存储器、随机存储器、磁盘或光盘等。
依据本发明的一个实施方式，还提供一种终端，一种计算机程序产品，包括指令，该指令在计算机执行所称的程序时，使计算机执行上述本发明实施方式中的图像配准方法的全部或部分步骤。
利用上述本发明实施方式中的图像配准方法、装置和/或包含实现图像配准的终端/计算机程序产品，能够实现图像的高精度纠偏，特别适于高精度图像纠偏要求的场景。
本发明实施方式的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本发明实施方式的实践了解到。
附图说明
图1是本发明具体实施方式中的图像配准方法的流程示意图。
图2是本发明具体实施方式中的图像纠偏过程及纠偏结果示意图。
图3是本发明的具体实施方式中的图像配准方法的流程示意图。
图4是本发明具体实施方式中的候选亮斑的对应的矩阵以及连同像素示意图。
图5是本发明具体实施方式中的图像配准方法的流程示意图。
图6是本发明具体实施方式中的以像素点矩阵的中心像素点为中心的m1*m2范围的像素值示意图。
图7是本发明具体实施方式中的依据第二亮斑检测阈值进行判定之前和之后的亮斑检测结果对比示意图。
图8是本发明具体实施方式中的图像配准装置示意图。
具体实施方式
下面详细描述本发明的实施方式，所述实施方式的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施方式是示例性的，仅用于解释本发明，而不能理解为对本发明的限制。
在本发明的描述中，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量或者顺序。在本发明的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。
本发明实施方式提供一种图像配准方法，亦即一种图像纠偏方法，如图1所示，包括：S10基于参考图像对待配准图像进行第一配准，参考图像和待配准图像对应相同对象，参考图像和待配准图像均包含多个亮斑，包括确定待配准图像上的预定区域和参考图像上的相应预定区域的第一偏移量，基于第一偏移量移动待配准图像上的所有亮斑，获得第一配准后的待配准图像；S20基于参考图像对第一配准后的待配准图像进行第二配准，包括合并第一配准后的待配准图像和参考图像，获得合并图像，计算合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个重合亮斑，基于该第二偏移量移动第一配准后的待配准图像上的所有亮斑，以实现对待配准图像的配准。该方法通过两次关联配准，可相对称为粗配准和细配准，包括利用图像上的亮斑进行细配准，能够基于少量数据信息快速地实现图像的高精度纠偏，特别适 于高精度图像纠偏要求的场景。例如，单分子级别的图像检测，比如来自第三代测序平台的测序反应的图像。所称单分子级别指分辨率为单个或少数几个分子的大小，例如10个、8个、5个、4个或3个以下分子。
所称的“亮斑”(也称为“亮点”，spots或peaks)，指图像上的发光点，一个发光点占有至少一个像素点。所称“像素点”同“像素”。
在某些具体实施方式中，待配准图像来自利用光学成像原理进行序列测定的测序平台。所称的测序，也称为序列测定，指核酸序列测定，包括DNA测序和/或RNA测序，包括长片段测序和/或短片段测序，测序生化反应包括碱基的延伸。测序可以通过测序平台进行，测序平台可选择但不限于Illumina公司的Hisq/Miseq/Nextseq测序平台、Thermo Fisher/Life Technologies公司的Ion Torrent平台、华大基因的BGISEQ平台和单分子测序平台；测序方式可以选择单端测序，也可以选择双末端测序；获得的测序结果/数据即测读出来的片段，称为读段(reads)，读段的长度称为读长。所称的“亮斑”对应延伸碱基或碱基簇的光学信号。
所称的图像上的预定区域，可以是整个图像，也可以是图像的一部分。在一个示例中，图像上的预定区域为图像的一部分，例如为图像中心的512*512区域。所称的图像中心，为该视野的中心，成像系统的光轴与成像平面的交点可称为图像中心点，以该中心点为中心的区域可视为图像中心区域。
在某些具体实施方式中，待配准图像来自核酸测序平台，该平台包括成像系统和核酸样本承载系统，带有光学检测标记的待测核酸分子固定于反应器中，该反应器也称为芯片，芯片装载在一个可移动台子上，通过该移动台子带动芯片运动来实现对位于芯片不同位置(不同视野)的待测核酸分子进行图像采集。一般地，光学系统和/或移动台子的运动存在精度限制，例如，指令指定运动至某个位置和该机械结构实际运动达到的位置存在偏差，特别是在对精度高要求的应用情景，由此，在依据指令移动硬件以对不同时间点的同一位置(视野)进行多次图像采集的过程中，不同时间点采集的同一视野的多个图像难以完全对齐，对该些图像进行纠偏对齐，有利于基于该多个时间点采集的多个图像中的信息的变化来准确确定核酸分子核苷酸顺序。
在某些具体实施方式中，所称的参考图像是通过构建获得的，参考图像可以在对待配准图像进行配准时构建，也可以预先构建保存需要时调用。
在一些示例中，构建参考图像包括：获取第一图像和第二图像，第一图像和第二图像与待配准图像对应相同对象；基于第一图像对第二图像进行粗配准，包括确定第二图像和第一图像的偏移量，基于该偏移量移动第二图像，获得粗配准后的第二图像；合并第一图像和粗配准后的第二图像，以获得参考图像，第一图像和第二图像均包含多个亮斑。如此，利用构建获得包含更多或相对更完整的信息的图像，利用该图像作为纠偏的基准，利于实现更准确的图像配准。对于核酸序列测定得到的图像，利用多个图像进行参考图像构建，利于使得该参考图像获得完整的对应核酸分子的亮斑信息，利于基于亮斑的图像纠偏。
在一些实施例中，第一图像、第二图像分别来自核酸序列测定反应(测序反应)的不同时刻的同一个视野。这里，定义实现A/U、T、G和C四种类型碱基的一次延伸为实现一轮测序反应。在一个示例中，一轮测序反应包括多次碱基延伸反应，例如单色测序，利用的四种类型碱基对应的反应底物(核苷酸类似物)均带有同一种荧光染料，一轮测序反应包括四次碱基延伸反应(4repeats)，对于一个视野来说，一次碱基延伸反应包含一次图像采集，第一图像和第二图像分别为不同次的碱 基延伸反应的同一视野。如此，通过处理以及集合第一图像和第二图像的信息获得的参考图像作为纠偏的基准，利于进行更准确的图像纠偏。
在另一个示例中，单分子双色测序反应，利用的四种类型碱基对应的反应底物(核苷酸类似物)中的两种带有一种荧光染料、另两种带有另一种不同激发波长的荧光染料，一轮测序反应包括两次碱基延伸反应，带有不同染料的两种类型碱基反应底物于一次碱基延伸反应中进行结合反应，对于一个视野，一次碱基延伸反应包括两次于不同激发波长下的图像采集，第一图像和第二图像分别来自不同次的碱基延伸反应或者同一次碱基延伸反应中的不同激发波长下的同一视野。如此，通过处理以及集合第一图像和第二图像的信息获得的参考图像作为纠偏的基准，利于进行更准确的图像纠偏。
在又一个示例中，一轮测序反应包括一次碱基延伸反应，例如二代测序平台的双色测序反应，四种类型碱基反应底物(例如核苷酸类似物)分别带有染料a、带有染料b、带有染料a和染料b以及不带任何染料，染料a和染料b的激发波长不一样；四种类型反应底物于同一次碱基延伸反应中实现一轮测序反应，第一图像和第二图像分别来自不同轮测序反应或者同一轮测序反应中的不同激发波长下的同一视野。如此，通过处理以及集合第一图像和第二图像的信息获得的参考图像作为纠偏的基准，利于进行更准确的图像纠偏。
第一图像和/或第二图像，可以是一个图像也可以是多个图像。进一步地，在一些具体实施方式中，还包括利用第三图像和第四图像构建所称的参考图像，待配准图像、第一图像、第二图像、第三图像和第四图像来自测序反应的相同视野，第一图像、第二图像、第三图像和第四图像分别对应A/U、T、G和C四种类型碱基延伸反应时的视野，碱基延伸反应时的该视野存在多个带有光学可检测标记的核酸分子，至少一部分核酸分子在图像上表现为亮斑，构建参考图像还包括：基于第一图像对第三图像进行粗配准，包括确定第三图像和第一图像的偏移量，基于该偏移量移动第三图像，获得粗配准后的第三图像；基于第一图像对第四图像进行粗配准，包括确定第四图像和第一图像的偏移量，基于该偏移量移动第四图像，获得粗配准后的第四图像；合并第一图像和粗配准后的第二图像、粗配准后的第三图像以及粗配准后的第四图像，以获得参考图像。
S10中，对于第一配准的实现方式不作限制，例如可利用傅里叶变换，使用频域配准，确定第一偏移量。具体地，例如可参考Kenji TAKITA et al,IEICE TRANS.FUNDAMENTALS,VOL.E86-A,NO.8AUGUST 2003.中的纯相位相关函数(Phase-Only Correlation Function)中的二维离散傅里叶变换确定第一偏移量、第二图像和第一图像的偏移量、第三图像和第一图像的偏移量和/或第四图像和第一图像的偏移量。第一配准/粗配准可达到1像素(1pixel)的精度。如此，可快速准确地确定第一偏移量和/或构建利于精确纠偏的参考图像。
在某些具体实施方式中，参考图像和待配准图像为二值化图像。如此，利于减少运算量快速纠偏。
在一个示例中，待纠偏图像和参考图像均为二值化图像，即图像中的各个像素非a即b，例如a为1，b为0，像素标记为1的较像素标记为0的亮，或者说强度大；这里，定义核酸测序过程中待测核酸分子延伸一个碱基或者一种碱基称为一轮(cycle)，参考图像是利用第1-4轮的图像cycle1-cycle4构建的，第一图像、第二图像选自图像cycle1-cycle4中的任一个、两个或三个。
在一个示例中，第一图像为图像cycle1，图像cycle2-4为第二图像，基于图像图像cycle1依次对图像cycle2-4进行粗配准，分别获得粗配准后的图像cycle2-4；合并图像cycle1和粗配准后的图 像cycle2-4，获得参考图像。所称的合并图像为合并图像中的重合亮斑。主要基于对应核酸分子的亮斑的大小和成像系统分辨率，在一个示例中，设定两个图像上距离不大于1.5个像素的两个亮斑为重合亮斑。这里，采用4个cycle的合成的图像中心区域作为参考图像，一来利于使得参考图像具有足够量的亮斑，利于后续配准，二来检测及定位出的图像中心区域中的亮斑，亮斑信息是相对更准确的，利于准确配准。
在一个示例中，进行如下步骤对图像进行纠偏：1)对采集自第五轮反应的某个视野的图cycle5进行粗纠偏，cycle5为二值化后的图像，取该图像中心例如512*512区域，与cycle1-cycle4合成的中心图像(相应参考图像的中心512*512区域)，进行二维离散傅里叶变换，使用频域配准，得到偏移量offset(x0,y0)，即实现图像粗配准，x0、y0能达到1pixel的精度；2)将上述粗配准后的图像和参考图像基于图像上的亮斑进行合并(merge)，包括计算cycle5图像的中心区域内与参考图像相应区域内的重合亮斑的偏移量offset(x1,y1)＝待纠偏图像的该亮斑的坐标位置-参考图像上的相应亮斑的坐标位置，可表示为offset(x1,y1)＝curCyclePoints-basePoints；求取所有重合亮斑的平均偏移量，从而得到[0,0]到[1,1]范围内的细偏移量。在一个示例中，设定两个图像上距离不大于1.5个像素的两个亮斑为重合亮斑；3)综上，得到一个视野图像(fov)不同cycle的偏移量(x0,y0)-(x1,y1)，对于一个亮斑(peak)可表示为：curCyclePoints+(x0,y0)-(x1,y1)，curCyclePoints表示该亮斑原始坐标，即在纠偏前的图像中的坐标。上述图像纠偏获得的纠偏结果具有较高的准确性，且纠偏精度小于或等于0.1像素。图2示意纠偏过程及结果，图2中，基于图像A对图像C进行纠偏，图像A和图像C中的圆圈表示亮斑、相同数字标记的亮斑为重合亮斑，图像C->A表示纠偏结果，即图像C对齐至图像A的结果。
在某些具体实施方式中，请参阅图3，图像配准方法还包括S01识别亮斑，包括利用k1*k2矩阵对图像进行亮斑检测，判定矩阵的中心像素值不小于矩阵非中心任一像素值的矩阵对应一个候选亮斑，以及确定候选亮斑是否为亮斑，k1和k2均为大于1的奇数，k1*k2矩阵包含k1*k2个像素点。所称的图像选自待配准图像、构建参考图像的图像中的至少一个。利用该方式检测图像上的亮斑，能够快速有效地实现图像上的亮斑(spots或peaks)的检测，特别是对采集自核酸序列测定反应的图像。该方法对待检测图像即原始输入数据没有特别的限制，适用于任何利用光学检测原理进行核酸序列测定的平台所产生的图像的处理分析，包括但不限于二代和三代测序，具有高准确性和高效的特点，能从图像中获取更多的代表序列的信息。特别是对于随机图像及高准确度要求的信号识别，尤其具有优势。
在一些实施例中，图像来自核酸序列测定反应，核酸分子上带有光学可检测标记，利如荧光标记，荧光分子在特定波长激光照射下能够被激发发出荧光，通过成像系统采集图像。采集到的图像包括可能与荧光分子所在位置相对应的光斑/亮斑。可以理解地，当处于焦面位置时，所采集到的图像中的与荧光分子所在位置相对应的亮斑的尺寸较小且亮度较高；当位于非焦面位置时，所采集到的图像中的与荧光分子所在位置相对应的亮斑的尺寸较大且亮度较低。另外，视野中的可能存在其它非目标或者后续难以利用的物质/信息，比如杂质等；进一步地，在对单分子视野进行拍照中，大量分子聚集(簇)等也会干扰目标单分子信息采集。所称的单分子为一个少数几个分子，例如分子数目不大于10，例如为一个、两个、三个、四个、五个、六个、八个或者十个。
在一些示例中，矩阵的中心像素值大于第一预设值，矩阵非中心任一像素值大于第二预设值，第一预设值和第二预设值与图像的平均像素值相关。
在一些实施例中，可以利用k1*k2矩阵对图像进行遍历检测，所称的第一预设值和/或第二预设值的设置与该图像的平均像素值相关。对于灰度图像，像素值同灰度值。k1*k2矩阵，k1、k2可以相等也可以不相等。在一个示例中，成像系统相关参数为：物镜60倍，电子传感器的尺寸为6.5μm，经过显微镜成的像再经过电子传感器，能看到的最小尺寸为0.1μm，获得的图像或者输入的图像可为512*512、1024*1024或2048*2048的16位的灰度或彩色图像，k1和k2的取值范围均为大于1且小于10。在一个示例中，k1＝k2＝3；在另一个示例中，k1＝k2＝5。若图像是彩色图像，彩色图像的一个像素点具有三个像素值，可以将彩色图像转化为灰度图像，再进行亮斑检测，以降低图像检测过程的计算量和复杂度。可选择但不限于利用浮点算法、整数方法、移位方法或平均值法等将非灰度图像转换成灰度图像。
在一个示例中，发明人经过大量图像处理统计，取第一预设值为该图像的平均像素的1.4倍，取第二预设值为该图像的平均像素值的1.1倍，能够排除干扰、获得来自于光学检测标记的亮斑检测结果。
可利用大小、与理想亮斑的相似程度和/或强度来对候选亮斑进一步进行筛选判断。在某些具体实施方式中，利用候选亮斑对应的连通域的大小来定量反映比较图像上候选亮斑的大小，以此来筛选判断候选亮斑是否为要的亮斑。
在一个示例中，确定候选亮斑是否为亮斑包括：计算一个候选亮斑对应的连通域的大小Area＝A*B，判定对应的连通域的大小大于第三预设值的候选亮斑为一个亮斑，A表示以该候选亮斑对应的矩阵的中心的所在行的相连像素/连通像素的大小，B表示以该候选亮斑对应的矩阵的中心的所在列的相连像素/连通像素的大小，定义一个k1*k2矩阵中大于平均像素值的相连像素为一个所称的候选亮斑对应的连通域。如此，能够能够有效获得对应标记分子且符合后续序列识别的亮斑，获得核酸序列信息。
在一个例子中，以该图像的平均像素值作为基准，相邻的不小于平均像素值的两个或多个像素为所称的相连像素/连通像素(pixel connectivity)，如图4所示，加粗加大的表示候选亮斑对应的矩阵的中心，粗线框表示候选亮斑对应的3*3矩阵，标记为1的像素为不小于该图像的平均像素值的像素点，标记为0的像素为小于平均像素值的像素点，可看出A＝3，B＝6，该候选亮斑对应的连通域的大小为A*B＝3*6。
所称的第三预设值可依据该图像上所有候选亮斑对应的连通域的大小这一信息来确定。例如通过计算该图上各候选亮斑对应的连通域的大小，取亮斑的连通域大小的平均值代表该图像一个特性，作为第三预设值；又例如，可将该图像上各个候选亮斑对应的连通域大小按从小到大排序，取第50、第60、第70、第80或第90分位数连通域大小作为该第三预设值。如此，可有效获得亮斑信息，利于后续识别核酸序列。
在某些示例中，通过统计设置参数来定量反映比较候选亮斑的强度特征，以此来筛选候选亮斑。在一个示例中，确定候选亮斑是否为亮斑包括：计算一个候选亮斑的分值Score＝((k1*k2-1)CV-EV)/((CV+EV)/(k1*k2))，判定分值大于第四预设值的候选亮斑为一个亮斑，CV表示候选亮斑对应的矩阵的中心像素值，EV表示亮斑对应的矩阵的非中心像素值的总和。如此，能够能够有效获得对应标记分子且符合后续序列识别的亮斑，获得核酸序列信息。
所称的第四预设值可依据该图像上所有候选亮斑的分值的大小这一信息来确定。例如，当该图像上的候选亮斑的数量大于一定数目符合统计上对量的要求，例如该图像上候选亮斑的数目大于30， 可计算且将该图像的所有候选亮斑的Score值按升序排序，第四预设值可设置为第50、第60、第70、第80或90分位数Score值，如此，可排除掉小于第50、第60、第70、第80或第90分位数Score值的候选亮斑，利于有效获得目标亮斑，利于后续碱基序列准确识别。进行该处理或者说该筛选设置的依据是，一般地，认为中心与边缘强度/像素值差异大且汇聚的亮斑为与待检分子所在位置相对应的亮斑。一般情况下，图像上的候选亮斑的数量大于50、大于100或大于1000。
在某些示例中，结合形态和强度/亮度对候选亮斑进行筛选。在一个示例中，确定候选亮斑是否为亮斑包括：计算一个候选亮斑对应的连通域的大小Area＝A*B，以及计算一个候选亮斑的分值Score＝((k1*k2-1)CV-EV)/((CV+EV)/(k1*k2))，A表示以该候选亮斑对应的矩阵的中心的所在行的相连像素/连通像素的大小，B表示以该候选亮斑对应的矩阵的中心的所在列的相连像素/连通像素的大小，定义一个k1*k2矩阵中大于平均像素值的相连像素为一个所称的候选亮斑对应的连通域，CV表示候选亮斑对应的矩阵的中心像素值，EV表示亮斑对应的矩阵的非中心像素值的总和；判定对应的连通域的大小大于第三预设值且分值大于第四预设值的候选亮斑为一个亮斑。如此，能够有效地获得对应核酸分子且利于后续序列识别的亮斑信息。对于所称的第三预设值和/或第四预设值，可以参照前面具体实施方式进行考虑和设置。
在某些具体实施方式中，请参阅图5，图像配准方法还包括S03识别亮斑，待配准和/或参考图像来自发生碱基延伸反应的一个视野，发生碱基延伸反应的该视野上存在多个带有光学可检测标记的核酸分子，至少一部分核酸分子在图像上表现为亮斑，S03包括：预处理图像，获得预处理后的图像，所称的图像选自待配准图像和构建参考图像的图像中的至少一个；确定临界值以简化预处理后的图像，包括对小于临界值的预处理后的图像上的像素点的像素值赋值为第一预设值，对不小于临界值的预处理后的图像上的像素点的像素值赋值为第二预设值，以获得简化图像；基于预处理后的图像确定第一亮斑检测阈值c1；基于预处理后的图像和简化图像识别图像上的候选亮斑，包括判定满足以下a)-c)中至少两个条件的像素点矩阵为一个候选亮斑，a)在预处理后的图像中，像素点矩阵的中心像素点的像素值为最大，像素点矩阵可表示为r1*r2，r1和r2均为大于1的奇数，r1*r2像素点矩阵包含r1*r2个像素点，b)在简化图像中，像素点矩阵的中心像素点的像素值为第二预设值并且像素点矩阵的连通像素大于2/3*r1*r2，以及c)在预处理后的图像中的像素点矩阵的中心像素点的像素值大于第三预设值，并且满足g1*g2>c1，g1为以像素点矩阵的中心像素点为中心的m1*m2范围的二维高斯分布的相关系数，g2为该m1*m2范围的像素，m1和m2均为大于1的奇数，m1*m2范围包含m1*m2个像素点；以及确定候选亮斑是否为亮斑。利用该方式检测图像上的亮斑，包括利用发明人通过大量数据训练确定的判断条件或判断条件的组合，能够快速有效地实现图像上的亮斑的检测，特别是对采集自核酸序列测定反应的图像。该方法对待检测图像即原始输入数据没有特别的限制，适用于任何利用光学检测原理进行核酸序列测定的平台所产生的图像的处理分析，包括但不限于二代和三代测序，具有高准确性和高效的特点，能从图像中获取更多的代表序列的信息。特别是对于随机图像及高准确度要求的信号识别，尤其具有优势。
对于灰度图像，像素值同灰度值。若图像是彩色图像，彩色图像的一个像素点具有三个像素值，可以将彩色图像转化为灰度图像，再进行亮斑检测，以降低图像检测过程的计算量和复杂度。可选择但不限于利用浮点算法、整数方法、移位方法或平均值法等将非灰度图像转换成灰度图像。
在一些实施例中，预处理图像包括：利用开运算确定图像的背景；基于背景，利用顶帽运算将图像转化为第一图像；对第一图像进行高斯模糊处理，获得第二图像；对第二图像进行锐化，以获 得所称的预处理后的图像。如此，能对图像进行有效的降噪或者说提高图像的信噪比，利于亮斑的准确检测。
开运算是一种形态学处理，即先膨胀后腐蚀的过程，腐蚀操作会使得前景(感兴趣的部分)变小，而膨胀会使得前景变大；开运算可以用来消除小物体，在纤细点处分离物体，并且在平滑较大物体的边界的同时不明显改变其面积。该实施方式对图像做开运算的结构元p1*p2(用来处理图像的基本模板)的大小不作特别限制，p1和p2为奇数。在一个示例中，结构元p1*p2可以为15*15、31*31等，最终都能够获得利于后续处理分析的预处理后的图像。
顶帽运算往往用来分离比临近点(亮点/亮斑)亮一些的斑块，在一幅图像具有大幅的背景，而微小物品比较有规律的情况下，可以使用顶帽运算进行背景提取。在一个示例中，对图像进行顶帽变换包括先对图像做开运算，进而利用原图像减去开运算结果，获得第一图像即顶帽变换后的图像。顶帽变换的数学表达式为dst＝tophat(src,element)＝src-open(src,element)。发明人认为，开运算的结果放大了裂缝或者局部低亮度的区域，因此从原图中减去开运算后的图，得到的图像突出了比原图轮廓周围的区域更明亮的区域，这一操作与选择的核的大小相关，可以认为与亮点/亮斑的预期大小相关，若亮点不是预期大小，处理后的效果会使得整张图产生许多小凸起，具体可以参考虚焦图片，即亮点/亮斑晕染成一团。在一个示例中，亮点的预期大小即选择的核的大小为3*3，得到的顶帽变换后的图像利于后续进一步去噪处理。
高斯模糊(Gaussian Blur)也称为高斯滤波，是一种线性平滑滤波，适用于消除高斯噪声，广泛应用于图像处理的减噪过程。通俗的讲，高斯滤波就是对整幅图像进行加权平均的过程，每一个像素点的值，都由其本身和邻域内的其他像素值经过加权平均后得到。高斯滤波的具体操作是：用一个模板(或称卷积、掩模)扫描图像中的每一个像素，用模板确定的邻域内像素的加权平均灰度值去替代模板中心像素点的值。在一个示例中，对第一图像进行高斯模糊处理，在OpenCV中使用高斯滤波GaussianBlur函数进行，高斯分布参数Sigma取0.9，所使用的二维滤波器矩阵(卷积核)是3*3，从图像角度看经过该高斯模糊处理后，第一图像上的小突起被抹平，图像边缘光滑。进一步地，对第二图像即高斯过滤后的图像进行锐化，例如进行二维拉普拉斯锐化，从图像角度看经过处理后，边缘被锐化，高斯模糊后的图像得以恢复。
在一些实施例中，简化预处理后的图像包括：基于背景和预处理后的图像，确定临界值；比较预处理后的图像上的像素点的像素值与临界值，对小于临界值的预处理后的图像上的像素点的像素值赋值为第一预设值，对不小于临界值的预处理后的图像上的像素点的像素值赋值为第二预设值，获得简化图像。如此，根据发明人大量测试数据总结的确定临界值的方式以及确定的临界值，据此将预处理后的图像简化，例如二值化，利于后续亮斑准确检测，利于后续碱基准确识别、获得高质量数据等。
具体地，在一些示例中，获得简化图像包括：将预处理后获得的锐化后的结果除以开运算结果，获得和图像像素点对应的一组数值；通过该组数值，确定二值化预处理后的图像的临界值。例如，可将该组数值按大小升序排列，取该组数值中第20、30或40百分位数对应的数值作为二值化临界值/阈值。如此，获得的二值化图像利于后续亮斑的准确检测识别。
在一个示例中，图像预处理时的开运算的结构元为p1*p2，所称的将预处理后的图像(锐化后的结果)除以开运算结果，获得一组和结构元一样大小的数组/矩阵p1*p2，在每个数组中，将该数组包含的p1*p2个数值按大小升序排列，取该数组中第三十百分位数对应的数值作为该区域(数值 矩阵)的二值化临界值/阈值，如此，分别确定阈值对图像上的各个区域进行二值化，最终获得的二值化结果在去噪的同时更加突出所需信息，利于后续亮斑的准确检测。
在一些示例中，利用大津法进行第一亮斑检测阈值的确定。大津法(OTSU算法)也可称为最大类间方差法，大津法利用类间方差最大来分割图像，意味着错分概率小，准确性高。假设预处理后的图像的前景和背景的分割阈值为T(c1)，属于前景的像素点数占整幅图像的比例为w0，其平均灰度为μ0；属于背景的像素点数占整幅图像的比例为w1，其平均灰度为μ1。待处理图像的总平均灰度记为μ，类间方差记为var，则有：μ＝ω 0*μ 0+ω 1*μ 1；var＝ω 0(μ 0-μ) 2+ω 1(μ 1-μ) 2，将后者代入前者，得到等价公式：var＝ω 0ω 1(μ 1-μ 0) 2。采用遍历的方法得到使类间方差最大的分割阈值T，即为所求的第一亮斑检测阈值c1。
在一些实施例中，基于预处理后的图像和简化图像识别图像上的候选亮斑，包括判断同时满足a)-c)三个条件的像素点矩阵为一个候选亮斑。如此，能有效地提高后续基于亮斑信息确定核酸序列的准确性和下机数据的质量。
具体地，在一个示例中，候选亮斑的判定需要满足的条件包括a)，k1、k2可以相等也可以不相等。在一个示例中，成像系统相关参数为：物镜60倍，电子传感器的尺寸为6.5μm，经过显微镜成的像再经过电子传感器，能看到的最小尺寸为0.1μm，获得的图像或者输入的图像可为512*512、1024*1024或2048*2048的16位的灰度或彩色图像，k1和k2的取值范围均为大于1且小于10。在一个示例中，在一个预处理后的图像中，依据亮斑的预期大小设置k1＝k2＝3；在另一个示例中，设置k1＝k2＝5。
在一个示例中，候选亮斑的判定需要满足的条件包括b)，在简化图像中，像素点矩阵的中心像素点的像素值为第二预设值，并且该像素点矩阵的连通像素大于2/3*k1*k2，即中心像素点的像素值大于临界值且连通像素大于矩阵的三分之二。这里，相邻的像素值都为第二预设值的两个或多个像素为所称的相连像素/连通像素(pixel connectivity)，例如，简化图像为二值化图像，第一预设值为0，第二预设值为1，如图4所示，加粗加大的表示所称的像素点矩阵的中心，粗线框表示像素点矩阵3*3，即k1＝k2＝3，该矩阵的中心像素点的像素值为1，连通像素为4，小于2/3*k1*k2＝6，该像素点矩阵不满足条件b)，非候选亮斑。
在一个示例中，候选亮斑的判定需要满足的条件包括c)，在预处理图像中，g2为修正后的m1*m2范围的像素，即为修正后的m1*m2范围像素总和。在一个例子中，依据简化图像相应m1*m2范围中像素值为第二预设值的像素点所占的比例进行修正，例如，如图6所示，m1＝m2＝5，所称的简化图像相应m1*m2范围中像素值为第二预设值的像素点所占的比例为13/25(13个“1”)，修正后的g2为原来的13/25。如此，利于更准确的检测识别亮斑，利于后续亮斑信息的分析读取。
在一些示例中，所称的判定候选亮斑是否为亮斑还包括：基于预处理后的图像确定第二亮斑检测阈值，以及判定像素值不小于第二亮斑检测阈值的候选亮斑为亮斑。在具体示例中，以候选亮斑的坐标所在的像素点的像素值作为该候选亮斑的像素值。通过利用基于预处理后的图像确定的第二 亮斑检测阈值对候选亮斑的进一步筛选，能够排除掉至少一部分更可能是图像背景但亮度(强度)和/或形状表现为“亮斑”的亮斑，利于后续基于亮斑的序列的准确识别，提高下机数据的质量。
在一个示例中，可利用重心法获取候选亮斑的坐标，包括亚像素级坐标。利用双线性插值法计算候选亮斑的坐标位置的灰度值。
在某些具体示例中，判定候选亮斑是否为亮斑包括：将预处理后的图像划分为预定大小的一组区域(block)，对该区域中的像素点的像素值进行排序，以确定该区域对应的第二亮斑检测阈值；对于位于区域的候选亮斑，判定像素值不小于该区域对应的第二亮斑检测阈值的候选亮斑为亮斑。如此，区分图像的不同区域的差异比如光强的整体落差，分开进行亮斑的进一步检测识别，利于准确识别亮斑并且获得更多的亮斑。
所称的将预处理后的图像划分为预定大小的一组区域(block)，block之间可以有重叠也可以没有重叠。在一个示例中，block之间没有重叠。在一些实施例中，预处理后的图像的大小不小于512*512，例如为512*512、1024*1024、1800*1800或者2056*2056等，所称预定大小的区域可以设为为200*200。如此，利于快速计算判断识别亮斑。
在一些实施例中，确定该区域对应的第二亮斑检测阈值时，对每个block中的像素点的像素值按大小进行升序排列，取p10+(p10-p1)*4.1作为该block对应的第二亮斑检测阈值，即该block的背景，p1表示第百分之一分位的像素值，p10表示第百分之十分位的像素值。该阈值是发明人通过大量数据训练测试得出的较为稳定的阈值，能够消除大量背景上的亮斑。可以理解地，当光学系统调整，图像整体像素分布发生改变时，此阈值可能需要适当调整。图7为进行该处理之前和之后的亮斑检测结果对比示意图，即排除掉区域背景前后的亮斑检测结果示意图，图7的上半部分为作该处理后的亮斑检测结果、下半部分为不作该处理的亮斑检测结果，十字标记的为候选亮斑或亮斑。
上述在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的序列表，可以具体实现在任何计算机可读存储介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，"计算机可读存储介质"可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。计算机可读存储介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读存储介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
请参阅图8，本发明实施方式提供一种图像配准装置100，该装置用以实施上述任一实施例中的图像配准方法的全部或部分步骤，该装置100包括：第一配准模块110，用于基于参考图像对待配准图像进行第一配准，包括确定待配准图像上的预定区域和参考图像上的相应预定区域的第一偏移量，基于第一偏移量移动待配准图像上的所有亮斑，获得第一配准后的待配准图像，参考图像和待配准图像对应相同对象，参考图像和待配准图像均包含多个亮斑；第二配准模块120，用于基于参考图像对来自第一配准模块的第一配准后的待配准图像进行第二配准，包括合并第一配准后的待配 准图像和参考图像，获得合并图像，计算合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个重合亮斑，基于该第二偏移量移动第一配准后的待配准图像上的所有亮斑，以实现对待配准图像的配准。
上述对本发明任一实施例中的图像配准方法的技术特征和优点的描述，也适用于该图像配准装置，在此不再赘述。可以理解地，上述任一实施例中的图像配准方法的附加技术特征，包括子步骤、附加步骤、可选择可替代或较佳的设置或处理等，可通过使得该装置或者该装置的模块进一步包括单元/模块或者子单元/子模块得以实施。
例如，在一些示例中，图像配准装置100还包括参考图像构建模块109，参考图像构建模块用于构建参考图像，包括：获取第一图像和第二图像，第一图像和第二图像与待配准图像对应相同对象；基于第一图像对第二图像进行粗配准，包括确定第二图像和第一图像的偏移量，基于该偏移量移动第二图像，获得粗配准后的第二图像；合并第一图像和粗配准后的第二图像，以获得参考图像，第一图像和第二图像均包含多个亮斑。
在一些示例中，在参考图像构建模块109中构建参考图像还包括利用第三图像和第四图像，待配准图像、第一图像、第二图像、第三图像和第四图像来自测序反应的相同视野，第一图像、第二图像、第三图像和第四图像分别对应A/U、T、G和C四种类型碱基延伸反应时的视野，碱基延伸反应时的该视野存在多个带有光学可检测标记的核酸分子，至少一部分核酸分子在图像上表现为亮斑，构建参考图像还包括：基于第一图像对第三图像进行粗配准，包括确定第三图像和第一图像的偏移量，基于该偏移量移动第三图像，获得粗配准后的第三图像；基于第一图像对第四图像进行粗配准，包括确定第四图像和第一图像的偏移量，基于该偏移量移动第四图像，获得粗配准后的第四图像；合并第一图像和粗配准后的第二图像、粗配准后的第三图像以及粗配准后的第四图像，以获得参考图像。
在一些示例中，参考图像和待配准图像为二值化图像。
在一些示例中，利用二维离散傅里叶变换确定第一偏移量、第二图像和第一图像的偏移量、第三图像和第一图像的偏移量和/或述第四图像和第一图像的偏移量。
在一些示例中，图像配准装置100还包括第一亮斑检测模块107，第一亮斑检测模块用于利用k1*k2矩阵对图像进行亮斑检测，判定矩阵的中心像素值不小于矩阵非中心任一像素值的矩阵对应一个候选亮斑，以及确定候选亮斑是否为亮斑，k1和k2均为大于1的奇数，k1*k2矩阵包含k1*k2个像素点。
在一些示例中，所称的矩阵的中心像素值大于第一预设值，矩阵非中心任一像素值大于第二预设值，第一预设值和/或第二预设值与图像的平均像素值相关。
在一些示例中，在第一亮斑检测模块107中，确定候选亮斑是否为亮斑包括：计算一个候选亮斑对应的连通域的大小Area＝A*B，判定对应的连通域的大小大于第三预设值的候选亮斑为一个亮斑，A表示以候选亮斑对应的矩阵的中心的所在行的连通大小，B表示以候选亮斑对应的矩阵的中心的所在列的连通大小，定义大于平均像素值的相连像素点为一个连通域，和/或计算一个候选亮斑的分值Score＝((k1*k2-1)CV-EV)/((CV+EV)/(k1*k2))，判定分值大于第四预设值的候选亮斑为一个亮斑，CV表示候选亮斑对应的矩阵的中心像素值，EV表示亮斑对应的矩阵的非中心像素值的总和。
在一些示例中，图像配准装置100还包括第二亮斑检测模块105，待配准图像或者构成参考图像的任何图像都采集自发生碱基延伸反应的一个视野，发生碱基延伸反应的该视野上存在多个带有 光学可检测标记的核酸分子，至少一部分核酸分子在图像上表现为亮斑，第二亮斑检测模块105用于：预处理图像，获得预处理后的图像；确定临界值以简化预处理后的图像，包括对小于临界值的预处理后的图像上的像素点的像素值赋值为第一预设值，对不小于临界值的预处理后的图像上的像素点的像素值赋值为第二预设值，以获得简化图像；基于预处理后的图像确定第一亮斑检测阈值c1；基于预处理后的图像和简化图像识别图像上的候选亮斑，包括判定满足以下a)-c)中至少两个条件的像素点矩阵为一个候选亮斑，a)在预处理后的图像中，像素点矩阵的中心像素点的像素值为最大，像素点矩阵可表示为r1*r2，r1和r2均为大于1的奇数，r1*r2像素点矩阵包含r1*r2个像素点，b)在简化图像中，像素点矩阵的中心像素点的像素值为第二预设值并且像素点矩阵的连通像素大于2/3*r1*r2，以及c)在预处理后的图像中的像素点矩阵的中心像素点的像素值大于第三预设值，并且满足g1*g2>c1，g1为以像素点矩阵的中心像素点为中心的m1*m2范围的二维高斯分布的相关系数，g2为该m1*m2范围的像素，m1和m2均为大于1的奇数，m1*m2范围包含m1*m2个像素点；以及确定候选亮斑是否为亮斑。
在一些示例中，在第二亮斑检测模块105中，确定候选亮斑是否为亮斑包括：基于预处理后的图像确定第二亮斑检测阈值，以及判定像素值不小于第二亮斑检测阈值的候选亮斑为亮斑。
在一些示例中，候选亮斑的像素值为该候选亮斑的坐标所在的像素点的像素值。
在一些示例中，在第二亮斑检测模块105中，基于预处理后的图像确定第二亮斑检测阈值，判定像素值不小于第二亮斑检测阈值的候选亮斑为亮斑，包括：将预处理后的图像划分为预定大小的一组区域，对该区域中的像素点的像素值进行排序，以确定该区域对应的第二亮斑检测阈值，对于位于区域的候选亮斑，判定像素值不小于该区域对应的第二亮斑检测阈值的候选亮斑为亮斑。
在一些示例中，在第二亮斑检测模块105中，预处理图像包括：利用开运算确定图像的背景，基于背景，利用顶帽运算将图像转化为第一图像，对第一图像进行高斯模糊处理，获得第二图像，对第二图像进行锐化，获得预处理后的图像。
在一些示例中，在第二亮斑检测模块105中，确定临界值以简化预处理后的图像，获得简化图像包括：基于背景和预处理后的图像，确定临界值，比较预处理后的图像上的像素点的像素值与临界值，以获得简化图像。
在一些示例中，在第二亮斑检测模块105中，g2为修正后的m1*m2范围的像素，依据简化图像相应m1*m2范围中像素值为第二预设值的像素点所占的比例进行修正。
在一些示例中，在第一亮斑检测模块107和/或第二亮斑检测模块105中，确定候选亮斑是否为亮斑还包括：若判定候选亮斑为亮斑，计算亮斑的亚像素中心坐标和/或亚像素中心坐标的强度值，若判定候选亮斑非为亮斑，丢弃候选亮斑。
本发明实施方式还提供一种计算机程序产品，该产品包括指令，所称的指令在计算机执行该程序时，使计算机执行上述任一实施例中图像配准方法的全部或部分步骤。
本领域技术人员知晓，除了以纯计算机可读程序代码方式实现控制器/处理器外，完全可以通过将方法步骤进行逻辑变成来使得控制器以逻辑门、开关、专用集成电路、可编辑逻辑控制器和嵌入微控制器等的形式来实现相同的功能。因此，这种控制器/处理器可以被认为是一种硬件部件，而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构。或者甚至，可以将用于实现各种功能的装置视为既可以是实现方法的的软件模块又可以是硬件部件内的结构。
在本说明书的描述中，一个实施方式、一些实施方式、一个或一些具体实施方式、一个或一些 实施例、示例等的描述意指结合该实施方式或示例描述的具体特征、结构或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构等特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
尽管已经示出和描述了本发明的实施例，本领域的普通技术人员可以理解：在不脱离本发明的原理和宗旨的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由权利要求及其等同限定。

Claims (33)

  1. 一种图像配准方法，其特征在于，包括：
    基于参考图像对待配准图像进行第一配准，所述参考图像和所述待配准图像对应相同对象，所述参考图像和所述待配准图像均包含多个亮斑，包括，
    确定所述待配准图像上的预定区域和所述参考图像上的相应预定区域的第一偏移量，基于所述第一偏移量移动所述待配准图像上的所有亮斑，获得第一配准后的待配准图像；
    基于所述参考图像对第一配准后的待配准图像进行第二配准，包括，
    合并所述第一配准后的待配准图像和所述参考图像，获得合并图像，
    计算所述合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个所述重合亮斑，
    基于该第二偏移量移动所述第一配准后的待配准图像上的所有亮斑，以实现对所述待配准图像的配准。
  2. 权利要求1的方法，其特征在于，所述参考图像通过构建获得，构建所述参考图像包括：
    获取第一图像和第二图像，所述第一图像和所述第二图像与所述待配准图像对应相同对象；
    基于第一图像对第二图像进行粗配准，包括确定所述第二图像和所述第一图像的偏移量，基于该偏移量移动所述第二图像，获得粗配准后的第二图像；
    合并所述第一图像和粗配准后的第二图像，以获得所述参考图像，所述第一图像和所述第二图像均包含多个亮斑。
  3. 权利要求2的方法，其特征在于，构建所述参考图像还包括利用第三图像和第四图像，所述待配准图像、第一图像、第二图像、第三图像和第四图像来自测序反应的相同视野，所述第一图像、第二图像、第三图像和第四图像分别对应A/U、T、G和C四种类型碱基延伸反应时的所述视野，碱基延伸反应时的该视野存在多个带有光学可检测标记的核酸分子，至少一部分所述核酸分子在所述图像上表现为所述亮斑，构建所述参考图像还包括：
    基于第一图像对第三图像进行粗配准，包括确定所述第三图像和所述第一图像的偏移量，基于该偏移量移动所述第三图像，获得粗配准后的第三图像；
    基于第一图像对第四图像进行粗配准，包括确定所述第四图像和所述第一图像的偏移量，基于该偏移量移动所述第四图像，获得粗配准后的第四图像；
    合并所述第一图像和粗配准后的第二图像、粗配准后的第三图像以及粗配准后的第四图像，以获得所述参考图像。
  4. 权利要求1的方法，其特征在于，所述参考图像和所述待配准图像为二值化图像。
  5. 权利要求1-4任一方法，其特征在于，利用二维离散傅里叶变换确定所述第一偏移量、所述第二图像和所述第一图像的偏移量、所述第三图像和所述第一图像的偏移量和/或述第四图像和所述第一图像的偏移量。
  6. 权利要求1-5任一方法，其特征在于，还包括识别所述亮斑，包括利用k1*k2矩阵对图像进行检测，判定所述矩阵的中心像素值不小于所述矩阵非中心任一像素值的矩阵对应一个候选亮斑，以及确定所述候选亮斑是否为所述亮斑，k1和k2均为大于1的奇数，k1*k2矩阵包含k1*k2个像素点。
  7. 权利要求6的方法，其特征在于，所述矩阵的中心像素值大于第一预设值，所述矩阵非中心任一像素值大于第二预设值，所述第一预设值和所述第二预设值与所述图像的平均像素值相关。
  8. 权利要求6或7的方法，其特征在于，确定所述候选亮斑是否为所述亮斑包括：
    计算一个所述候选亮斑对应的连通域的大小Area＝A*B，判定对应的连通域的大小大于第三预设值的候选亮斑为一个所述亮斑，A表示以所述候选亮斑对应的矩阵的中心的所在行的连通大小，B表示以所述候选亮斑对应的矩阵的中心的所在列的连通大小，定义大于平均像素值的相连像素点为一个连通域，和/或
    计算一个所述候选亮斑的分值Score＝((k1*k2-1)CV-EV)/((CV+EV)/(k1*k2))，判定分值大于第四预设值的所述候选亮斑为一个所述亮斑，CV表示所述候选亮斑对应的矩阵的中心像素值，EV表示所述亮斑对应的所述矩阵的非中心像素值的总和。
  9. 权利要求1-5任一方法，其特征在于，还包括识别所述亮斑，所述图像采集自发生碱基延伸反应的一个视野，发生碱基延伸反应的该视野上存在多个带有光学可检测标记的核酸分子，至少一部分所述核酸分子在所述图像上表现为亮斑，所述方法包括：
    预处理所述图像，获得预处理后的图像；
    确定临界值以简化所述预处理后的图像，包括对小于所述临界值的所述预处理后的图像上的像素点的像素值赋值为第一预设值，对不小于所述临界值的所述预处理后的图像上的像素点的像素值赋值为第二预设值，以获得简化图像；
    基于所述预处理后的图像确定第一亮斑检测阈值c1；
    基于所述预处理后的图像和所述简化图像识别所述图像上的候选亮斑，包括判定满足以下a)-c)中至少两个条件的像素点矩阵为一个所述候选亮斑，
    a)在所述预处理后的图像中，像素点矩阵的中心像素点的像素值为最大，所述像素点矩阵可表示为r1*r2，r1和r2均为大于1的奇数，r1*r2像素点矩阵包含r1*r2个像素点，
    b)在所述简化图像中，所述像素点矩阵的中心像素点的像素值为第二预设值并且所述像素点矩阵的连通像素大于2/3*r1*r2，以及
    c)在所述预处理后的图像中的所述像素点矩阵的中心像素点的像素值大于第三预设值，并且满足g1*g2>c1，g1为以所述像素点矩阵的中心像素点为中心的m1*m2范围的二维高斯分布的相关系数，g2为该m1*m2范围的像素，m1和m2均为大于1的奇数，m1*m2范围包含m1*m2个像素点；以及
    确定所述候选亮斑是否为所述亮斑。
  10. 权利要求9的方法，其特征在于，所述确定候选亮斑是否为亮斑包括：
    基于所述预处理后的图像确定第二亮斑检测阈值，以及
    判定像素值不小于所述第二亮斑检测阈值的候选亮斑为所述亮斑。
  11. 权利要求10的方法，其特征在于，所述候选亮斑的像素值为该候选亮斑的坐标所在的像素点的像素值。
  12. 权利要求10或11的方法，其特征在于，所述基于预处理后的图像确定第二亮斑检测阈值，判定像素值不小于所述第二亮斑检测阈值的候选亮斑为所述亮斑，包括：
    将所述预处理后的图像划分为预定大小的一组区域，
    对该区域中的像素点的像素值进行排序，以确定该区域对应的第二亮斑检测阈值，
    对于位于所述区域的候选亮斑，判定像素值不小于该区域对应的第二亮斑检测阈值的候选亮斑为所述亮斑。
  13. 权利要求9-12任一方法，其特征在于，预处理所述图像，包括：
    利用开运算确定所述图像的背景，
    基于所述背景，利用顶帽运算将所述图像转化为第一图像，
    对所述第一图像进行高斯模糊处理，获得第二图像，
    对所述第二图像进行锐化，获得所述预处理后的图像。
  14. 权利要求13的方法，其特征在于，所述确定临界值以简化所述预处理后的图像，获得简化图像，包括：
    基于所述背景和所述预处理后的图像，确定所述临界值，
    比较所述预处理后的图像上的像素点的像素值与所述临界值，以获得所述简化图像。
  15. 权利要求9-14任一方法，其特征在于，g2为修正后的m1*m2范围的像素，依据所述简化图像相应m1*m2范围中像素值为第二预设值的像素点所占的比例进行所述修正。
  16. 权利要求6-15任一方法，其特征在于，确定所述候选亮斑是否为所述亮斑还包括：
    若判定所述候选亮斑为所述亮斑，计算所述亮斑的亚像素中心坐标和/或所述亚像素中心坐标的强度值，
    若判定所述候选亮斑非为所述亮斑，丢弃所述候选亮斑。
  17. 一种图像配准装置，其特征在于，包括：
    第一配准模块，用于基于参考图像对待配准图像进行第一配准，包括，
    确定所述待配准图像上的预定区域和所述参考图像上的相应预定区域的第一偏移量，基于所述第一偏移量移动所述待配准图像上的所有亮斑，获得第一配准后的待配准图像，所述参考图像和所述待配准图像对应相同对象，所述参考图像和所述待配准图像均包含多个亮斑；
    第二配准模块，用于基于所述参考图像对来自所述第一配准模块的第一配准后的待配准图像进行第二配准，包括，
    合并所述第一配准后的待配准图像和所述参考图像，获得合并图像，
    计算所述合并图像上的预定区域的所有重合亮斑的偏移量，以确定第二偏移量，距离小于预定像素的两个或多个亮斑为一个所述重合亮斑，
    基于该第二偏移量移动所述第一配准后的待配准图像上的所有亮斑，以实现对所述待配准图像的配准。
  18. 权利要求17的装置，其特征在于，还包括参考图像构建模块，所述参考图像构建模块用于构建所述参考图像，包括：
    获取第一图像和第二图像，所述第一图像和所述第二图像与所述待配准图像对应相同对象；
    基于第一图像对第二图像进行粗配准，包括确定所述第二图像和所述第一图像的偏移量，基于该偏移量移动所述第二图像，获得粗配准后的第二图像；
    合并所述第一图像和粗配准后的第二图像，以获得所述参考图像，所述第一图像和所述第二图像均包含多个亮斑。
  19. 权利要求18的装置，其特征在于，在所述参考图像构建模块中构建所述参考图像还包括利用第三图像和第四图像，所述待配准图像、第一图像、第二图像、第三图像和第四图像来自测序反 应的相同视野，所述第一图像、第二图像、第三图像和第四图像分别对应A/U、T、G和C四种类型碱基延伸反应时的所述视野，碱基延伸反应时的该视野存在多个带有光学可检测标记的核酸分子，至少一部分所述核酸分子在所述图像上表现为所述亮斑，构建所述参考图像还包括：
    基于第一图像对第三图像进行粗配准，包括确定所述第三图像和所述第一图像的偏移量，基于该偏移量移动所述第三图像，获得粗配准后的第三图像；
    基于第一图像对第四图像进行粗配准，包括确定所述第四图像和所述第一图像的偏移量，基于该偏移量移动所述第四图像，获得粗配准后的第四图像；
    合并所述第一图像和粗配准后的第二图像、粗配准后的第三图像以及粗配准后的第四图像，以获得所述参考图像。
  20. 权利要求17的装置，其特征在于，所述参考图像和所述待配准图像为二值化图像。
  21. 权利要求17-20任一装置，其特征在于，利用二维离散傅里叶变换确定所述第一偏移量、所述第二图像和所述第一图像的偏移量、所述第三图像和所述第一图像的偏移量和/或述第四图像和所述第一图像的偏移量。
  22. 权利要求17-21任一装置，其特征在于，还包括第一亮斑检测模块，所述第一亮斑检测模块用于利用k1*k2矩阵对图像进行亮斑检测，判定所述矩阵的中心像素值不小于所述矩阵非中心任一像素值的矩阵对应一个候选亮斑，以及确定所述候选亮斑是否为所述亮斑，k1和k2均为大于1的奇数，k1*k2矩阵包含k1*k2个像素点。
  23. 权利要求22的装置，其特征在于，所述矩阵的中心像素值大于第一预设值，所述矩阵非中心任一像素值大于第二预设值，所述第一预设值和/或所述第二预设值与所述图像的平均像素值相关。
  24. 权利要求22或23的装置，其特征在于，在所述第一亮斑检测模块中，确定所述候选亮斑是否为所述亮斑包括：
    计算一个所述候选亮斑对应的连通域的大小Area＝A*B，判定对应的连通域的大小大于第三预设值的候选亮斑为一个所述亮斑，A表示以所述候选亮斑对应的矩阵的中心的所在行的连通大小，B表示以所述候选亮斑对应的矩阵的中心的所在列的连通大小，定义大于平均像素值的相连像素点为一个连通域，和/或
    计算一个所述候选亮斑的分值Score＝((k1*k2-1)CV-EV)/((CV+EV)/(k1*k2))，判定分值大于第四预设值的所述候选亮斑为一个所述亮斑，CV表示所述候选亮斑对应的矩阵的中心像素值，EV表示所述亮斑对应的所述矩阵的非中心像素值的总和。
  25. 权利要求17-21任一装置，其特征在于，还包括第二亮斑检测模块，所述图像采集自发生碱基延伸反应的一个视野，发生碱基延伸反应的该视野上存在多个带有光学可检测标记的核酸分子，至少一部分所述核酸分子在所述图像上表现为亮斑，所述第二亮斑检测模块用于：
    预处理所述图像，获得预处理后的图像；
    确定临界值以简化所述预处理后的图像，包括对小于所述临界值的所述预处理后的图像上的像素点的像素值赋值为第一预设值，对不小于所述临界值的所述预处理后的图像上的像素点的像素值赋值为第二预设值，以获得简化图像；
    基于所述预处理后的图像确定第一亮斑检测阈值c1；
    基于所述预处理后的图像和所述简化图像识别所述图像上的候选亮斑，包括判定满足以下a)-c)中至少两个条件的像素点矩阵为一个所述候选亮斑，
    a)在所述预处理后的图像中，像素点矩阵的中心像素点的像素值为最大，所述像素点矩阵可表示为r1*r2，r1和r2均为大于1的奇数，r1*r2像素点矩阵包含r1*r2个像素点，
    b)在所述简化图像中，所述像素点矩阵的中心像素点的像素值为第二预设值并且所述像素点矩阵的连通像素大于2/3*r1*r2，以及
    c)在所述预处理后的图像中的所述像素点矩阵的中心像素点的像素值大于第三预设值，并且满足g1*g2>c1，g1为以所述像素点矩阵的中心像素点为中心的m1*m2范围的二维高斯分布的相关系数，g2为该m1*m2范围的像素，m1和m2均为大于1的奇数，m1*m2范围包含m1*m2个像素点；以及
    确定所述候选亮斑是否为所述亮斑。
  26. 权利要求25的装置，其特征在于，在所述第二亮斑检测模块中，所述确定候选亮斑是否为亮斑包括：
    基于所述预处理后的图像确定第二亮斑检测阈值，以及
    判定像素值不小于所述第二亮斑检测阈值的候选亮斑为所述亮斑。
  27. 权利要求26的装置，其特征在于，所述候选亮斑的像素值为该候选亮斑的坐标所在的像素点的像素值。
  28. 权利要求26或27的装置，其特征在于，在所述第二亮斑检测模块中，所述基于预处理后的图像确定第二亮斑检测阈值，判定像素值不小于所述第二亮斑检测阈值的候选亮斑为所述亮斑，包括：
    将所述预处理后的图像划分为预定大小的一组区域，
    对该区域中的像素点的像素值进行排序，以确定该区域对应的第二亮斑检测阈值，
    对于位于所述区域的候选亮斑，判定像素值不小于该区域对应的第二亮斑检测阈值的候选亮斑为所述亮斑。
  29. 权利要求25-28任一装置，其特征在于，在所述第二亮斑检测模块中，预处理所述图像包括：
    利用开运算确定所述图像的背景，
    基于所述背景，利用顶帽运算将所述图像转化为第一图像，
    对所述第一图像进行高斯模糊处理，获得第二图像，
    对所述第二图像进行锐化，获得所述预处理后的图像。
  30. 权利要求29的装置，其特征在于，在所述第二亮斑检测模块中，所述确定临界值以简化所述预处理后的图像，获得简化图像包括：
    基于所述背景和所述预处理后的图像，确定所述临界值，
    比较所述预处理后的图像上的像素点的像素值与所述临界值，以获得所述简化图像。
  31. 权利要求25-30任一装置，其特征在于，在所述第二亮斑检测模块中，g2为修正后的m1*m2范围的像素，依据所述简化图像相应m1*m2范围中像素值为第二预设值的像素点所占的比例进行所述修正。
  32. 权利要求22-30任一装置，其特征在于，在所述第一亮斑检测模块和/或所述第二亮斑检测模块中，确定所述候选亮斑是否为所述亮斑还包括：
    若判定所述候选亮斑为所述亮斑，计算所述亮斑的亚像素中心坐标和/或所述亚像素中心坐标的强度值，
    若判定所述候选亮斑非为所述亮斑，丢弃所述候选亮斑。
  33. 一种计算机程序产品，包括指令，所述指令在所述计算机执行所述程序时，使所述计算机执行如权利要求1-16任一项的方法的全部或部分步骤。
PCT/CN2018/101815 2018-08-22 2018-08-22 图像配准方法、装置和计算机程序产品 WO2020037570A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/270,423 US12008775B2 (en) 2018-08-22 2018-08-22 Method and device for image registration, and computer program product
PCT/CN2018/101815 WO2020037570A1 (zh) 2018-08-22 2018-08-22 图像配准方法、装置和计算机程序产品
EP18930564.2A EP3843032A4 (en) 2018-08-22 2018-08-22 METHOD AND DEVICE FOR IMAGE REGISTRATION AND COMPUTER PROGRAM PRODUCT

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/101815 WO2020037570A1 (zh) 2018-08-22 2018-08-22 图像配准方法、装置和计算机程序产品

Publications (1)

Publication Number Publication Date
WO2020037570A1 true WO2020037570A1 (zh) 2020-02-27

Family

ID=69592158

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/101815 WO2020037570A1 (zh) 2018-08-22 2018-08-22 图像配准方法、装置和计算机程序产品

Country Status (3)

Country Link
US (1) US12008775B2 (zh)
EP (1) EP3843032A4 (zh)
WO (1) WO2020037570A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115516075A (zh) * 2020-05-12 2022-12-23 株式会社日立高新技术 核酸分析装置、核酸分析方法以及机器学习方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12008775B2 (en) * 2018-08-22 2024-06-11 Genemind Biosciences Company Limited Method and device for image registration, and computer program product
CN118279141B (zh) * 2024-06-03 2024-08-30 浙江大华技术股份有限公司 图像拼接方法、电子设备以及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101950419A (zh) * 2010-08-26 2011-01-19 西安理工大学 同时存在平移和旋转情况下的快速图像配准方法
CN102663720A (zh) * 2012-03-31 2012-09-12 哈尔滨工业大学 一种基于最小均方误差准则的图像拼接方法
CN104318568A (zh) * 2014-10-24 2015-01-28 武汉华目信息技术有限责任公司 一种图像配准的方法和系统
CN105303533A (zh) * 2015-11-03 2016-02-03 华中科技大学 一种超声图像滤波方法
CN107918931A (zh) * 2016-10-10 2018-04-17 深圳市瀚海基因生物科技有限公司 图像处理方法及系统

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6301377B1 (en) * 1999-10-05 2001-10-09 Large Scale Proteomics Corporation Gel electrophoresis image warping
AU2003298655A1 (en) * 2002-11-15 2004-06-15 Bioarray Solutions, Ltd. Analysis, secure access to, and transmission of array images
JP2007315772A (ja) 2006-05-23 2007-12-06 Canon Inc 蛍光検出装置および生化学反応分析装置
US8131476B2 (en) * 2006-08-07 2012-03-06 General Electric Company System and method for co-registering multi-channel images of a tissue micro array
CN101206116B (zh) 2007-12-07 2010-08-18 北京机械工业学院 目标点全局自动定位方法
US8880351B2 (en) 2008-03-25 2014-11-04 General Electric Company Method and apparatus for analysis of tissue microarrays
JP5499732B2 (ja) 2009-06-23 2014-05-21 ソニー株式会社 生体サンプル像取得装置、生体サンプル像取得方法及び生体サンプル像取得プログラム
US8965076B2 (en) * 2010-01-13 2015-02-24 Illumina, Inc. Data processing system and methods
US8774494B2 (en) 2010-04-30 2014-07-08 Complete Genomics, Inc. Method and system for accurate alignment and registration of array for DNA sequencing
CN102174384B (zh) 2011-01-05 2014-04-02 深圳华因康基因科技有限公司 对基因测序仪的测序及信号处理进行控制的方法及系统
JP5413408B2 (ja) 2011-06-09 2014-02-12 富士ゼロックス株式会社 画像処理装置、プログラム及び画像処理システム
CN102354398A (zh) 2011-09-22 2012-02-15 苏州大学 基于密度中心与自适应的基因芯片处理方法
WO2014034275A1 (ja) * 2012-08-30 2014-03-06 株式会社 日立ハイテクノロジーズ 核酸分析装置
KR101348680B1 (ko) 2013-01-09 2014-01-09 국방과학연구소 영상추적기를 위한 표적포착방법 및 이를 이용한 표적포착장치
US20140349281A1 (en) 2013-05-22 2014-11-27 Sunpower Technologies Llc System and Method for Dispensing Barcoded Solutions
JP6573894B2 (ja) * 2014-02-04 2019-09-11 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． 少なくとも二画像をレジストレーション及び視覚化するための方法
US20150317433A1 (en) 2014-04-30 2015-11-05 Complete Genomics, Inc. Using doublet information in genome mapping and assembly
CN104297249A (zh) 2014-09-15 2015-01-21 浙江大学 基于心肌细胞传感器的药物心脏毒性检测分析方法
CN104376537B (zh) 2014-11-25 2018-01-30 中国兵器工业集团第二一四研究所苏州研发中心 一种去除emccd图像亮点的方法
CN107111874B (zh) 2014-12-30 2022-04-08 文塔纳医疗系统公司 用于共表达分析的系统和方法
CN105039147B (zh) 2015-06-03 2016-05-04 西安交通大学 一种高通量基因测序碱基荧光图像捕获系统装置及方法
CN105205788B (zh) 2015-07-22 2018-06-01 哈尔滨工业大学深圳研究生院 一种针对高通量基因测序图像的去噪方法
CN105389581B (zh) 2015-10-15 2019-08-06 哈尔滨工程大学 一种胚芽米胚芽完整度智能识别系统及其识别方法
EP3371776B1 (en) * 2015-11-03 2019-08-28 Ventana Medical Systems, Inc. Computer-implemented composite tissue image with real-time adjustable interface
CN105524827B (zh) 2015-12-02 2017-06-23 北京中科紫鑫科技有限责任公司 一种具有联动调整的dna图像采集测序系统
CN105550990B (zh) 2015-12-10 2017-07-28 北京中科紫鑫科技有限责任公司 一种基于傅里叶配准的dna图像处理方法及装置
CN105427327B (zh) 2015-12-10 2017-05-31 北京中科紫鑫科技有限责任公司 一种dna测序的图像配准方法及装置
CN105551034B (zh) 2015-12-10 2018-06-05 北京中科紫鑫科技有限责任公司 一种dna测序的图像识别的预处理方法及装置
CN105741266B (zh) 2016-01-22 2018-08-21 北京航空航天大学 一种病理图像细胞核快速定位方法
CN106295124B (zh) 2016-07-27 2018-11-27 广州麦仑信息科技有限公司 多种图像检测技术综合分析基因子图相似概率量的方法
CN107730541A (zh) 2016-08-12 2018-02-23 广州康昕瑞基因健康科技有限公司 图像配准方法和系统及图像拍摄对位方法和系统
US10467749B2 (en) 2016-10-10 2019-11-05 Genemind Biosciences Company Limited Method and system for processing an image comprising spots in nucleic acid sequencing
CN108022259A (zh) 2016-11-04 2018-05-11 北京建筑大学 干涉sar复图像配准方法和系统
CN108229098A (zh) 2016-12-09 2018-06-29 深圳市瀚海基因生物科技有限公司 单分子的识别、计数方法及装置
CN108192953A (zh) 2017-11-22 2018-06-22 深圳市瀚海基因生物科技有限公司 检测核酸特异性和/或非特异性吸附的方法
WO2020037574A1 (zh) * 2018-08-22 2020-02-27 深圳市真迈生物科技有限公司 基于图像构建测序模板的方法、碱基识别方法和装置
US12008775B2 (en) * 2018-08-22 2024-06-11 Genemind Biosciences Company Limited Method and device for image registration, and computer program product

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101950419A (zh) * 2010-08-26 2011-01-19 西安理工大学 同时存在平移和旋转情况下的快速图像配准方法
CN102663720A (zh) * 2012-03-31 2012-09-12 哈尔滨工业大学 一种基于最小均方误差准则的图像拼接方法
CN104318568A (zh) * 2014-10-24 2015-01-28 武汉华目信息技术有限责任公司 一种图像配准的方法和系统
CN105303533A (zh) * 2015-11-03 2016-02-03 华中科技大学 一种超声图像滤波方法
CN107918931A (zh) * 2016-10-10 2018-04-17 深圳市瀚海基因生物科技有限公司 图像处理方法及系统

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KENJI TAKITA ET AL., IEICE TRANS. FUNDAMENTALS, vol. E86-A, no. 8, August 2003 (2003-08-01)
See also references of EP3843032A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115516075A (zh) * 2020-05-12 2022-12-23 株式会社日立高新技术 核酸分析装置、核酸分析方法以及机器学习方法
EP4151709A4 (en) * 2020-05-12 2024-02-21 Hitachi High-Tech Corporation NUCLEIC ACID ANALYSIS DEVICE, NUCLEIC ACID ANALYSIS METHOD, AND MACHINE LEARNING METHOD

Also Published As

Publication number Publication date
EP3843032A1 (en) 2021-06-30
EP3843032A4 (en) 2021-09-15
US12008775B2 (en) 2024-06-11
US20210217186A1 (en) 2021-07-15

Similar Documents

Publication Publication Date Title
WO2021030952A1 (zh) 碱基识别方法、系统、计算机程序产品和测序系统
CN107945150B (zh) 基因测序的图像处理方法及系统及计算机可读存储介质
WO2020037572A1 (zh) 检测图像上的亮斑的方法和装置、图像配准方法和装置
CN103543277B (zh) 一种基于灰度分析与种类识别的血型结果识别算法
WO2020037573A1 (zh) 检测图像上的亮斑的方法、装置和计算机程序产品
US8073233B2 (en) Image processor, microscope system, and area specifying program
EP3306566B1 (en) Method and system for processing image
WO2020037570A1 (zh) 图像配准方法、装置和计算机程序产品
CN112289377B (zh) 检测图像上的亮斑的方法、装置和计算机程序产品
US11538261B2 (en) Systems and methods for automated cell segmentation and labeling in immunofluorescence microscopy
CN112289381B (zh) 基于图像构建测序模板的方法、装置和计算机产品
CN113012757A (zh) 识别核酸中的碱基的方法和系统
CN115082314B (zh) 一种自适应特征提取的光学表面缺陷图像分步拼接的方法
WO2020037571A1 (zh) 基于图像构建测序模板的方法、装置和计算机程序产品
CN112288781B (zh) 图像配准方法、装置和计算机程序产品
CN110246139A (zh) 基于双阈值的浮游生物原位图像roi快速提取方法
CN116563298B (zh) 基于高斯拟合的十字线中心亚像素检测方法
WO2020037574A1 (zh) 基于图像构建测序模板的方法、碱基识别方法和装置
CN116597016A (zh) 一种光纤内窥镜图像标定方法
CN112285070B (zh) 检测图像上的亮斑的方法和装置、图像配准方法和装置
CN112288783B (zh) 基于图像构建测序模板的方法、碱基识别方法和装置
CN113128500A (zh) 一种基于Mask-RCNN的非机动车车牌识别方法及系统
CN118096640A (zh) 用于鉴定图像序列中的分析物的方法
CN117789214A (zh) 一种芯片表面字符识别方法、系统及存储介质
CN114821057A (zh) 局部最大值点阈值膨胀的mpi图像分割方法、系统及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18930564

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018930564

Country of ref document: EP

Effective date: 20210322