WO2023045116A1 - 图像构建方法 - Google Patents

图像构建方法 Download PDF

Info

Publication number
WO2023045116A1
WO2023045116A1 PCT/CN2021/138107 CN2021138107W WO2023045116A1 WO 2023045116 A1 WO2023045116 A1 WO 2023045116A1 CN 2021138107 W CN2021138107 W CN 2021138107W WO 2023045116 A1 WO2023045116 A1 WO 2023045116A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
image
imaging
target
splitting device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/138107
Other languages
English (en)
French (fr)
Inventor
李剑平
陈涛
马文齐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2023045116A1 publication Critical patent/WO2023045116A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Definitions

  • the invention relates to the technical field of photoelectric imaging, in particular to an image construction method.
  • optical parameters of an optical imaging system include optical resolution, depth of field, dynamic range, color state, polarization state, and spectral state, etc. These properties play a vital role in the practical application of the imaging system.
  • optical imaging method itself or the working principle of the photosensitive device, it is difficult to achieve a high level of the above optical parameters at the same time, such as:
  • the imaging system must sacrifice a certain optical resolution and sensitivity to perform color imaging
  • the imaging system sacrifices the diversity of imaging targets to achieve high dynamic range imaging of a certain type of target.
  • the Chinese invention patent application document with the application publication number CN113052766A discloses a multi-scale imaging device, a large field of view and high resolution image stitching method and system, in which for the gap between the large field of view and high resolution Imaging Contradiction proposes a solution to achieve the effect of coordinating the two by splicing images.
  • deep learning technology usually has higher requirements on the data set, not only the number of image pairs in the data set is large, but also the diversity is good.
  • the application of deep learning technology in the enhancement of optical parameters of imaging systems is still relatively small.
  • the present invention proposes an image construction method, which can construct image pairs or image clusters or video pairs of various optical parameters, satisfying the data set of deep learning Require.
  • the technical solution adopted in the present invention is an image construction method, the steps of which include:
  • the information of the observed sample is divided into multiple beams of signal light through the beam splitting device, and each beam of light emitted by the beam splitting device passes through a different imaging unit to form a target image containing the information;
  • Construct paired images or clustered images change the optical parameters of the different beams emitted by the beam splitting device when they are imaged, so that different imaging units simultaneously form target images with different optical parameters, and obtain paired images or clustered images of the same observation sample image;
  • Image registration processing align multiple images of the same observation target to obtain image pairs or image clusters or video pairs for each observation target;
  • the optical parameters at least include at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • the beam splitting device emits two beams, and the two beams respectively pass through an imaging unit to form an image containing the information, and the optical parameters are optical resolution, defocus distance, brightness, polarization state, color, spectrum , one of fluorescence.
  • the beam splitting device emits three or more light beams, and each beam of light emitted by the beam splitting device passes through an imaging unit to form an image containing the information, and the optical parameters include at least optical resolution, One or more of defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • At least one pair of imaging units adopts lenses of the same magnification, one of which adopts a color camera and the other adopts a monochrome camera, adjusting the two imaging units
  • the focal planes of the units are coincident and imaged at the same time.
  • At least one pair of imaging units adopts lenses with the same magnification, and one or more filters are inserted between one of the imaging units and the beam splitting device and select different wavelength bands of light beams to enter the imaging unit for imaging, and then adjust the two imaging units to image clearly, and then image at the same time.
  • At least one pair of imaging units adopts lenses with the same magnification, adjusts the exposure time and gain of one of the imaging unit cameras, and then adjusts the imaging of both imaging units to be clear After that, image simultaneously.
  • At least one pair of imaging units adopts lenses with different magnifications, and then adjust the focal planes of the two imaging units to overlap, adjust the depth of field to the illumination range of the observed sample Inside, imaging at the same time.
  • At least one pair of imaging units adopts lenses with the same magnification, and the focal planes of the two imaging units are adjusted to coincide, and one of the imaging units is fixed.
  • the other imaging unit moves along the optical axis direction of the light beam, and the two imaging units simultaneously image images at different positions.
  • At least one pair of imaging units uses lenses with the same magnification, and adjusts the light beam that irradiates the observation sample to monochromatic light, and replaces the beam splitting device It is a dichroic mirror, after adjusting the two imaging units to image clearly, they image simultaneously.
  • the light beam irradiating the observation sample is allowed to pass through a polarizer first, and one of the beams emitted by the beam splitting device is then allowed to pass through a polarizer, wherein the other One light beam does not pass through the polarizer, and the two light beams are simultaneously imaged by two different imaging units.
  • the information of the observed sample is divided into multiple beams of signal light by the beam splitting device and emitted, specifically: the illuminator is used to emit a beam of light, and after the beam is irradiated on the observed sample, it is scattered to the beam splitting device .
  • the illuminator is located at one side of the observation container, and the light beam of the illuminator is emitted along the optical axis of the incident light of the beam splitting device.
  • the illuminator is arranged on one side of the observation container, and the angle between the light beam of the illuminator and the optical axis of the incident light of the beam splitting device is greater than 90°.
  • the illuminator is arranged between the observation container and the beam splitting device, and the angle between the light beam of the illuminator and the optical axis of the incident light of the beam splitting device is less than 90°.
  • the illuminator is arranged around the observation container, and the light beam emitted by the illuminator is perpendicular to the optical axis of the incident light of the beam splitting device.
  • the devices used when constructing pairs of images or clustered images include:
  • a sample observation device which sends out sample information in the form of an optical signal
  • a beam splitting device which receives the light beam containing the optical signal and splits it into multiple beams with different paths;
  • An imaging device which includes at least two imaging units, and respectively receives at least one of the beams containing optical signals emitted by the beam splitting device for simultaneous imaging to obtain paired images or clustered images of the same observation sample;
  • An adjustment device which is arranged at the imaging unit, and/or between the imaging unit and the beam splitting device, to adjust the optical parameters of the imaging unit;
  • the optical parameters at least include at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • the adjustment device includes at least one of the following structures:
  • a position parameter adjustment structure arranged at the imaging unit and/or the beam splitting device, to adjust the distance between each of the imaging units and the beam splitting device;
  • An optical element adjustment structure arranged between the imaging device and the beam splitting device and/or between the illuminator and the sample observation device, to adjust the polarization state and/or wavelength band and/or brightness of the light beam;
  • An imaging unit adjustment structure the imaging unit includes a camera and a lens/multiple different lenses, and the imaging unit adjustment mechanism is used to switch the positions of multiple lenses in an imaging unit;
  • the beam splitting device adjustment structure the beam splitting device is one or more three-dimensional beam splitters or beam splitters or dichroic mirrors, the beam splitting device adjusting structure is used to adjust the number or type of the beam splitting device.
  • the position parameter adjustment structure includes an adjustment track arranged along the optical axis direction of the outgoing beam of the beam splitting device, and each of the imaging units is slidably fitted on the adjustment track to adjust the alignment with the beam splitter. distance between devices.
  • the optical element adjustment structure includes a mounting part, and the mounting part is detachably connected/replaceably connected to at least one of one or more polarizing plates, attenuating plates, and optical filters.
  • the image registration process includes:
  • aligning the target area whose judging result is clear with the target area contained in the other images in the paired images or clustered images refers to: extracting the coordinate information of the target area whose judging result is clear, and transforming the coordinate information to obtain According to the coordinate information of the same target area contained in the other images in the paired images or the clustered images, multiple target areas of the same observation target are cropped according to their respective coordinate information.
  • aligning the clear target area with the target area contained in the paired image or the remaining images in the clustered image refers to: using the clear target area as a template, and the paired image or clustered image Template matching is performed on other images in the image, and the area with the highest matching degree is the same target area, and image pairs or image clusters or video pairs are obtained by cropping and storing.
  • extracting the coordinate information of the target area whose judgment result is clear refers to: extracting the coordinates of the four vertices of the target area whose judgment result is clear, and transforming the coordinates of the four vertices to obtain paired images or clustered images. The coordinates of the four vertices corresponding to the target area in the remaining images.
  • the transformation of the coordinates of the four vertices is assisted by a registration model
  • the registration model includes a target
  • the target provides a plurality of plane/space features that can be detected or identified, and each The features are assigned coordinate information.
  • the target is a planar grid, and each intersection point of the grid can be detected or identified.
  • the target is a flat plate with a plurality of evenly distributed prisms protruding from it, and each corner point of the prisms can be detected or identified.
  • the present invention has the following beneficial effects:
  • the collected paired images or clustered images are aligned to form image pairs or image clusters or video pairs, which can meet the high requirements for images in deep learning, and then facilitate the direct construction of observation target dataset;
  • Fig. 1 is a schematic diagram of the system of the two-way imaging unit of the present invention.
  • Fig. 2 is a system schematic diagram of a three-way imaging unit of the present invention
  • Fig. 3 is the structural representation of observation vessel of the present invention.
  • Fig. 4 is a schematic diagram of a light source installation structure
  • Fig. 5 is a schematic diagram of the first light source installation method
  • Fig. 6 is a schematic diagram of the second light source installation method
  • Fig. 7 is a schematic diagram of a third light source installation method
  • Figure 8 is a front view of the first target
  • Fig. 9 is a side view of Fig. 8.
  • Fig. 10 is the front view of the second target
  • Figure 11 is a comparison diagram of plankton imaging at high magnification and low magnification
  • Figure 12 is a comparison diagram of high-resolution and low-resolution target imaging.
  • Image construction method the steps include:
  • the information of the observed sample is divided into multiple beams of signal light by the beam splitting device, and each beam of light emitted by the beam splitting device passes through a different imaging unit 7 to form a target image containing the information;
  • Constructing paired images or clustered images changing the optical parameters of the different beams emitted by the beam splitting device when they are imaged, so that different imaging units 7 simultaneously form target images with different optical parameters, and obtain paired images or clustered images of the same observation sample. cluster image;
  • Image registration processing align multiple images of the same observation target 3 to obtain an image pair or image cluster or video pair for each observation target 3;
  • the optical parameters at least include at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • each image contains images of multiple observation objects 3, and there is a certain deviation between each image, so it is impossible to directly form an image pair, image cluster, or video pair of the same observation object 3. Therefore, when constructing an image pair and image cluster of a certain observation target 3, it is still necessary to reprocess the imaged image, that is, after the above-mentioned image registration processing, the image pair or image corresponding to each observation target 3 can be obtained.
  • Image clusters or video pairs when constructing image pairs or image clusters or video pairs.
  • different optical parameters of the beams emitted by the beam splitting device can be changed to realize different image pairs or image clusters or video pairs of the same observation target 3 under different optical parameters, satisfying the depth Diversity requirements for learning technologies.
  • the imaging unit 7 When the imaging unit 7 performs imaging, different numbers of optical paths can be formed depending on the beam splitting device, that is, the number of beams emitted by the beam splitting device corresponds to the number of imaging units 7. Based on this, the actual imaging has the following conditions:
  • the beam splitting device emits two beams, as shown in Figure 1, the two beams respectively pass through an imaging unit 7 to form an image containing the information, the optical parameters are optical resolution, defocus distance, brightness, polarization state, color One of , spectrum and fluorescence;
  • the beam splitting device emits three or more beams of light, as shown in Figure 2, each beam of light emitted by the beam splitting device passes through an imaging unit 7 to form an image containing the information, and the optical parameters include at least optical resolution One or more of power, defocus distance, brightness, polarization state, color, spectrum, fluorescence.
  • the imaging pair or imaging cluster is obtained by simultaneously imaging the same observation target 3 under different optical parameters, when the beam splitting device only emits two beams, only two optical paths are formed, so only two imaging paths need to be set correspondingly
  • the unit 7 is enough, and the images formed by the two imaging units 7 can constitute a pair of images. Only when the beam splitting device emits three or more beams of light paths, can a plurality of imaging units 7 be set correspondingly, and then by changing the optical parameters of each imaging unit 7, the same observation target 3 can be formed under different optical parameters.
  • the different optical parameters can be the same optical parameter such as the numerical adjustment of the magnification, or it can refer to the simultaneous change of multiple optical parameters, such as changing the magnification of one beam and changing the magnification of the other beam
  • the change of the polarization state, the third channel can be used as the control group of the first two channels, so as to obtain a set of consistent image clusters.
  • optical parameters whether it is two optical paths or multiple optical paths, at least the following methods can be used when adjusting optical parameters:
  • At least one pair of imaging units 7 use lenses with the same magnification, one of which uses a color camera and the other uses a monochrome camera, adjust the two imaging units
  • the focal planes of unit 7 are coincident and imaged at the same time;
  • At least one pair of imaging units 7 should use lenses with the same magnification, and one or more optical filters should be inserted between one of the imaging units 7 and the beam splitting device , and select light beams of different wavelength bands to enter the imaging unit 7 for imaging, and then adjust the two imaging units 7 to image clearly and simultaneously image;
  • At least one pair of imaging units 7 should use lenses with the same magnification, adjust the exposure time and gain of one of the imaging unit 7 cameras, and then adjust the imaging of both imaging units 7 to be clear After that, imaging at the same time;
  • At least one pair of imaging units 7 should use lenses with different magnifications, and then adjust the focal planes of the two imaging units 7 to overlap and adjust the depth of field to within the illumination range of the observed sample , while imaging;
  • At least one pair of imaging units 7 adopts lenses with the same magnification, and the focal planes of the two imaging units 7 are adjusted to coincide, and one of the imaging units 7 is fixed.
  • Another imaging unit 7 moves along the optical axis direction of the light beam, and the two imaging units 7 image simultaneously at different positions;
  • At least one pair of imaging units 7 should use lenses with the same magnification, and adjust the light beam illuminating the observation sample to monochromatic light, and replace the beam splitting device with two
  • the dichroic mirror is used to adjust the imaging of the two imaging units 7 to form images at the same time;
  • the optical parameters to be adjusted are not limited to the adjustment methods listed above, and other optical parameters can also be appropriately adjusted by referring to the aforementioned methods, which will not be exhaustive in this embodiment.
  • plankton in order to collect images of plankton with higher contrast and more detailed information, the plankton in larger water bodies can be imaged, and at the same time, high signal-to-noise images can be obtained for smaller-sized plankton.
  • dark-field imaging can be used to collect data pairs with different defocus degrees and image pairs with different optical resolutions, so as to expand the depth of field of the imaging system through deep learning methods and improve the optical resolution at the same time. Imaging that satisfies high contrast, large spatial range and high spatial resolution;
  • reflective illumination can be used to construct bright fields with different defocus degrees and different optical resolutions. Image pairs or image clusters or video pairs, and then enhance the depth of field and optical resolution of the reflective brightfield imaging system through deep learning technology;
  • the fluorescence can be excited by laser grazing illumination, and at the same time, the reflected light can be used for bright field imaging. Imaging, constructing image pairs of bright-field microscopy images and fluorescence images, combined with deep learning technology, makes the bright-field microscopy imaging system have the ability of fluorescence imaging.
  • the information of the observed sample is divided into multiple beams of signal light by the beam splitting device, specifically, the illuminator is used to emit a beam of light, and after the beam is irradiated on the observed sample, it is then scattered to the beam splitter.
  • the arrangement of the illuminators is as follows:
  • the illuminator is located on one side of the observation container 1, as shown in FIG. 5, and the light beam of the illuminator is emitted along the optical axis of the incident light of the beam splitting device to form a bright field imaging mode;
  • the illuminator is arranged on one side of the observation container 1, as shown in FIG. 6, and the angle between the light beam of the illuminator and the optical axis of the incident light of the beam splitting device is greater than 90°, forming a dark field imaging mode;
  • the illuminator is arranged between the observation container 1 and the beam splitting device, as shown in Figure 7, and the angle between the light beam of the illuminator and the optical axis of the incident light of the beam splitting device is less than 90°, forming a reflective bright light. field imaging mode;
  • the illuminator is arranged around the observation container 1 , as shown in FIG. 1 , and the light beam emitted by the illuminator is perpendicular to the optical axis of the incident light of the beam splitting device.
  • the illuminator can illuminate the observation sample by various illumination methods, and according to the above-mentioned various adjustment methods, the required paired images or clustered images are formed in one or more imaging units 7, Due to the misalignment between each image, image registration processing is also required, including:
  • aligning the target area whose judging result is clear with the target areas contained in the other images in the paired images or clustered images refers to: extracting the coordinate information of the target area whose judging result is clear, and transforming the coordinate information to obtain the For the coordinate information of the same target area contained in the other images in the image or clustered images, multiple target areas of the same observation target 3 are cropped according to their respective coordinate information.
  • Extracting the coordinate information of the target area whose judgment result is clear refers to: extracting the coordinates of the four vertices of the target area whose judgment result is clear, and transforming the coordinates of the four vertices to obtain the paired images or the remaining images in the clustered images. The coordinates of the four vertices corresponding to the target area.
  • a registration model can be formed by making the target 6, as shown in Figure 8-10, the target 6 provides multiple plane/space features that can be detected or identified, and each of the features Assign coordinate information.
  • the target 6 can be a planar grid, that is, the coordinates of each intersection point of the grid are known; The coordinates of the corner points are all known, and the prism can be a triangular prism, a quadrangular prism or a multi-prism.
  • Another method for aligning the target area in which the target area that is judged to be clear is aligned with the target area contained in the other images in the paired image or clustered image, referring to: using the result
  • the clear target area is used as a template, and template matching is performed with other images in the paired image or clustered image.
  • the area with the highest matching degree is the same target area, and the image pair or image cluster or video pair is obtained by cropping and storing.
  • the image pair or image cluster or video pair of the same observation target 3 can be obtained, and different optical parameters can be changed to meet the sample diversity requirements of deep learning technology, and at the same time meet the requirements in terms of quantity.
  • Learning technology provides support for data set construction. Compared with the imaging of the observation target 3 in the prior art, this solution is also applicable to the observation target 3 in motion, with a wider application range and higher imaging precision.
  • Embodiment 1 an apparatus for constructing image pairs or image clusters or video pairs, as shown in Figure 1-2, includes:
  • a sample observation device which sends out sample information in the form of an optical signal
  • a beam splitting device which receives the light beam containing the optical signal and splits it into multiple beams with different paths;
  • An imaging device which includes at least two imaging units 7, and respectively receives at least one of the light beams containing optical signals emitted by the beam splitting device for simultaneous imaging to obtain paired images or clustered images of the same observation sample;
  • An adjustment device which is arranged at the imaging unit 7, and/or, between the imaging unit 7 and the beam splitting device, to adjust the optical parameters of the imaging unit 7;
  • the optical parameters at least include at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • the observation sample is a liquid containing plankton
  • the observation target 3 is a single plankton, as shown in Figure 1-4
  • the observation container 1 is a hollow cuboid, and one side extends horizontally outwards with a prominent observation Part 11, the observation part 11 is also hollowly arranged and communicated with the cuboid body part of the observation container 1.
  • the observation part 11 can be a cuboid or a cylinder, and an illuminator is installed on it.
  • the illuminator is a ring lamp.
  • the upper part of the observation container 1 is connected with a storage device for storing the sample solution through a pipeline, so as to replenish liquid in time
  • the bottom of the observation container 1 is also connected to the liquid discharge device through a pipeline
  • the liquid discharge pipeline is provided with a valve for controlling opening and closing, In order to discharge the sample solution after observation.
  • the draining pipeline is connected with an air pump.
  • the air pump starts, not only the observed sample solution in the observation container 1 can be discharged cleanly, but also the inside of the observation container 1 can be pumped into a negative pressure to realize the sample solution.
  • Automatic liquid feeding reducing manual operation.
  • the effect of automatic imaging and automatic replacement of observation samples can be realized.
  • a horizontal partition is also installed in the middle of the observation container 1.
  • plate 13 the bottom of the partition 13 should be level with the bottom of the observation part 11 , and the drain pipe passes through the partition 13 to communicate with the outside.
  • the target 6 also needs to be installed, so an installation groove 12 is formed on the upper surface of the horizontally extending observation part 11, and the opening direction of the installation groove 12 is in line with the extension direction of the observation part 11. vertical.
  • the entire observation container 1 is made of highly transparent material, such as acrylic, quartz glass and the like.
  • the illuminator in this embodiment is an annular light source 2, as shown in FIG. 4, which may be an LED aperture, which is sleeved on the observation part 11 to form an annular illumination.
  • a light source 2 adjustment frame and a controller are provided. The controller is used to control the light source 2 to turn on and off, and the light source 2 adjustment frame is used to fix the position of the light source 2 and adjust the position according to requirements.
  • the light source 2 adjustment frame includes two telescopic uprights 5, and the bottom of the uprights 5 is installed and connected with the fixed surface by bolts, so that the light source 2 can adjust the height, and also can adjust the installation position with the fixed surface.
  • the upper end of the column 5 of the light source 2 adjustment frame is also fixed with a horizontal connecting rod 51, the connecting rod 51 and the light source 2 are detachably connected by bolts, and the connecting rod 51 is a horizontal waist-shaped hole for the bolts to pass through, so that The position between the light source 2 and the connecting rod 51 can also be changed, and the adjustment of the light source 2 in multiple directions can be realized through the above arrangement.
  • the ring light source 2 is only one of the embodiments. According to the lighting requirements of different imaging methods, different lighting methods as mentioned above can also be used. Correspondingly, different installation forms and structures need to be changed , which will not be repeated here.
  • the beam splitting device can adopt a stereo beam splitter 4 or a beam splitter or a dichroic mirror. According to the requirements of different optical parameters, three types can be replaced and selected. In this embodiment, a stereo beam splitter 4 is used. And by changing the number of beam splitters, the number of imaging light paths can be changed. If it is necessary to adjust the three optical parameters of the same observation target 3, two stereoscopic beam splitters 4 arranged along the optical axis can be provided, so that three optical paths can be formed, and three imaging units 7 can be provided correspondingly.
  • a camera, a telecentric lens and a lens holder are used in this embodiment, and the telecentric lens is placed on the lens holder.
  • the observation part 11 of the observation container 1, the stereoscopic beam splitter 4 and the imaging unit 7 are all installed along the axis of the optical path.
  • an adjustment device which includes at least one of the following structures:
  • a position parameter adjustment structure arranged at the imaging unit 7 and/or the beam splitting device, to adjust the distance between each imaging unit 7 and the beam splitting device;
  • An optical element adjustment structure arranged between the imaging device and the beam splitting device and/or between the illuminator and the sample observation device, to adjust the polarization state and/or wavelength band and/or brightness of the light beam;
  • An imaging unit adjustment structure the imaging unit 7 includes a camera and a lens/multiple different lenses, and the imaging unit adjustment mechanism is used to switch the positions of multiple lenses in an imaging unit 7;
  • the beam splitting device adjustment structure the beam splitting device is one or more three-dimensional beam splitters 4 or beam splitters or dichroic mirrors, the beam splitting device adjusting structure is used to adjust the number or type of the beam splitting device.
  • the adjusting device includes a position parameter adjusting structure, an optical element adjusting structure, an imaging unit adjusting structure and a beam splitting device adjusting structure.
  • the position parameter adjustment structure includes an adjustment track arranged along the optical axis direction of the outgoing beam of the beam splitting device, and the imaging unit 7 that needs to adjust the distance position is slidably fitted on the adjustment track to adjust the distance with the beam splitter. distance between devices.
  • the optical element adjustment structure includes a mounting part, and the mounting part is detachably connected/replaceably connected with one or more polarizers, attenuation sheets, and optical filters, and the polarizers, attenuation sheets, and optical filters are all sequentially installed on each imaging On the optical path between unit 7 and the fractionator.
  • the installation part includes a mounting frame, and a vertical mounting plate is connected to the mounting frame in rotation, and each mounting plate is detachably fixed with a plurality of polarizers or attenuation plates or filters with different parameters or quantities. You can switch between different optical parameters of the same optical path by installing the disk.
  • the imaging unit adjustment structure is similar to the installation part structure of the optical element adjustment structure. Multiple cameras with different magnifications or gains or exposure times are installed on a rotatable mounting disk. By rotating the mounting disk, the same imaging unit can be realized. Switching of cameras with different parameters in .
  • the adjustment of the beam splitter is mainly realized by the cylinder, that is, the adjustment structure of the beam splitter includes a plurality of cylinders, and the end of the telescopic rod of each cylinder is connected with a mounting plate, and a three-dimensional beam splitter 4 or a beam splitter or beam splitter is positioned on the mounting plate.
  • Dichroic mirrors when the beam splitter needs to be adjusted, because the beam splitters driven by the ends of multiple cylinders are different, thus driving different cylinders, they can be combined to form different numbers or types of beam splitters, such as in a certain Only a single stereoscopic beam splitter 4 is placed in one optical path, or two stereoscopic beam splitters 4 can be placed in sequence, or the stereoscopic beam splitter 4 can be replaced with a dichroic mirror.
  • Embodiment 2 the device for constructing image pairs or image clusters or video pairs, differs from Embodiment 1 in that Embodiment 1 is a general-purpose device that can satisfy most optical parameter adjustments and has wide applicability, but the structure is also It is relatively complicated, and part of the structure is simplified in this embodiment, which is a customized type formulated for special observation situations.
  • the observation of plankton is taken as an example, and the purpose is to obtain image pairs or image clusters or video pairs of plankton in the sample solution.
  • the device comprises an illuminator, an observation container 1, a beam splitter, an imaging device and an adjustment device, the structure of the illuminator and the observation container 1 is the same as in Embodiment 1, and the beam splitter is a three-dimensional beam splitter 4 ,
  • the imaging device contains two groups of telecentric lenses, a camera and a lens holder.
  • the adjustment device (not shown in the figure) includes a position parameter adjustment structure and an imaging unit adjustment structure.
  • the position parameter adjustment structure is an adjustment track laid under each imaging device.
  • the imaging unit adjustment structure includes a fixed column 5, and the column 5 A fixed disk is connected with the upper rotation, and multiple lenses with different magnifications are fixedly mounted on the fixed disk.
  • the position of the two-way lens and one of the cameras can be adjusted by adjusting the track, and the distance between it and the beam splitter can be changed to obtain paired images with different degrees of defocus;
  • the rotation of the disk switches the lenses of different magnifications, so as to obtain paired images of different magnifications. Both can provide original images for subsequent image alignment processing.
  • two three-dimensional beam splitters 4 placed in sequence may also be provided, and the splitting ratio of the first-stage beam splitter is 1:2, and the splitting ratio of the second-stage beam splitter is 1: 1.
  • the optical signal emitted from the observation target 3 is divided into three paths, and the corresponding imaging device also includes three groups of imaging units 7 .
  • the adjustment device includes a position parameter adjustment structure and an imaging unit adjustment structure.
  • the position parameter adjustment structure is an adjustment track laid under each imaging device;
  • the imaging unit adjustment structure includes a fixed column 5, and a fixed plate is rotatably connected to the column 5 A plurality of lenses with different magnifications are fixedly mounted on the fixed plate, and different magnifications are realized by rotating the fixed plate.
  • three images can be obtained in one imaging, and the three images can form two pairs of images, which is more convenient.
  • the three optical paths can also be used to adjust the magnification or the distance at the same time, so that three clustered images with different magnifications or three clustered images with different defocus degrees can be obtained.
  • the optical parameters to be adjusted are also different, so the type and quantity of the beam splitting device and the number of imaging units 7 can be adjusted according to actual requirements, and the adjustment can be changed accordingly The composition and installation location of the device.
  • Embodiment 3 image construction method, it adopts the device in embodiment 1 or embodiment 2, concrete steps are:
  • Step 1 put the solution of the observation sample into the observation container 1, install and fix the illuminator and turn it on, the information of the observation sample is divided into multiple beams of signal light by the beam splitter and emitted, and each beam of light emitted by the beam splitter passes through different
  • the imaging unit 7 forms an object image containing said information.
  • the optical parameter is at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.
  • Step 2 changing the optical parameters of the different light beams emitted by the beam splitting device when imaging separately, so that different imaging units 7 simultaneously form target images with different optical parameters. Observe paired or clustered images of the sample.
  • the present embodiment provides the following control mode:
  • the camera working in the free running mode outputs the exposure status signal, triggers the non-emitting camera and illuminator working in the external trigger mode, and realizes the simultaneous acquisition of image pairs of stationary or moving targets;
  • the cameras of the multi-channel imaging system work in the external trigger mode at the same time, it can be set to generate a trigger signal through an external signal source, trigger two cameras and illuminator to work at the same time, and collect image pairs.
  • the external signal source can be a shutter button triggered manually, or an illuminator start signal automatically recognized.
  • optical parameters Regardless of whether it is two optical paths or multiple optical paths, at least the following methods can be used when adjusting optical parameters:
  • At least one pair of imaging units 7 use lenses with the same magnification, one of which uses a color camera and the other uses a monochrome camera, adjust the two imaging units
  • the focal planes of unit 7 are coincident and imaged at the same time;
  • At least one pair of imaging units 7 should use lenses with the same magnification, and one or more optical filters should be inserted between one of the imaging units 7 and the beam splitting device , and select light beams of different wavelength bands to enter the imaging unit 7 for imaging, and then adjust the two imaging units 7 to image clearly and simultaneously image;
  • At least one pair of imaging units 7 should use lenses with the same magnification, adjust the exposure time and gain of one of the imaging unit 7 cameras, and then adjust the imaging of both imaging units 7 to be clear After that, imaging at the same time;
  • At least one pair of imaging units 7 should use lenses with different magnifications, and then adjust the focal planes of the two imaging units 7 to overlap and adjust the depth of field to within the illumination range of the observed sample , while imaging;
  • At least one pair of imaging units 7 adopts lenses with the same magnification, and the focal planes of the two imaging units 7 are adjusted to coincide, and one of the imaging units 7 is fixed.
  • Another imaging unit 7 moves along the optical axis direction of the light beam, and the two imaging units 7 image simultaneously at different positions;
  • At least one pair of imaging units 7 should use lenses with the same magnification, and adjust the light beam illuminating the observation sample to monochromatic light, and replace the beam splitting device with two
  • the dichroic mirror is used to adjust the imaging of the two imaging units 7 to form images at the same time;
  • the device is a three-way or more imaging unit 7, the above-mentioned multiple optical parameters can be adjusted for one observation target 3 at the same time, and multiple images can be obtained at one time to form a clustered image; What is more important is that two imaging units 7 perform imaging, then only one optical parameter can be adjusted at a time to obtain a pair of images.
  • Step 3 Align multiple images of the same observation target 3 to obtain an image pair or image cluster or video pair for each observation target 3 . Due to errors in the assembly and processing of parts of the multi-channel imaging part, and the inconsistency of the magnification of the imaging part, the image of the same target in the image pair or image cluster or video pair collected by the multi-channel imaging part is not complete in the image space.
  • the specific method of alignment processing is as follows:
  • step 2 perform basic preprocessing on the paired images or clustered images obtained by camera imaging in step 2, such as color correction, background subtraction, CLAHE contrast enhancement, etc.;
  • the target detection algorithm commonly used in the prior art to detect the observation target 3 on one image in the paired images or clustered images, and obtain the target regions (ROIs) of multiple observation targets 3 , wherein, if the paired images or the clustered images If the cluster images contain images with different magnifications, images with high magnifications should be selected for target detection;
  • the sharpness judgment of the image detected by the observation target 3 can be judged manually, or can be judged and screened by means of an algorithm, and the clear target area of the judgment result is retained, and the blurred target area of the judgment result is discarded. If there is no clear target area, go back to the previous step and select an image again;
  • the coordinates of the four vertices of the target area in the original image are extracted during the observation target detection, and then the coordinates of the four vertices are transformed using the registration model to obtain paired images or clusters Four vertex coordinates of the same target area on the remaining images in the image;
  • the imaging coordinates of an observation target 3 in all images are known, and can be cut and saved according to the coordinates, and the image pair or image cluster or video pair of the observation target 3 can be obtained.
  • a registration model is needed.
  • the construction of the registration model mainly relies on the target 6 to assist acquisition.
  • the target 6 provides multiple planes/spaces that can be detected or identified feature, and assign coordinate information to each of the features.
  • the size of the target 6 is based on the principle that it can completely cover the field of view of the imaging unit 7 after being placed into the observation container 1.
  • the distribution and quantity of the features are based on the resolution and the viewing angle Depends on the size of the field.
  • the target 6 can adopt different forms and structures, such as the dark-field imaging illumination in Embodiment 2.
  • the target 6 is a three-dimensional structure, which includes a vertical flat plate, and a plurality of protrusions are arranged on the flat plate. Uniformly distributed quadrangular prism, each corner point of the prism can be detected or identified; if it is in the form of bright field illumination, the target 6 can be a two-dimensional grid, and each intersection point of the grid can be detected or identify.
  • multiple features distributed according to a certain spatial law can be constructed in the field of view, and registration can be established through a series of standard processes of image registration such as feature extraction, feature matching, and least squares solution.
  • the model can obtain the coordinate information of the same observation target 3 in the rest of the images after obtaining the coordinate information of the four points of the target area.
  • another image alignment processing method is also provided, and its specific processing method is:
  • step 2 perform basic preprocessing on the paired images or clustered images obtained by camera imaging in step 2, such as color correction, background subtraction, CLAHE contrast enhancement, etc.;
  • paired images or clustered images with the same magnification select one of the images for target detection and crop the target region (ROI).
  • ROI target detection and crop the target region
  • the sharpness judgment of the image detected by the observation target 3 can be judged manually, or can be judged and screened by means of an algorithm, and the clear target area of the judgment result is retained, and the blurred target area of the judgment result is discarded. If there is no clear target area, go back to the previous step and select an image again;
  • the clear target area is used as a template to perform template matching with other images in the paired image or clustered images, and the highest matching area in the image is cut out, and
  • the image of the target area in the previous step constitutes an image pair or image cluster or video pair and saves it;
  • Embodiment 4 image construction method, it adopts the device in embodiment 2, and the difference with embodiment 3 is that two three-dimensional beam splitters 4 placed in sequence are provided, and the light splitting ratio of the first-order beam splitter is 1:2, the light splitting ratio of the second-stage beam splitter is 1:1, so that the optical signal emitted from the observation target 3 is divided into three paths, and the corresponding imaging device also includes three groups of imaging units 7 .
  • control method is as follows:
  • the imaging unit adjustment structure is used to change the lens of the three-way imaging unit 7 to form three different magnifications, and three clusters of the same observation target 3 with different magnifications can be obtained after simultaneous imaging image;
  • the three-way imaging part uses telecentric lenses with the same magnification, and inserts band-pass filters of different bands into two of them through the installation part in the optical element adjustment structure, and the remaining one does not Without adding optical components, the camera collects images at a fixed frame rate, and obtains clustered images of the same target containing different bands;
  • the three-way imaging system uses telecentric lenses with the same magnification, and both the first and second beam splitters use dichroic mirrors, but the two dichroic mirrors Reflection and transmission wavelengths are different, the camera captures images at a certain frame rate, and imaging at the same time can obtain clustered images of the same target containing fluorescence and scattered light of different bands;
  • the three-way imaging part uses a telecentric lens with the same magnification, uses polarized light for illumination, and adjusts the position of the installation part in the three-way imaging part through optical elements.
  • Add polarizers to two paths adjust the angles of polarizers in the imaging part of the two paths, but the angles of the polarizers between the two paths are different, and collect images at the same time, you can get the same target including non-polarized images and clusters with different polarization angles image.
  • optical parameters for three-way imaging, different optical parameters can also be combined, such as:
  • Two of the three-way imaging systems use telecentric lenses of the same magnification, and the last one uses telecentric lenses of different magnifications for imaging.
  • the three-way imaging part adopts a telecentric lens with the same magnification.
  • a band-pass filter is inserted in one of the paths, and a polarizer is inserted in the other path.
  • the remaining path does not include optical elements.
  • the camera Acquire images at a fixed frame rate, and obtain clustered images of the same target including different wave bands and different polarization states.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Microscoopes, Condenser (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

一种图像构建方法，包括获得目标图像；改变分束装置射出的不同光束各自成像时的光学参数，使不同成像单元（7）同时形成不同光学参数的目标图像，得到同一观察样品的成对图像或成簇图像；将同一观察目标（3）的多个图像进行对齐处理，得到每个观察目标（3）的图像对或图像簇或视频对。通过上述设置，通过调节不同的光学参数或者同一个光学参数的不同数值，可以形成不同要求的多种成对图像或成簇图像，再进行图像配准处理，达到对齐的效果，满足了深度学习的多样性、高精度和数量上的要求，为构建数据集提供了可行的基础方案。

Description

图像构建方法 技术领域
本发明涉及光电成像技术领域，尤其涉及图像构建方法。
 背景技术
光学成像系统的光学参数包括光学分辨率、景深、动态范围、色彩状态、偏振状态和光谱状态等多种，这些性能对于成像系统的实际应用起到至关重要的作用。但受到光学成像方法本身的限制，或者感光器件的工作原理的限制，导致上述光学参数很难同时达到较高的水平，如：
光学分辨率与景深和视场的矛盾，导致成像系统难以对大空间范围内的目标进行高分辨率地成像；
受到感光器件只能对光强进行响应的特征，成像系统要牺牲一定的光学分辨率和灵敏度来进行彩色成像；
观测目标物理属性复杂的原因，成像系统牺牲成像目标的多样性来实现对某一类目标的高动态范围成像。
针对上述问题，申请公布号为CN113052766A的中国发明专利申请文件中即公开了一种多尺度成像装置、大视场高分辨率图像拼接方法及系统，其中针对大视场和高分辨率之间的成像矛盾提出了一种方案，通过拼接图像的方式达到协调二者的效果。
但是上述方案不仅较为麻烦，而且针对性较强，只能适用于大视场和高分辨率的问题上，无法满足对其他多种参数的调节需求。基于此，随着基于数据驱动的人工智能技术的发展，越来越多的研究都放在了通过构建上述不同光学参数的图像对数据集来训练相应的模型，从而改善成像系统的矛盾和限制，增强成像系统的成像性能这一方法上。比如，通过构建低分辨率和高分辨率图像对训练模型，从而增强低分辨率图像的分辨率，同时保持原有的视场和景深，实现大范围内的高分辨率成像。
为了保证模型对实际应用场景的高性能和高鲁棒性，深度学习技术通常对数据集的要求较高，不仅需要数据集中图像对的数量较多，还需其多样性较好。但由于实际不同光学参数的图像对数据集的构建难度较大，导致目前深度学习技术在成像系统光学参数增强方面的应用仍然较少。
申请人发现在现有技术中有提出一种水下光学成像的装置，其通过一个透镜收集光线后，经过分光板将光线分成两路进行成像，通过设置两路成像的相机的安装位置与分光板距离的不同，获得不同离焦程度的一对图像。
但是很明显的，上述方案仅仅提供了获取同一目标在不同离焦程度下的一对图像的方法，其无法满足深度学习技术在构建图像对数据集时对多样性的要求，而且，由于不同的成像单元的硬件无法完全一致，也导致各自成像存在一定程度的不对齐，这样得到的成对的图像也无法应用在深度学习技术中。
 技术问题
为了解决现有技术中无法采集多种光学参数的图像对或图像簇的缺陷，本发明提出图像构建方法，能够构建多种光学参数的图像对或图像簇或视频对，满足深度学习的数据集要求。
 技术解决方案
本发明采用的技术方案是，图像构建方法，步骤包括：
获得目标图像：将观察样品的信息通过分束装置分成多束信号光射出，所述分束装置射出的每一束光均通过不同的成像单元形成包含所述信息的目标图像；
构建成对图像或成簇图像：改变所述分束装置射出的不同光束各自成像时的光学参数，使不同成像单元同时形成不同光学参数的目标图像，得到同一观察样品的成对图像或成簇图像；
图像配准处理：将同一观察目标的多个图像进行对齐处理，得到每个观察目标的图像对或图像簇或视频对；
所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
优选的，所述分束装置射出两束光束，两束光束分别通过一个成像单元形成包含所述信息的图像，所述光学参数为光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种。
优选的，所述分束装置射出三束或多束光束，所述分束装置射出的每一束光均通过一个成像单元形成包含所述信息的图像，所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种或多种。
优选的，获得不同色彩的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，其中一个成像单元采用彩色相机，另一个成像单元采用单色相机，调节两个成像单元的焦面至重合，同时成像。
优选的，获得不同波段与光谱成分的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，在其中一个成像单元与分束装置之间插入一片或多片滤光片，并选择不同波段的光束进入该成像单元内成像，再调节两个成像单元均成像清晰后，同时成像。
优选的，获得不同亮度的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，调节其中一个成像单元相机的曝光时间和增益，再调节两个成像单元均成像清晰后，同时成像。
优选的，获得不同分辨率的成对图像或成簇图像时，至少让一对成像单元采用不同放大倍率的镜头，再调节两个成像单元的焦面至重合、调节景深至观察样品的照明范围内，同时成像。
优选的，获得不同离焦程度的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，调节两个成像单元的焦面至重合，其中一个成像单元固定不动，另一个成像单元沿着光束的光轴方向移动，且两个成像单元在不同位置处同时成像。
优选的，获得荧光和散射光成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，且调节照射所述观察样品的光束为单色光，并将分束装置替换为二向色镜，调节两个成像单元成像清晰后，同时成像。
优选的，获得偏振和非偏振成对图像或成簇图像时，让照射所述观察样品的光束先经过偏振片，且让分束装置射出的其中一束光束再经过一张偏振片，其中另一束光束不经过偏振片，两个所述光束分别由不同的两个成像单元同时成像。
优选的，所述将观察样品的信息通过分束装置分成多束信号光射出，具体为：采用照明器发出光束，且该光束照射于观察样本上后，再通过散射射向所述分束装置。
优选的，所述照明器位于所述观察容器的一侧，且所述照明器的光束沿着所述分束装置的入射光光轴发出。
优选的，所述照明器设置于所述观察容器的一侧，且所述照明器的光束与所述分束装置的入射光光轴夹角大于90°。
优选的，所述照明器设置于所述观察容器与分束装置之间，且所述照明器的光束与所述分束装置的入射光光轴夹角小于90°。
优选的，所述照明器布设于所述观察容器周边，且所述照明器发出的光束与所述分束装置的入射光光轴垂直。
优选的，构建成对图像或成簇图像时采用的装置包括：
样品观察装置，其将样品信息以光信号的形式发出；
分束装置，其接收含有所述光信号的光束并分成不同路径的多束射出；
成像装置，其至少包括两个成像单元，并分别接收所述分束装置射出的至少其中一束含有光信号的光束以同时成像，得到同一观察样品的成对图像或成簇图像；
调节装置，其设置于所述成像单元处，和/或，所述成像单元与所述分束装置之间，以调节所述成像单元的光学参数；
所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
优选的，所述调节装置包括以下结构中至少一个：
位置参数调节结构，设置于所述成像单元和/或所述分束装置处，以调节每个所述成像单元与所述分束装置之间的间距；
光学元件调节结构，设于所述成像装置和所述分束装置之间和/或照明器与所述样品观察装置之间，以调节光束的偏振状态和/或波段和/或亮度；
成像单元调节结构，所述成像单元包括相机和一个镜头/多个不同的镜头，所述成像单元调节机构用于将一个成像单元中的多个镜头的位置进行切换；
分束装置调节结构，所述分束装置为一个或多个立体分束器或分光板或二向色镜，所述分束装置调节结构用于调节分束装置的数量或类型。
优选的，所述位置参数调节结构包括沿所述分束装置的射出光束的光轴方向布置的调节轨道，每个所述成像单元滑动配合于所述调节轨道上，以调节与所述分束装置之间的距离。
优选的，所述光学元件调节结构包括安装部，所述安装部可拆连接/可更换连接有一片或多片偏振片、衰减片、滤光片中的至少一种。
优选的，所述图像配准处理包括：
取成对图像或成簇图像中的一张图像进行观察目标检测，得到与每个观察目标所对应的目标区域；
对所得到的每个目标区域进行清晰度判断，并将判断结果为模糊的目标区域丢弃；
将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，并将同一观察目标的多个目标区域裁剪、存储，得到图像对或图像簇或视频对。
优选的，将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：提取判断结果为清晰的目标区域坐标信息，根据该坐标信息变换得到成对图像或成簇图像中其余图像上含有的相同目标区域的坐标信息，根据各自的坐标信息，将同一观察目标的多个目标区域裁剪。
优选的，将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：利用结果为清晰的目标区域为模板，与成对图像或成簇图像中其它图像进行模板匹配，匹配程度最高的区域即为相同目标区域，裁剪存储得到图像对或图像簇或视频对。
优选的，提取判断结果为清晰的目标区域坐标信息指的是：提取判断结果为清晰的目标区域的四个顶点坐标，并对四个顶点的坐标进行变换，得到成对图像或成簇图像中其余图像中对应的目标区域的四个顶点的坐标。
优选的，所述对四个顶点的坐标进行变换为通过配准模型辅助获取，所述配准模型包括靶标，所述靶标提供多个可以被检测或识别的平面/空间特征，并对每个所述特征赋予坐标信息。
优选的，所述靶标为平面的网格，所述网格的每个交点均可以被检测或识别。
优选的，所述靶标为平板上凸出设置有多个均布的棱柱，所述棱柱的每个角点均可以被检测或识别。
 有益效果
与现有技术相比，本发明具有以下有益效果：
1、通过调节不同的光学参数或者同一个光学参数的不同数值，可以形成不同要求的多种成对图像或成簇图像，满足了深度学习的多样性和数量上的要求，为构建数据集提供了可行的基础方案；
2、设置调节装置将需要更换的光学参数均提前布置好，改变光学参数时通过调节装置中各个部件的配合可以直接实现，集成度高，能够满足各种参数调节的需求，减轻了人为调节的繁琐操作，而且精准度高，能够保障试验的及时进行；
3、通过图像配准处理，将采集得到的成对图像或成簇图像进行对齐，从而形成图像对或图像簇或视频对，能够符合深度学习中对于图像的高要求，进而有利于直接构建观察目标的数据集；
4、设置了多种光源形式，并且还提供了多种光源的照明方式，可以根据不同的观察目标和观察要求，形成多种多样的照明条件，有利于成像的清晰和完整。
 附图说明
下面结合实施例和附图对本发明进行详细说明，其中：
图1是本发明两路成像单元的系统示意图；
图2是本发明三路成像单元的系统示意图；
图3是本发明观察容器结构示意图；
图4是光源安装结构示意图；
图5是第一种光源安装方式示意图；
图6是第二种光源安装方式示意图；
图7是第三种光源安装方式示意图；
图8是第一种靶标的正视图；
图9是图8的侧视图；
图10是第二种靶标的正视图；
图11是高放大倍率和低放大倍率的浮游生物成像对比图；
图12是高分辨率和低分辨率的靶标成像对比图。
1、观察容器；11、观察部分；12、安装槽；13、隔板；2、光源；3、观察目标；4、立体分束器；5、立柱；51、连接杆；6、靶标；7、成像单元。
 本发明的最佳实施方式
图像构建方法，步骤包括：
获得目标图像：将观察样品的信息通过分束装置分成多束信号光射出，所述分束装置射出的每一束光均通过不同的成像单元7形成包含所述信息的目标图像；
构建成对图像或成簇图像：改变所述分束装置射出的不同光束各自成像时的光学参数，使不同成像单元7同时形成不同光学参数的目标图像，得到同一观察样品的成对图像或成簇图像；
图像配准处理：将同一观察目标3的多个图像进行对齐处理，得到每个观察目标3的图像对或图像簇或视频对；
所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
由于一般观察样品中均含有较多数量的观察目标3，因此在构建图像对或图像簇或视频对时，通过成像单元7得到的多个成对图像或成簇图像时，每一张图像上均含有多个观察目标3的成像，且每张成像之间也存在一定偏差，无法直接形成同一个观察目标3的图像对或图像簇或视频对。因此在构建某一个观察目标3的图像对和图像簇时，还是需要对成像的图像进行再处理，即经过上述的图像配准处理之后，才可以得到每个观察目标3所对应的图像对或图像簇或视频对。
而且由于构建成对图像或成簇图像时，可以通过改变分束装置射出的光束的不同光学参数，来实现不同光学参数下同一个观察目标3的不同图像对或图像簇或视频对，满足深度学习技术的多样性要求。
成像单元7进行成像时，根据分束装置的不同，可以形成不同数量的光路，即分束装置射出的光束数量与成像单元7的数量是对应的，基于此，实际成像时，具有以下情况：
分束装置射出两束光束，如图1所示，两束光束分别通过一个成像单元7形成包含所述信息的图像，所述光学参数为光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种；
分束装置射出三束或多束光束，如图2所示，所述分束装置射出的每一束光均通过一个成像单元7形成包含所述信息的图像，所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种或多种。
由于成像对或成像簇是针对同一个观察目标3在不同的光学参数下同时成像得到的，因此当分束装置只射出两个光束时，只形成两道光路，因此也只需要对应设置两个成像单元7即可，其两个成像单元7形成的图像即可构成一对图像。只有在分束装置射出三束或更多束光路时，才可以对应的设置多个成像单元7，再通过改变每个成像单元7的光学参数，即可以形成同一个观察目标3在不同光学参数下的成簇图像；其中不同光学参数可以是同一个光学参数如放大倍率的数值大小调节，也可以指的是多个光学参数的同时改变，如一路光束进行放大倍率的改变，另一路光束进行偏振状态的改变，第三路可以作为前两路的对照组，从而得到一组符合的图像簇。
具体地来说，不论是两路光路还是多路光路，在进行光学参数的调节时，至少可以采用如下的方法：
获得不同色彩的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，其中一个成像单元7采用彩色相机，另一个成像单元7采用单色相机，调节两个成像单元7的焦面至重合，同时成像；
获得不同波段与光谱成分的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，在其中一个成像单元7与分束装置之间插入一片或多片滤光片，并选择不同波段的光束进入该成像单元7内成像，再调节两个成像单元7均成像清晰后，同时成像；
获得不同亮度的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，调节其中一个成像单元7相机的曝光时间和增益，再调节两个成像单元7均成像清晰后，同时成像；
获得不同分辨率的成对图像或成簇图像时，至少让一对成像单元7采用不同放大倍率的镜头，再调节两个成像单元7的焦面至重合、调节景深至观察样品的照明范围内，同时成像；
获得不同离焦程度的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，调节两个成像单元7的焦面至重合，其中一个成像单元7固定不动，另一个成像单元7沿着光束的光轴方向移动，且两个成像单元7在不同位置处同时成像；
获得荧光和散射光成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，且调节照射所述观察样品的光束为单色光，并将分束装置替换为二向色镜，调节两个成像单元7成像清晰后，同时成像；
获得偏振和非偏振成对图像或成簇图像时，让照射所述观察样品的光束先经过偏振片，且让分束装置射出的其中一束光束再经过一张偏振片，其中另一束光束不经过偏振片，两个所述光束分别由不同的两个成像单元7同时成像。
根据实际的成像要求，调节的光学参数不仅仅局限于上述列出的调节方法，也可以参照前述的方法对其余光学参数进行适当的调节，本实施例中不再穷举。
当然，如果是在多路光路的成像系统中，由于有三个以上的成像单元7存在，因此可以如前述的对多种不同光学参数进行调节，以获得复合的图像簇，此时可以采用上述方法中的两种或多种进行光学参数的调节。
另外，在观察不同的样品和观察目标3时，对于照明的光源2有不同的要求，例如：
对于水下浮游生物观测而言，为了采集更高对比对，包含更多细节信息的浮游生物图像，对更大水体的浮游生物进行成像，同时还能对较小尺寸的浮游生物进行高信噪比的成像，则可以通过暗场成像的方式，采集不同离焦程度的数据对，以及不同光学分辨率的图像对，从而通过深度学习的方法扩展成像系统的景深，同时提升光学分辨率，从而满足高对比度、大空间范围和高空间分辨率的成像；
对于工业检测而言，往往需要在流水线上对较大的芯片尺寸进行高空间分辨率的成像，对于这类需求，则可以通过反射式的照明，构建明场不同离焦程度和不同光学分辨率的图像对或图像簇或视频对，再通过深度学习技术增强反射式明场成像系统的景深和光学分辨率；
对于生物组织荧光成像方面，需要能减少光毒性对细胞的伤害，同时以较低的成本实现高质量的荧光成像，则可以通过激光掠射式照明的方式激发荧光，同时利用反射光进行明场成像，构建明场显微图像和荧光图像的图像对，再结合深度学习技术，使得明场显微成像系统拥有荧光成像的能力。
因此，出于上述要求，将观察样品的信息通过分束装置分成多束信号光射出，具体为采用照明器发出光束，且该光束照射于观察样本上后，再通过散射射向所述分束装置，其中照明器的布置方式有如下几种：
照明器位于所述观察容器1的一侧，如图5所示，且所述照明器的光束沿着所述分束装置的入射光光轴发出，形成明场成像模式；
照明器设置于所述观察容器1的一侧，如图6所示，且所述照明器的光束与所述分束装置的入射光光轴夹角大于90°，形成暗场成像模式；
照明器设置于所述观察容器1与分束装置之间，如图7所示，且所述照明器的光束与所述分束装置的入射光光轴夹角小于90°，形成反射式明场成像模式；
照明器布设于所述观察容器1周边，如图1所示，且所述照明器发出的光束与所述分束装置的入射光光轴垂直。
经过如前阐述，照明器可以由多种照射方式将观察样品照亮，且根据前述的多种调节方法，在一对或多个成像单元7中形成所需要的成对图像或成簇图像，由于每张成像之间存在不对齐的问题，还需要进行图像配准处理，具体包括：
取成对图像或成簇图像中的一张图像进行观察目标3检测，根据检测得到的观察目标3数量和位置，得到与每个观察目标3所对应的目标区域；
对所得到的每个目标区域进行清晰度判断，并将判断结果为模糊的目标区域丢弃；
将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，并将同一观察目标3的多个目标区域裁剪、存储，得到图像对或图像簇或视频对。
其中，将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：提取判断结果为清晰的目标区域坐标信息，根据该坐标信息变换得到成对图像或成簇图像中其余图像上含有的相同目标区域的坐标信息，根据各自的坐标信息，将同一观察目标3的多个目标区域裁剪。
提取判断结果为清晰的目标区域坐标信息指的是：提取判断结果为清晰的目标区域的四个顶点坐标，并对四个顶点的坐标进行变换，得到成对图像或成簇图像中其余图像中对应的目标区域的四个顶点的坐标。
进行上述的坐标变换时，可以通过制作靶标6形成配准模型的方式，如图8-10所示，靶标6提供多个可以被检测或识别的平面/空间特征，并对每个所述特征赋予坐标信息。靶标6可以是平面的网格，即网格的每个交点的坐标为已知的；也可以是三维的，如在平板上凸出设置有多个均布的棱柱，所述棱柱的每个角点坐标均为已知，棱柱可以是三棱柱、四棱柱或多棱柱。
同时，还给出了另一种将目标区域对其的方法，其中将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：利用结果为清晰的目标区域为模板，与成对图像或成簇图像中其它图像进行模板匹配，匹配程度最高的区域即为相同目标区域，裁剪存储得到图像对或图像簇或视频对。
经过上述的步骤，可以得到同一观察目标3的图像对或图像簇或视频对，而且能够改变不同的光学参数，满足深度学习技术的样本多样性要求，同时也可以在数量上满足要求，为深度学习技术提供数据集构建的支撑。相比于现有技术中对于观察目标3的成像，本方案对于运动中的观察目标3也可以适用，适用范围更广，成像精度也更高。
需要说明的是，在获得观察目标的视频对时，调节成像单元的帧率至满足制作视频对的要求即可，其一般较图片拍摄的帧率要求更高，但是也应该根据视频对的清晰程度具体来定。
实施例1，一种构建图像对或图像簇或视频对的装置，如图1-2所示，其包括：
样品观察装置，其将样品信息以光信号的形式发出；
分束装置，其接收含有所述光信号的光束并分成不同路径的多束射出；
成像装置，其至少包括两个成像单元7，并分别接收所述分束装置射出的至少其中一束含有光信号的光束以同时成像，得到同一观察样品的成对图像或成簇图像；
调节装置，其设置于所述成像单元7处，和/或，所述成像单元7与所述分束装置之间，以调节所述成像单元7的光学参数；
所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
本实施例中观察样品为含有浮游生物的液体，观察目标3则是单个浮游生物，如图1-4所示，因此观察容器1为空心的长方体，且一侧向外水平延伸有突出的观察部分11，观察部分11也是空心设置并与观察容器1的长方体主体部分连通，观察部分11为长方体或圆柱体均可，其上安装有照明器，在本实施例中照明器为环形灯具，套设在观察部分11上。
必要的，观察容器1的上部通过管道连通有储存样本溶液的储存装置，以便及时补液，观察容器1的底部也通过管道连通至排液装置，该排液的管道设置有控制启闭的阀，以便将观察过后的样本溶液排出干净。在一个实施例中，排液的管道连接有气泵，气泵启动时不仅可以将观察容器1中已经观测过的样本溶液排出干净，还可以将观察容器1内部抽成负压，以实现样本溶液的自动进液，减轻人工操作。而且，通过设置气泵的自动启闭时间，匹配成像的时间，可以实现自动成像、自动更换观察样本的效果。
在一个实施例中，如图3所示，为了缩小样本溶液中浮游生物的活动范围，使其更加集中的在观察部分11中被观测成像，在观察容器1的中部位置还安装有水平的隔板13，隔板13的底部与观察部分11的底部高度平齐即可，且排液的管道穿过该隔板13连通至外部。
在一个实施例中，如图3所示，还需要安装靶标6，因此在水平延伸的观察部分11的上表面开有一道安装槽12，该安装槽12的开设方向与观察部分11的延伸方向垂直。
为了方便观察，整个观察容器1采用高透明的材质制成，如亚克力、石英玻璃等。
本实施例中的照明器为环形光源2，如图4所示，可以是LED光圈，其套设在观察部分11上，形成环形照明。安装时，设置有光源2调节架和控制器，控制器用于控制光源2的亮灭，光源2调节架用于固定光源2的位置，并根据需求对位置进行调节。
具体地，光源2调节架包括两根可以伸缩的立柱5，立柱5的底部通过螺栓与固定面进行安装连接，这样光源2可以调节高度，也可以调节与固定面的安装位置。进一步的，光源2调节架的立柱5上端还固定有水平的连接杆51，连接杆51与光源2通过螺栓实现可拆连接，连接杆51上供螺栓穿过的为水平的腰型孔，这样光源2与连接杆51之间的位置也可以进行改变，通过上述设置以实现光源2在多方位位置的调节。
需要强调的是，该环形光源2只是其中一个实施例，根据不同的成像方式的照明需求，也可以采用如前所述的不同的照明方式，相对应的，也需要改变不同的安装形式和结构，在此不再赘述。
分束装置可以采用立体分束器4或分光板或二向色镜，根据不同的光学参数的要求，可以在三种类型之间更换选择，在本实施例中，采用立体分束器4，并且通过改变分束器的数量，达到改变成像光路的数量。如需要对同一个观察目标3的三个光学参数进行调节时，可以设置两个沿光轴方向布置的立体分束器4，这样可以形成三路光路，对应设置三处成像单元7即可。
当然，如果要调节更多的光学参数，也可以增加数量更多的分束器，并同步增加对应数量的成像单元7，即可得到不同参数的成簇图像了。
关于成像单元7，本实施例中采用相机、远心镜头和镜头支架，远心镜头放置在镜头支架上。上述装置在安装时，观察容器1的观察部分11、立体分束器4和成像单元7都沿光路的轴心安装。
由于光学参数众多，不同的观测目标需要改变不同的成像光学参数，因此还设置了调节装置，其包括以下结构中至少一个：
位置参数调节结构，设置于所述成像单元7和/或所述分束装置处，以调节每个所述成像单元7与所述分束装置之间的间距；
光学元件调节结构，设于所述成像装置和所述分束装置之间和/或照明器与所述样品观察装置之间，以调节光束的偏振状态和/或波段和/或亮度；
成像单元调节结构，所述成像单元7包括相机和一个镜头/多个不同的镜头，所述成像单元调节机构用于将一个成像单元7中的多个镜头的位置进行切换；
分束装置调节结构，所述分束装置为一个或多个立体分束器4或分光板或二向色镜，所述分束装置调节结构用于调节分束装置的数量或类型。
在本实施例中，调节装置包括位置参数调节结构、光学元件调节结构、成像单元调节结构和分束装置调节结构。具体地，位置参数调节结构包括沿所述分束装置的射出光束的光轴方向布置的调节轨道，需要调节距离位置的成像单元7滑动配合于所述调节轨道上，以调节与所述分束装置之间的距离。
光学元件调节结构包括安装部，所述安装部可拆连接/可更换连接有一片或多片偏振片、衰减片、滤光片，偏振片、衰减片、滤光片均依次安装在每一个成像单元7和分数器之间的光路上。具体安装时，安装部包括安装架，安装架上转动连接有竖直的安装盘，每一个安装盘上可拆固定有多片参数或数量不同的偏振片或衰减片或滤光片，通过转动安装盘，即可切换同一个光路的不同光学参数。
成像单元调节结构与光学元件调节结构的安装部结构类似，多个不同放大倍率或增益或曝光时间的相机安装在一个可转动的安装盘上，通过转动该安装盘，可以实现同一个成像单元7中的不同参数的相机的切换。
分束装置的调节，其主要由气缸实现，即分束装置调节结构包括多根气缸，每根气缸的伸缩杆端部连接有安装板，安装板上定位有立体分束器4或分光板或二向色镜，在需要对分束装置进行调节时，由于多根气缸的端部驱动的分束装置不同，从而驱动不同的气缸，可以组合形成不同数量或种类的分束装置，如在某一个光路中只放置单个立体分束器4，也可以依次放置有两个立体分束器4，也可以用二向色镜替换立体分束器4。
实施例2，构建图像对或图像簇或视频对的装置，与实施例1的区别在于，实施例1为该装置的通用型，可以满足大部分光学参数调整，适用性较广泛，但是结构也较为复杂，本实施例中简化了其中的部分结构，是针对特地观察情况而制定的定制型。
本实施例以观察浮游生物为例，以取得样本溶液中浮游生物的图像对或图像簇或视频对为目的。如图1所示，该装置包括照明器、观察容器1、分束装置、成像装置和调节装置，照明器和观察容器1结构均与实施例1相同，分束装置为一个立体分束器4，成像装置含有两组远心镜头、相机和镜头支架。
调节装置（图中未示出）包括位置参数调节结构和成像单元调节结构，位置参数调节结构为铺设于每一个成像装置下方的调节轨道，成像单元调节结构包括一根固定的立柱5，立柱5上转动连接有一片固定盘，固定盘上则固定安装有多个放大倍率不同的镜头。
成像时，通过调节轨道可以调节两路镜头和相机中的一路的位置，改变其与分束器之间的间距，达到获得不同离焦程度的成对图像；也可以通过成像单元调节结构中固定盘的转动，切换不同放大倍率的镜头，从而获得不同放大倍率的成对图像。均可以为后续的图像对齐处理提供原始图像。
在另一个实施例中，还可以设置有两个依次摆放的立体分束器4，且第一级分束器的分光比为1:2，第二级分束器的分光比为1：1，这样将从观察目标3发出的光信号分成三路，对应的成像装置也包括有三组成像单元7。调节装置包括位置参数调节结构和成像单元调节结构，位置参数调节结构为铺设于每一个成像装置下方的调节轨道；成像单元调节结构包括一根固定的立柱5，立柱5上转动连接有一片固定盘，固定盘上则固定安装有多个放大倍率不同的镜头，通过转动固定盘实现不同放大倍率。这样可以在一次成像中得到三张成像，且三张成像可以组成两对图像对，更加方便。当然三路光路也可以同时用来调节放大倍率或同时用来调节距离，这样可以得到三张不同放大倍率的成簇图像，或者三张不同离焦程度的成簇图像。
通过暗场成像的方式，采集不同离焦程度的数据对，以及不同光学分辨率的图像对，从而通过深度学习的方法扩展成像系统的景深，同时提升光学分辨率，从而满足高对比度、大空间范围和高空间分辨率的成像。
在其他的实施例中，根据观察目标3的不同，其所要调节的光学参数也有所差异，因此可以根据实际要求调整分束装置的种类和数量，以及成像单元7的数量，并对应的改变调节装置的组成和安装位置。
实施例3，图像构建方法，其采用实施例1或实施例2中的装置，具体步骤为：
步骤一，将观察样品的溶液放入观察容器1中，安装固定好照明器后打开，观察样品的信息通过分束装置分成多束信号光射出，分束装置射出的每一束光均通过不同的成像单元7形成包含所述信息的目标图像。
光学参数为光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
步骤二，改变所述分束装置射出的不同光束各自成像时的光学参数，使不同成像单元7同时形成不同光学参数的目标图像，此时控制多路成像单元7中的相机同时成像，得到同一观察样品的成对图像或成簇图像。
为了实现多路成像单元7中的相机可以同时成像，本实施例给出了如下的控制方式：
如果两路或多路成像系统的相机工作在不同模式下，即部分相机工作在外部触发模式，则另一部分相机工作在自由运转模式，反之亦然。工作在自由运转模式下的相机输出曝光状态信号，触发工作在外部触发模式的不发相机及照明器，实现同时对静止或运动目标采集图像对；
如果多路成像系统的相机同时工作在外部触发模式，则可以设置为通过一个外部信号源产生触发信号，同时触发两台相机及照明器进行工作，采集图像对。该外部信号源可以是人为触发的快门按钮，也可以是自动识别的照明器开启信号。
不论是两路光路还是多路光路，在进行光学参数的调节时，至少可以采用如下的方法：
获得不同色彩的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，其中一个成像单元7采用彩色相机，另一个成像单元7采用单色相机，调节两个成像单元7的焦面至重合，同时成像；
获得不同波段与光谱成分的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，在其中一个成像单元7与分束装置之间插入一片或多片滤光片，并选择不同波段的光束进入该成像单元7内成像，再调节两个成像单元7均成像清晰后，同时成像；
获得不同亮度的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，调节其中一个成像单元7相机的曝光时间和增益，再调节两个成像单元7均成像清晰后，同时成像；
获得不同分辨率的成对图像或成簇图像时，至少让一对成像单元7采用不同放大倍率的镜头，再调节两个成像单元7的焦面至重合、调节景深至观察样品的照明范围内，同时成像；
获得不同离焦程度的成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，调节两个成像单元7的焦面至重合，其中一个成像单元7固定不动，另一个成像单元7沿着光束的光轴方向移动，且两个成像单元7在不同位置处同时成像；
获得荧光和散射光成对图像或成簇图像时，至少让一对成像单元7采用相同放大倍率的镜头，且调节照射所述观察样品的光束为单色光，并将分束装置替换为二向色镜，调节两个成像单元7成像清晰后，同时成像；
获得偏振和非偏振成对图像或成簇图像时，让照射所述观察样品的光束先经过偏振片，且让分束装置射出的其中一束光束再经过一张偏振片，其中另一束光束不经过偏振片，两个所述光束分别由不同的两个成像单元7同时成像。
由上述操作方法可知，如果采取装置为三路或更多路成像单元7，可以针对一个观察目标3对上述多个光学参数同时进行调节，一次性获得多张图像，构成成簇图像；如果采用的是两路成像单元7进行成像，则一次只能调节一个光学参数得到成对图像。
步骤三，将同一观察目标3的多个图像进行对齐处理，得到每个观察目标3的图像对或图像簇或视频对。由于多路成像部分部件的装配、加工等误差，以及成像部分放大倍率不一致等原因，使得多路成像部分采集的图像对或图像簇或视频对中对同一目标的像在图像空间中并不完全重合，所以对齐处理的具体方法如下：
首先对步骤二中相机成像得到的成对图像或者成簇图像进行基本的预处理，如色彩校正、减背景、CLAHE对比度增强等；
再利用现有技术中常用的目标检测算法将成对图像或者成簇图像中的一张图像进行观察目标3检测，得到多个观察目标3的目标区域（ROI），其中，如果成对图像或者成簇图像中含有放大倍率不同的图像，则应当选取高放大倍率的图像进行目标检测；
随后对观察目标3检测的图像进行清晰度判断，可以人为判断，也可以借助算法判断筛选，将判断结果的清晰的目标区域保留，并将判断结果为模糊的目标区域丢弃，如果一张图像中没有判断为清晰的目标区域，则直接返回上一步重新选择一张图像；
对于判断结果为清晰的目标区域，进行观察目标检测时提取该目标区域的四个顶点在原始图像中的坐标，再利用配准模型对四个顶点的坐标进行变换，得到成对图像或者成簇图像中其余图像上相同目标区域的四个顶点坐标；
此时一个观察目标3在所有图像中的成像的坐标均已经知道了，可以根据坐标进行裁剪、保存，即可得到该观察目标3的图像对或图像簇或视频对了。
步骤三中，为了实现目标区域的坐标变换，需要借助配准模型，本实施例中配准模型的构建主要依靠于靶标6来辅助获取，靶标6提供多个可以被检测或识别的平面/空间特征，并对每个所述特征赋予坐标信息，该靶标6的尺寸以在放置进观察容器1后能全部覆盖成像单元7的视场为原则，特征的分布方式和数量依据分辨率大小和视场大小而定。
根据照明方式的不同，靶标6可以采用不同的形式和结构，如实施例2中的暗场成像照明，此时靶标6为三维结构，其包括竖直的平板、平板上凸出设置有多个均布的四棱柱，所述棱柱的每个角点均可以被检测或识别；如果是明场照明的形式，则靶标6可以为二维的网格，网格的每个交点均可以被检测或识别。这样在靶标6的辅助下，可以在视场中构建多个按照一定空间规律分布的特征，通过特征提取、特征匹配和最小二乘法求解等一系列图像配准的标准过程，即可建立配准模型，可以在获取目标区域的四点坐标信息后，进行转换得到其余图像中同一个观察目标3的坐标信息。
在另一个实施例中，还给出了另一种图像对齐的处理方法，其具体处理方法为：
首先对步骤二中相机成像得到的成对图像或者成簇图像进行基本的预处理，如色彩校正、减背景、CLAHE对比度增强等；
对于放大倍率一致的成对图像或者成簇图像，选择其中一张图像进行目标检测并将目标区域（ROI）裁剪出来，对于放大倍率不一致的成对图像或者成簇图像，选择放大倍率最大的一张图像进行目标检测和裁剪；
随后对观察目标3检测的图像进行清晰度判断，可以人为判断，也可以借助算法判断筛选，将判断结果的清晰的目标区域保留，并将判断结果为模糊的目标区域丢弃，如果一张图像中没有判断为清晰的目标区域，则直接返回上一步重新选择一张图像；
对于放大倍率一致的成对图像或者成簇图像，利用清晰的目标区域作为模板，与成对图像或者成簇图像中其它图像进行模板匹配，并将图像中匹配程度最高的区域被裁剪下来，与上一步中的目标区域图像构成图像对或图像簇或视频对进行保存；对于放大倍率不一致的成对图像或者成簇图像，将清晰的目标区域进行下采样L（L=高放大倍率图像的放大倍率/低放大倍率图像的放大倍率）作为模板与成对图像或者成簇图像的其它图像进行模板匹配，图像中匹配程度最高的区域被裁剪下，上一步中的目标区域图像构成图像对或图像簇或视频对进行保存。
实施例4，图像构建方法，其采用实施例2中的装置，与实施例3的区别在于，设置有两个依次摆放的立体分束器4，且第一级分束器的分光比为1:2，第二级分束器的分光比为1：1，这样将从观察目标3发出的光信号分成三路，对应的成像装置也包括有三组成像单元7。
三路成像时，控制方法如下：
获取不同放大倍率的图像簇时，采用成像单元调节结构改变三路成像单元7的镜头，形成三个不同的放大倍率，同时成像后即可得到同一观察目标3的三张不同放大倍率的成簇图像；
获取不同离焦程度的成簇图像时，使用三个相同放大倍率的远心镜头，通过位置参数调节结构，改变其中一路成像单元7沿着光轴以固定步长向正方向运动以缩小其与分束器的距离，改变另一路成像单元7沿着光轴以固定步长向反方向运动以增大其与分束器的距离，第三路成像单元7的位置不变，并在不同的位置处同时成像，得到同一观察目标3的三张不同离焦程度的成簇图像；
获取不同亮度等级的成簇图像时，使用三个相同放大倍率的远心镜头，然后设置三路成像部分中的相机采用不同的曝光时间或增益，使得三路成像部分采集的图像中目标的亮度都不一致，分别对应过暗、正常和过曝三个等级，从而一次获取多种亮度等级的成簇图像；
获取不同波段的图像簇时，三路成像部分采用相同放大倍率的远心镜头，通过光学元件调节结构中的安装部在其中两路中插入不同波段的带通滤光片，剩下的一路不不加入光学元件，相机按照固定的帧率采集图像，获取同一目标包含不同波段成簇图像；
获取不同波段的荧光和散射光的成簇图像时，三路成像系统采用相同放大倍率的远心镜头，第一与第二分束器都采用二相色镜，但两个二相色镜的反射和透射波长不同，相机按照一定帧率采图，同时成像即可得到同一目标包含不同波段的荧光和散射光的成簇图像；
获取非偏振图像和不同偏振角度的成簇图像时，三路成像部分采用同一放大倍率的远心镜头，采用偏振光进行照明，并通过光学元件调节结构中的安装部在三路成像部分中的两路加入偏振片，调整两路成像部分中偏振片的角度，但两路之间的偏振片的夹角不同，同时采集图像，即可得到同一目标包含非偏振图像和不同偏振角度的成簇图像。
在另一个实施例中，针对三路成像，还可以对不同的光学参数进行组合，如：
三路成像系统中的两路采用相同倍率的远心镜头，最后一路采用不同倍率的远心镜头进行成像，先移动调节轨道使三路成像部分的焦面重合，然后通过调节轨道调节放大倍率相同的两路成像系统中的一路，并在每次移动后进行成像，从而同时获取同一目标包含不同放大倍率和不同离焦程度的成簇图像；
三路成像部分采用相同放大倍率的远心镜头，通过光学元件调节结构中的安装部在其中一路中插入带通滤光片，另一路插入偏振片，剩下的一路不不加入光学元件，相机按照固定的帧率采集图像，获取同一目标包含不同波段和不同偏振状态的成簇图像。
需要说明的是，在其他的实施例中，三路成像时，可以根据实际需求将前述的多种光学参数进行组合，形成多种不同的成像系统，而且在另外的实施例中，也可以是更多的成像光路，从而可以调节的光学参数也更多元，在此不再穷尽列举。
另外如图11所示，给出了申请人实验中得到的浮游生物在高放大倍率和低放大倍率下的成像对比图，图12中则给出了高分辨率和低分辨率的靶标6成像对比图。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。

Claims (26)

  1. 图像构建方法，其特征在于，步骤包括：
    获得目标图像：将观察样品的信息通过分束装置分成多束信号光射出，所述分束装置射出的每一束光均通过不同的成像单元形成包含所述信息的目标图像；
    构建成对图像或成簇图像：改变所述分束装置射出的不同光束各自成像时的光学参数，使不同成像单元同时形成不同光学参数的目标图像，得到同一观察样品的成对图像或成簇图像；
    图像配准处理：将同一观察目标的多个图像进行对齐处理，得到每个观察目标的图像对或图像簇或视频对；
    所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
  2. 根据权利要求1所述的方法，其特征在于，所述分束装置射出两束光束，两束光束分别通过一个成像单元形成包含所述信息的图像，所述光学参数为光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种。
  3. 根据权利要求1所述的方法，其特征在于，所述分束装置射出三束或多束光束，所述分束装置射出的每一束光均通过一个成像单元形成包含所述信息的图像，所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的一种或多种。
  4. 根据权利要求1-3任何一项所述的方法，其特征在于，获得不同色彩的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，其中一个成像单元采用彩色相机，另一个成像单元采用单色相机，调节两个成像单元的焦面至重合，同时成像。
  5. 根据权利要求1-3任何一项所述的方法，其特征在于，获得不同波段与光谱成分的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，在其中一个成像单元与分束装置之间插入一片或多片滤光片，并选择不同波段的光束进入该成像单元内成像，再调节两个成像单元均成像清晰后，同时成像。
  6. 根据权利要求1-3任何一项所述的方法，其特征在于，获得不同亮度的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，调节其中一个成像单元相机的曝光时间和增益，再调节两个成像单元均成像清晰后，同时成像。
  7. 根据权利要求1-3任何一项所述的方法，其特征在于，获得不同分辨率的成对图像或成簇图像时，至少让一对成像单元采用不同放大倍率的镜头，再调节两个成像单元的焦面至重合、调节景深至观察样品的照明范围内，同时成像。
  8. 根据权利要求1-3任何一项所述的方法，其特征在于，获得不同离焦程度的成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，调节两个成像单元的焦面至重合，其中一个成像单元固定不动，另一个成像单元沿着光束的光轴方向移动，且两个成像单元在不同位置处同时成像。
  9. 根据权利要求1-3任何一项所述的方法，其特征在于，获得荧光和散射光成对图像或成簇图像时，至少让一对成像单元采用相同放大倍率的镜头，且调节照射所述观察样品的光束为单色光，并将分束装置替换为二向色镜，调节两个成像单元成像清晰后，同时成像。
  10. 根据权利要求1-3任何一项所述的方法，其特征在于，获得偏振和非偏振成对图像或成簇图像时，让照射所述观察样品的光束先经过偏振片，且让分束装置射出的其中一束光束再经过一张偏振片，其中另一束光束不经过偏振片，两个所述光束分别由不同的两个成像单元同时成像。
  11. 根据权利要求1所述的方法，其特征在于，所述将观察样品的信息通过分束装置分成多束信号光射出，具体为：采用照明器发出光束，且该光束照射于观察样本上后，再通过散射射向所述分束装置。
  12. 根据权利要求11所述的方法，其特征在于，所述照明器位于所述观察容器的一侧，且所述照明器的光束沿着所述分束装置的入射光光轴发出。
  13. 根据权利要求11所述的方法，其特征在于，所述照明器设置于所述观察容器的一侧，且所述照明器的光束与所述分束装置的入射光光轴夹角大于90°。
  14. 根据权利要求11所述的方法，其特征在于，所述照明器设置于所述观察容器与分束装置之间，且所述照明器的光束与所述分束装置的入射光光轴夹角小于90°。
  15. 根据权利要求11所述的方法，其特征在于，所述照明器布设于所述观察容器周边，且所述照明器发出的光束与所述分束装置的入射光光轴垂直。
  16. 根据权利要求1所述的方法，其特征在于，构建成对图像或成簇图像时采用的装置包括：
    样品观察装置，其将样品信息以光信号的形式发出；
    分束装置，其接收含有所述光信号的光束并分成不同路径的多束射出；
    成像装置，其至少包括两个成像单元，并分别接收所述分束装置射出的至少其中一束含有光信号的光束以同时成像，得到同一观察样品的成对图像或成簇图像；
    调节装置，其设置于所述成像单元处，和/或，所述成像单元与所述分束装置之间，以调节所述成像单元的光学参数；
    所述光学参数至少包括光学分辨率、离焦距离、亮度、偏振态、色彩、光谱、荧光中的至少一种。
  17. 根据权利要求16所述的方法，其特征在于，所述调节装置包括以下结构中至少一个：
    位置参数调节结构，设置于所述成像单元和/或所述分束装置处，以调节每个所述成像单元与所述分束装置之间的间距；
    光学元件调节结构，设于所述成像装置和所述分束装置之间和/或照明器与所述样品观察装置之间，以调节光束的偏振状态和/或波段和/或亮度；
    成像单元调节结构，所述成像单元包括相机和一个镜头/多个不同的镜头，所述成像单元调节机构用于将一个成像单元中的多个镜头的位置进行切换；
    分束装置调节结构，所述分束装置为一个或多个立体分束器或分光板或二向色镜，所述分束装置调节结构用于调节分束装置的数量或类型。
  18. 根据权利要求17所述的方法，其特征在于，所述位置参数调节结构包括沿所述分束装置的射出光束的光轴方向布置的调节轨道，每个所述成像单元滑动配合于所述调节轨道上，以调节与所述分束装置之间的距离。
  19. 根据权利要求16所述的方法，其特征在于，所述光学元件调节结构包括安装部，所述安装部可拆连接/可更换连接有一片或多片偏振片、衰减片、滤光片中的至少一种。
  20. 根据权利要求1所述的方法，其特征在于，所述图像配准处理包括：
    取成对图像或成簇图像中的一张图像进行观察目标检测，得到与每个观察目标所对应的目标区域；
    对所得到的每个目标区域进行清晰度判断，并将判断结果为模糊的目标区域丢弃；
    将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，并将同一观察目标的多个目标区域裁剪、存储，得到图像对或图像簇或视频对。
  21. 根据权利要求20所述的方法，其特征在于，将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：提取判断结果为清晰的目标区域坐标信息，根据该坐标信息变换得到成对图像或成簇图像中其余图像上含有的相同目标区域的坐标信息，根据各自的坐标信息，将同一观察目标的多个目标区域裁剪。
  22. 根据权利要求20所述的方法，其特征在于，将判断结果为清晰的目标区域与成对图像或成簇图像中其余图像含有的目标区域进行对齐，指的是：利用结果为清晰的目标区域为模板，与成对图像或成簇图像中其它图像进行模板匹配，匹配程度最高的区域即为相同目标区域，裁剪存储得到图像对或图像簇或视频对。
  23. 根据权利要求21所述的方法，其特征在于，提取判断结果为清晰的目标区域坐标信息指的是：提取判断结果为清晰的目标区域的四个顶点坐标，并对四个顶点的坐标进行变换，得到成对图像或成簇图像中其余图像中对应的目标区域的四个顶点的坐标。
  24. 根据权利要求23所述的方法，其特征在于，所述对四个顶点的坐标进行变换为通过配准模型辅助获取，所述配准模型包括靶标，所述靶标提供多个可以被检测或识别的平面/空间特征。
  25. 根据权利要求24所述的方法，其特征在于，所述靶标为平面的网格，所述网格的每个交点均可以被检测或识别。
  26. 根据权利要求24所述的方法，其特征在于，所述靶标为平板上凸出设置有多个均布的棱柱，所述棱柱的每个角点均可以被检测或识别。
PCT/CN2021/138107 2021-09-26 2021-12-14 图像构建方法 Ceased WO2023045116A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111131952.3 2021-09-26
CN202111131952.3A CN115880207A (zh) 2021-09-26 2021-09-26 图像构建方法

Publications (1)

Publication Number Publication Date
WO2023045116A1 true WO2023045116A1 (zh) 2023-03-30

Family

ID=85719266

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138107 Ceased WO2023045116A1 (zh) 2021-09-26 2021-12-14 图像构建方法

Country Status (2)

Country Link
CN (1) CN115880207A (zh)
WO (1) WO2023045116A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116481981A (zh) * 2023-04-28 2023-07-25 苏州睿仟科技有限公司 一种细胞玻片成像系统、方法及计算机设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103959040A (zh) * 2011-08-21 2014-07-30 摩巴尔欧西提有限责任公司 在智能手机上附接光学相干断层成像系统
CN107340546A (zh) * 2017-07-24 2017-11-10 南京信息工程大学 一种水下探测分孔径双ccd实时偏振成像装置及方法
CN207529042U (zh) * 2017-12-15 2018-06-22 中山联合光电科技股份有限公司 一种不同方向光路同时成像的光学镜头
US20190223728A1 (en) * 2018-01-19 2019-07-25 The Regents Of The University Of California Optical coherence tomography for cancer screening and triage
CN110175971A (zh) * 2019-05-27 2019-08-27 大连海事大学 一种多光谱单像素成像的深度学习图像重构方法
CN110927945A (zh) * 2019-11-28 2020-03-27 清华大学 三维宽视场和高分辨层析成像方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103959040A (zh) * 2011-08-21 2014-07-30 摩巴尔欧西提有限责任公司 在智能手机上附接光学相干断层成像系统
CN107340546A (zh) * 2017-07-24 2017-11-10 南京信息工程大学 一种水下探测分孔径双ccd实时偏振成像装置及方法
CN207529042U (zh) * 2017-12-15 2018-06-22 中山联合光电科技股份有限公司 一种不同方向光路同时成像的光学镜头
US20190223728A1 (en) * 2018-01-19 2019-07-25 The Regents Of The University Of California Optical coherence tomography for cancer screening and triage
CN110175971A (zh) * 2019-05-27 2019-08-27 大连海事大学 一种多光谱单像素成像的深度学习图像重构方法
CN110927945A (zh) * 2019-11-28 2020-03-27 清华大学 三维宽视场和高分辨层析成像方法及装置

Also Published As

Publication number Publication date
CN115880207A (zh) 2023-03-31

Similar Documents

Publication Publication Date Title
JP5599314B2 (ja) 試料の検査のための方法および光学装置
US8154718B2 (en) Apparatus and method for inspecting micro-structured devices on a semiconductor substrate
US9013570B2 (en) Microscope and area determination method
JP2020509403A5 (zh)
JP7004808B2 (ja) スライドガラスの走査および処理のための対向縁部システム
JP6972188B2 (ja) 異なるサイズのスライド用の調整可能なスライドステージ
JP2021193459A (ja) 二重光学経路および単一撮像センサを使用した低解像度スライド撮像、スライドラベル撮像および高解像度スライド撮像
US20150185460A1 (en) Image forming method and image forming apparatus
CN109407297B (zh) 一种基于可编程led阵列照明的多模式光场显微成像方法
KR20200041983A (ko) 실시간 오토포커스 포커싱 알고리즘
CN101627899A (zh) 微循环双通道仿立体伪彩色显微视频成像系统
JP6940696B2 (ja) 二次元および三次元の固定式ｚ走査
CN102520507B (zh) 一种共聚焦显微成像的获取方法和装置
US11356593B2 (en) Methods and systems for single frame autofocusing based on color- multiplexed illumination
US8508589B2 (en) Imaging systems and associated methods thereof
JP6952891B2 (ja) ２×３および１×３スライド用のカルーセル
CN117330542A (zh) 高对比度各向同性高分辨荧光显微成像方法及成像系统
WO2018089948A1 (en) System and method for light sheet microscope and clearing for tracing
CN117011170B (zh) 一种用于双通道定量fret显微成像的重影矫正方法
CN115880207A (zh) 图像构建方法
WO2017170805A1 (ja) 多面画像取得システム、観察装置、観察方法、スクリーニング方法、および被写体の立体再構成方法
CN105403988A (zh) 基于lcd液晶面板的可编程孔径显微镜系统及其多模式成像方法
CN116519595B (zh) 一种用于晶圆宏观缺陷检测的显微装置
CN216351510U (zh) 大视场增强现实显微镜
CN208110156U (zh) 一种多功能式快速连续动态变焦台式显微镜

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21958227

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21958227

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21958227

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 24-09-2024)

122 Ep: pct application non-entry in european phase

Ref document number: 21958227

Country of ref document: EP

Kind code of ref document: A1