CN115880207A

CN115880207A - Image construction method

Info

Publication number: CN115880207A
Application number: CN202111131952.3A
Authority: CN
Inventors: 李剑平; 陈涛; 马文齐
Original assignee: Shenzhen Institute of Advanced Technology of CAS
Current assignee: Shenzhen Institute of Advanced Technology of CAS
Priority date: 2021-09-26
Filing date: 2021-09-26
Publication date: 2023-03-31
Also published as: WO2023045116A1

Abstract

The invention discloses an image construction method, which comprises the steps of obtaining a target image; changing optical parameters of different light beams emitted by the beam splitting device during respective imaging so that different imaging units simultaneously form target images with different optical parameters to obtain paired images or clustered images of the same observation sample; and aligning a plurality of images of the same observation target to obtain an image pair or an image cluster or a video pair of each observation target. According to the invention, through the arrangement, various paired images or clustered images with different requirements can be formed by adjusting different optical parameters or different values of the same optical parameter, and then image registration processing is carried out, so that the alignment effect is achieved, the requirements on diversity, high precision and quantity of deep learning are met, and a feasible basic scheme is provided for constructing a data set.

Description

Image construction method

Technical Field

The invention relates to the technical field of photoelectric imaging, in particular to an image construction method.

Background

The optical parameters of the optical imaging system include optical resolution, depth of field, dynamic range, color state, polarization state, and spectrum state, which play a crucial role in the practical application of the imaging system. However, due to the limitations of the optical imaging method itself or the working principle of the photosensitive device, it is difficult to simultaneously achieve the above optical parameters at a high level, such as:

the contradiction between the optical resolution and the depth of field and the field of view causes that the imaging system is difficult to image the target in a large space range with high resolution;

the imaging system needs to sacrifice certain optical resolution and sensitivity to carry out color imaging due to the characteristic that the photosensitive device can only respond to light intensity;

the imaging system sacrifices the diversity of the imaging target to realize the high dynamic range imaging of a certain type of target due to the complex physical attributes of the observation target.

In order to solve the above problems, chinese patent application publication No. CN113052766A discloses a multi-scale imaging device, a method and a system for splicing large-field high-resolution images, wherein a scheme is proposed for the imaging contradiction between the large field of view and the high resolution, and the effect of coordinating the two is achieved by means of splicing images.

However, the scheme is troublesome, has strong pertinence, can only be applied to the problems of large visual field and high resolution, and cannot meet the regulation requirements on other various parameters. Based on this, with the development of artificial intelligence technology based on data driving, more and more researches are put on a method of improving contradiction and limitation of an imaging system and enhancing imaging performance of the imaging system by constructing image pairs of different optical parameters to train corresponding models through data sets. For example, a model is trained by constructing low-resolution and high-resolution image pairs, so that the resolution of the low-resolution image is enhanced, the original field of view and depth of field are maintained, and high-resolution imaging in a large range is realized.

In order to ensure high performance and high robustness of the model to the actual application scene, the deep learning technology generally has high requirements on the data set, and not only needs a large number of image pairs in the data set, but also needs good diversity. However, the difficulty in constructing the data set of the image pair with different actual optical parameters is high, so that the application of the current deep learning technology in the aspect of enhancing the optical parameters of the imaging system is still low.

The applicant finds that after light is collected through a lens, the device divides the light into two paths through the light splitting plate for imaging, and obtains a pair of images with different defocusing degrees through the difference between the installation position of the camera with two paths of imaging and the distance of the light splitting plate.

However, it is obvious that the above solution only provides a method for acquiring a pair of images of the same target under different defocus degrees, which cannot meet the requirement of the deep learning technology on diversity when constructing an image set, and moreover, since hardware of different imaging units cannot be completely consistent, respective images are also misaligned to some extent, and thus the obtained paired images cannot be applied to the deep learning technology.

Disclosure of Invention

In order to overcome the defect that image pairs or image clusters of various optical parameters cannot be acquired in the prior art, the invention provides an image construction method which can construct the image pairs or the image clusters or the video pairs of various optical parameters and meet the requirement of a deep learning data set.

The invention adopts the technical scheme that the image construction method comprises the following steps:

obtaining a target image: dividing information of an observed sample into a plurality of signal lights through a beam splitting device and emitting the signal lights, wherein each light emitted by the beam splitting device forms a target image containing the information through a different imaging unit;

construction of paired or clustered images: changing optical parameters of different light beams emitted by the beam splitting device during respective imaging so that different imaging units simultaneously form target images with different optical parameters to obtain paired images or clustered images of the same observation sample;

image registration processing: aligning a plurality of images of the same observation target to obtain an image pair or an image cluster or a video pair of each observation target;

the optical parameters at least comprise at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

Preferably, the beam splitting device emits two light beams, the two light beams respectively pass through an imaging unit to form an image containing the information, and the optical parameter is one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

Preferably, the beam splitting device emits three or more beams, each beam emitted by the beam splitting device forms an image containing the information through an imaging unit, and the optical parameters at least include one or more of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

Preferably, when obtaining the pair images or cluster images of different colors, at least one pair of imaging units uses a lens with the same magnification, one of the imaging units uses a color camera, the other imaging unit uses a monochrome camera, and focal planes of the two imaging units are adjusted to be coincident and imaged simultaneously.

Preferably, when obtaining paired images or clustered images with different wavebands and spectral components, at least one pair of imaging units adopts lenses with the same magnification, one or more optical filters are inserted between one imaging unit and the beam splitting device, light beams with different wavebands are selected to enter the imaging units for imaging, and then the two imaging units are adjusted to simultaneously image after imaging is clear.

Preferably, when obtaining paired images or clustered images with different brightness, at least one pair of imaging units adopts lenses with the same magnification, the exposure time and the gain of one imaging unit camera are adjusted, and then the two imaging units are adjusted to simultaneously image after imaging clearly.

Preferably, when obtaining the pair images or cluster images with different resolutions, at least one pair of imaging units adopts lenses with different magnifications, and then the focal planes of the two imaging units are adjusted to be overlapped, and the depth of field is adjusted to be within the illumination range of the observation sample, and imaging is carried out simultaneously.

Preferably, when obtaining paired images or clustered images with different defocus degrees, at least one pair of imaging units adopts lenses with the same magnification, the focal planes of the two imaging units are adjusted to be overlapped, one imaging unit is fixed, the other imaging unit moves along the optical axis direction of the light beam, and the two imaging units simultaneously image at different positions.

Preferably, when obtaining paired images or clustered images of fluorescence and scattered light, at least one pair of imaging units adopts lenses with the same magnification, the light beam irradiating the observation sample is adjusted to be monochromatic light, the beam splitting device is replaced by a dichroic mirror, and the two imaging units are adjusted to image simultaneously after imaging is clear.

Preferably, when obtaining a polarized and unpolarized pair image or a clustered image, the light beam irradiating the observation sample passes through a polarizer, and one of the light beams emitted from the beam splitting device passes through another polarizer, and the other light beam does not pass through the polarizer, and the two light beams are imaged by two different imaging units at the same time.

Preferably, the splitting device splits the information of the observation sample into a plurality of signal lights to be emitted, specifically: the illuminator emits a light beam, and the light beam irradiates the observation sample and then emits to the beam splitting device through scattering.

Preferably, the illuminator is located on one side of the observation container, and a light beam of the illuminator is emitted along an incident light optical axis of the beam splitting device.

Preferably, the illuminator is arranged on one side of the observation container, and an included angle between a light beam of the illuminator and an optical axis of incident light of the beam splitting device is greater than 90 °.

Preferably, the illuminator is disposed between the observation container and the beam splitting device, and an included angle between a light beam of the illuminator and an optical axis of incident light of the beam splitting device is less than 90 °.

Preferably, the illuminator is arranged on the periphery of the observation container, and a light beam emitted by the illuminator is perpendicular to the optical axis of the incident light of the beam splitting device.

Preferably, the means employed in constructing the pair or cluster images comprises:

a sample observation device that emits sample information in the form of an optical signal;

a beam splitting device which receives the light beam containing the optical signal and splits the light beam into a plurality of beams with different paths to be emitted;

the imaging device at least comprises two imaging units and respectively receives at least one beam containing optical signals emitted by the beam splitting device to form images simultaneously so as to obtain paired images or clustered images of the same observation sample;

an adjusting device disposed at the imaging unit and/or between the imaging unit and the beam splitting device to adjust an optical parameter of the imaging unit;

Preferably, the adjusting device comprises at least one of the following structures:

the position parameter adjusting structure is arranged at the imaging unit and/or the beam splitting device so as to adjust the distance between each imaging unit and the beam splitting device;

the optical element adjusting structure is arranged between the imaging device and the beam splitting device and/or between the illuminator and the sample observation device so as to adjust the polarization state and/or the wave band and/or the brightness of the light beam;

an imaging unit adjustment structure, the imaging unit including a camera and a lens/lenses, the imaging unit adjustment structure for switching positions of the lenses in one imaging unit;

the beam splitting device adjusting structure is one or more stereoscopic beam splitters or beam splitting plates or dichroic mirrors and is used for adjusting the number or types of the beam splitting devices.

Preferably, the position parameter adjusting structure includes an adjusting rail arranged along an optical axis direction of an outgoing light beam of the beam splitting device, and each of the imaging units is slidably fitted on the adjusting rail to adjust a distance from the beam splitting device.

Preferably, the optical element adjusting structure includes a mounting portion, and the mounting portion is detachably/replaceably connected with at least one of one or more polarizing plates, an attenuating plate, and a filter.

Preferably, the image registration processing includes:

taking one image in the pair of images or the cluster images for observing target detection to obtain a target area corresponding to each observing target;

performing definition judgment on each obtained target area, and discarding the target area with a fuzzy judgment result;

and aligning the target area with clear judgment result with the target areas contained in the rest images in the paired images or the clustered images, and cutting and storing a plurality of target areas of the same observation target to obtain an image pair or an image cluster or a video pair.

Preferably, aligning the target region with which the determination result is clear with the target regions contained in the rest of the pair of images or the cluster image means: and extracting the coordinate information of the target area with clear judgment results, converting the coordinate information to obtain the coordinate information of the same target area contained in the rest images in the paired images or clustered images, and cutting a plurality of target areas of the same observation target according to the respective coordinate information.

Preferably, aligning the target region with which the determination result is clear with the target regions contained in the rest of the pair of images or the cluster image means: and (4) performing template matching with other images in the paired images or the clustered images by using the target area with a clear result as a template, wherein the area with the highest matching degree is the same target area, and cutting and storing to obtain an image pair or an image cluster or a video pair.

Preferably, the step of extracting the coordinate information of the target area with a clear judgment result includes: and extracting the coordinates of the four vertexes of the target area with clear judgment results, and transforming the coordinates of the four vertexes to obtain the coordinates of the four vertexes of the corresponding target area in the other images in the paired images or the clustered images.

Preferably, the transformation of the coordinates of the four vertices is assisted by a registration model comprising a target providing a plurality of planar/spatial features that can be detected or identified, and assigning coordinate information to each of the features.

Preferably, the target is a planar grid, each intersection of the grid being detectable or identifiable.

Preferably, the target is a prism with a plurality of uniform distribution protruding from a flat plate, and each corner point of the prism can be detected or identified.

Compared with the prior art, the invention has the following beneficial effects:

1. by adjusting different optical parameters or different values of the same optical parameter, various paired images or clustered images with different requirements can be formed, the requirements on diversity and quantity of deep learning are met, and a feasible basic scheme is provided for constructing a data set;

2. the optical parameters to be replaced are arranged in advance by arranging the adjusting device, the optical parameters can be directly changed by matching each component in the adjusting device, the integration level is high, the requirements of adjusting various parameters can be met, the complicated operation of manual adjustment is reduced, the accuracy is high, and the timely proceeding of a test can be guaranteed;

3. through image registration processing, aligning the acquired paired images or clustered images to form an image pair or an image cluster or a video pair, which can meet the high requirements on the images in deep learning and is further beneficial to directly constructing a data set of an observation target;

4. various light source forms are arranged, and illumination modes of various light sources are provided, so that various illumination conditions can be formed according to different observation targets and observation requirements, and the imaging is clear and complete.

Drawings

The invention is described in detail below with reference to examples and figures, in which:

FIG. 1 is a system schematic of a two-pass imaging unit of the present invention;

FIG. 2 is a system schematic of a three-way imaging unit of the present invention;

FIG. 3 is a schematic view of the viewing container of the present invention;

FIG. 4 is a schematic view of a light source mounting structure;

FIG. 5 is a schematic view of a first light source installation;

FIG. 6 is a schematic view of a second light source installation;

FIG. 7 is a schematic view of a third light source installation;

FIG. 8 is an elevation view of a first target;

FIG. 9 is a side view of FIG. 8;

FIG. 10 is an elevation view of a second target;

FIG. 11 is a comparison graph of plankton imaging at high magnification and low magnification;

figure 12 is a high resolution and low resolution target imaging contrast map.

1. Observing the container; 11. an observation portion; 12. mounting grooves; 13. a partition plate; 2. a light source; 3. observing the target; 4. a stereoscopic beam splitter; 5. a column; 51. a connecting rod; 6. a target; 7. an image forming unit.

Detailed Description

The image construction method comprises the following steps:

obtaining a target image: dividing information of an observed sample into a plurality of signal lights through a beam splitting device and emitting the signal lights, wherein each light emitted by the beam splitting device forms a target image containing the information through a different imaging unit 7;

construction of paired or clustered images: changing optical parameters of different light beams emitted by the beam splitting device during respective imaging so that different imaging units 7 simultaneously form target images with different optical parameters to obtain paired images or clustered images of the same observation sample;

image registration processing: aligning a plurality of images of the same observation target 3 to obtain an image pair or an image cluster or a video pair of each observation target 3;

the optical parameters comprise at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

Since a general observation sample contains a large number of observation targets 3, when a plurality of paired images or clustered images are obtained by the imaging unit 7 in constructing an image pair or an image cluster or a video pair, each image contains a plurality of images of the observation targets 3, and each image has a certain deviation, so that the image pair or the image cluster or the video pair of the same observation target 3 cannot be directly formed. Therefore, when constructing an image pair or an image cluster of a certain observation target 3, the imaged image still needs to be reprocessed, that is, after the image registration processing, the image pair or the image cluster or the video pair corresponding to each observation target 3 can be obtained.

And when the paired images or clustered images are constructed, different image pairs or image cluster or video pairs of the same observation target 3 under different optical parameters can be realized by changing different optical parameters of light beams emitted by the beam splitting device, so that the diversity requirement of the deep learning technology is met.

When the imaging unit 7 performs imaging, different numbers of optical paths can be formed according to the difference of the beam splitting device, that is, the number of beams emitted by the beam splitting device corresponds to the number of the imaging unit 7, and based on this, in actual imaging, the following situations exist:

the beam splitting device emits two beams, as shown in fig. 1, the two beams respectively pass through an imaging unit 7 to form an image containing the information, and the optical parameter is one of optical resolution, defocus distance, brightness, polarization state, color, spectrum and fluorescence;

the beam splitting means emits three or more beams, as shown in fig. 2, each of which forms an image containing the information by an imaging unit 7, and the optical parameters include at least one or more of optical resolution, defocus distance, brightness, polarization, color, spectrum, and fluorescence.

Because the imaging pair or the imaging cluster is obtained by imaging the same observation target 3 under different optical parameters at the same time, when the beam splitting device only emits two light beams, only two light paths are formed, and therefore, only two imaging units 7 need to be correspondingly arranged, and images formed by the two imaging units 7 can form a pair of images. Only when the beam splitting device emits three or more light paths, a plurality of imaging units 7 can be correspondingly arranged, and clustered images of the same observation target 3 under different optical parameters can be formed by changing the optical parameters of each imaging unit 7; the different optical parameters may be the same optical parameter, such as numerical adjustment of magnification, or simultaneous change of multiple optical parameters, such as change of magnification of one light beam, change of polarization state of the other light beam, and use of the third light beam as a control group of the previous two beams, thereby obtaining a group of coincident image clusters.

Specifically, when adjusting the optical parameters, whether two optical paths or multiple optical paths, at least the following method may be adopted:

when obtaining the paired images or clustered images with different colors, at least enabling a pair of imaging units 7 to adopt lenses with the same magnification, wherein one imaging unit 7 adopts a color camera, the other imaging unit 7 adopts a monochrome camera, adjusting the focal planes of the two imaging units 7 to be overlapped, and imaging at the same time;

when obtaining paired images or clustered images with different wave bands and spectral components, at least one pair of imaging units 7 adopts a lens with the same magnification, one or more optical filters are inserted between one imaging unit 7 and the beam splitting device, light beams with different wave bands are selected to enter the imaging unit 7 for imaging, and then the two imaging units 7 are adjusted to simultaneously image after imaging is clear;

when obtaining paired images or clustered images with different brightness, at least enabling a pair of imaging units 7 to adopt lenses with the same magnification, adjusting the exposure time and the gain of a camera of one imaging unit 7, and then adjusting two imaging units 7 to simultaneously image after imaging clearly;

when obtaining the paired images or clustered images with different resolutions, at least one pair of imaging units 7 adopts lenses with different magnifications, and then focal planes of the two imaging units 7 are adjusted to be overlapped and the scene depth is adjusted to be within the illumination range of the observation sample, and imaging is carried out simultaneously;

when obtaining the paired images or clustered images with different defocus degrees, at least one pair of imaging units 7 adopts lenses with the same magnification, the focal planes of the two imaging units 7 are adjusted to be overlapped, one imaging unit 7 is fixed, the other imaging unit 7 moves along the optical axis direction of the light beam, and the two imaging units 7 simultaneously image at different positions;

when paired images or clustered images of fluorescent light and scattered light are obtained, at least one pair of imaging units 7 adopt lenses with the same magnification, light beams irradiating the observation sample are adjusted to be monochromatic light, the beam splitting device is replaced by a dichroic mirror, and the two imaging units 7 are adjusted to image simultaneously after imaging is clear;

when polarized and unpolarized paired images or clustered images are obtained, the light beams irradiating the observation sample pass through the polarizing film, one of the light beams emitted by the beam splitting device passes through the polarizing film, the other light beam does not pass through the polarizing film, and the two light beams are imaged by two different imaging units 7 respectively.

The adjusted optical parameters are not limited to the above listed adjustment methods according to the actual imaging requirements, and the rest of the optical parameters can be adjusted appropriately by referring to the above methods, which is not exhaustive in the present embodiment.

Of course, if there are more than three imaging units 7 in the multi-path imaging system, the adjustment of the optical parameters can be performed by two or more of the above methods if the adjustment of the optical parameters can be performed as described above to obtain a composite image cluster.

In addition, when observing different samples and observation targets 3, different requirements are placed on the illuminating light source 2, such as:

for underwater plankton observation, in order to acquire plankton images which are higher in contrast and contain more detailed information, imaging planktons in a larger water body, and simultaneously performing high-signal-to-noise-ratio imaging on planktons in a smaller size, data pairs with different defocusing degrees and image pairs with different optical resolutions can be acquired in a dark field imaging mode, so that the depth of field of an imaging system is extended by a deep learning method, the optical resolution is improved, and imaging with high contrast, large spatial range and high spatial resolution is met;

for industrial detection, high spatial resolution imaging is often required to be performed on a larger chip size on a production line, and for such requirements, image pairs or image clusters or video pairs with different defocusing degrees and different optical resolutions in a bright field can be constructed through reflective illumination, and then the depth of field and the optical resolution of a reflective bright field imaging system are enhanced through a deep learning technology;

in the aspect of biological tissue fluorescence imaging, the damage of phototoxicity to cells needs to be reduced, high-quality fluorescence imaging is realized at low cost, fluorescence can be excited in a laser glancing illumination mode, meanwhile, bright field imaging is carried out by utilizing reflected light, an image pair of a bright field microscopic image and a fluorescence image is constructed, and then a deep learning technology is combined, so that a bright field microscopic imaging system has the capability of fluorescence imaging.

Therefore, in order to meet the above requirements, the information of the observed sample is divided into a plurality of signal lights by a beam splitting device and emitted, specifically, a light beam is emitted by an illuminator, and the light beam is irradiated on the observed sample and then emitted to the beam splitting device by scattering, wherein the illuminator is arranged in the following ways:

the illuminator is positioned at one side of the observation container 1, as shown in fig. 5, and the light beam of the illuminator is emitted along the optical axis of the incident light of the beam splitting device, so as to form a bright field imaging mode;

the illuminator is arranged at one side of the observation container 1, as shown in fig. 6, and an included angle between a light beam of the illuminator and an optical axis of incident light of the beam splitting device is greater than 90 degrees, so that a dark field imaging mode is formed;

the illuminator is arranged between the observation container 1 and the beam splitting device, as shown in fig. 7, and an included angle between a light beam of the illuminator and an optical axis of incident light of the beam splitting device is less than 90 degrees, so that a reflective bright field imaging mode is formed;

the illuminator is arranged at the periphery of the observation container 1, as shown in fig. 1, and the light beam emitted by the illuminator is perpendicular to the optical axis of the incident light of the beam splitting device.

As explained above, the illuminator can illuminate the observation sample by various illumination modes, and according to the aforementioned adjustment methods, the pair images or cluster images required are formed in one or more imaging units 7, and due to the misalignment problem between each imaging, the image registration process is also required, which specifically includes:

taking one image in the pair of images or the clustered images to detect the observation targets 3, and obtaining a target area corresponding to each observation target 3 according to the number and the positions of the observation targets 3 obtained by detection;

and aligning the target area with clear judgment result with the target areas contained in the rest images in the paired images or the clustered images, and cutting and storing a plurality of target areas of the same observation target 3 to obtain an image pair or an image cluster or a video pair.

Aligning the target area with the clear judgment result with the target areas contained in the rest images in the paired images or the clustered images, which means that: extracting the coordinate information of the target area with clear judgment result, converting the coordinate information to obtain the coordinate information of the same target area contained in the other images in the paired images or the clustered images, and cutting a plurality of target areas of the same observation target 3 according to the respective coordinate information.

The step of extracting the coordinate information of the target area with clear judgment result refers to: and extracting the coordinates of the four vertexes of the target area with clear judgment results, and transforming the coordinates of the four vertexes to obtain the coordinates of the four vertexes of the corresponding target area in the other images in the paired images or the clustered images.

In performing the above-described coordinate transformation, the target 6 may be constructed to form a registration model by creating the target 6, as shown in fig. 8-10, by providing a plurality of planar/spatial features that can be detected or identified, and assigning coordinate information to each of the features. The target 6 may be a grid of planes, i.e. the coordinates of each intersection of the grid are known; or three-dimensional, for example, a plurality of uniformly distributed prisms are convexly arranged on the flat plate, the coordinates of each corner point of the prisms are known, and the prisms can be triangular prisms, quadrangular prisms or polygonal prisms.

Meanwhile, another method for aligning the target area is provided, wherein the alignment of the target area with the clear judgment result with the target areas contained in the rest images in the paired images or the clustered images means that: and (4) performing template matching with other images in the paired images or the clustered images by using the target area with a clear result as a template, wherein the area with the highest matching degree is the same target area, and cutting and storing to obtain an image pair or an image cluster or a video pair.

Through the steps, the image pair or the image cluster or the video pair of the same observation target 3 can be obtained, different optical parameters can be changed, the requirement on sample diversity of the deep learning technology is met, the requirement on quantity can be met, and support for data set construction is provided for the deep learning technology. Compared with the imaging of the observation target 3 in the prior art, the scheme can also be applied to the observation target 3 in motion, the application range is wider, and the imaging precision is higher.

It should be noted that, when obtaining the video pair of the observation target, the frame rate of the imaging unit is adjusted to meet the requirement of making the video pair, which is generally higher than the frame rate requirement of picture taking, but should also be specifically determined according to the definition of the video pair.

Embodiment 1, an apparatus for constructing an image pair or image cluster or video pair, as shown in fig. 1-2, comprising:

the imaging device at least comprises two imaging units 7, and respectively receives at least one beam containing optical signals emitted by the beam splitting device to form images simultaneously so as to obtain paired images or clustered images of the same observation sample;

an adjusting device disposed at the imaging unit 7 and/or between the imaging unit 7 and the beam splitting device to adjust an optical parameter of the imaging unit 7;

In this embodiment, the observation sample is a liquid containing plankton, the observation target 3 is a single plankton, as shown in fig. 1-4, therefore, the observation container 1 is a hollow cuboid, and a protruding observation portion 11 extends horizontally outwards from one side, the observation portion 11 is also hollow and is communicated with the cuboid main body portion of the observation container 1, the observation portion 11 can be a cuboid or a cylinder, and is provided with an illuminator, in this embodiment, the illuminator is an annular lamp, and is sleeved on the observation portion 11.

The upper part of the observation container 1 is communicated with a storage device for storing sample solution through a pipeline so as to supplement the sample solution in time, the bottom of the observation container 1 is also communicated with a liquid drainage device through a pipeline, and the liquid drainage pipeline is provided with a valve for controlling opening and closing so as to completely drain the sample solution after observation. In one embodiment, the pipeline of flowing back is connected with the air pump, not only can will observe the sample solution that has surveyed in the container 1 clean the discharge when the air pump starts, can also be with observing that container 1 is inside to be pumped into the negative pressure to realize the automatic feed liquor of sample solution, alleviate manual operation. Moreover, the automatic opening and closing time of the air pump is set to match the imaging time, so that the effects of automatic imaging and automatic change of observation samples can be achieved.

In one embodiment, as shown in fig. 3, in order to reduce the range of activity of plankton in the sample solution and make it more concentrated to be observed and imaged in the observation part 11, a horizontal partition plate 13 is further installed at the middle position of the observation container 1, the bottom of the partition plate 13 is flush with the bottom of the observation part 11, and the pipe for discharging liquid is communicated to the outside through the partition plate 13.

In one embodiment, as shown in fig. 3, the target 6 is also installed, so that an installation groove 12 is formed on the upper surface of the observation portion 11 extending horizontally, and the installation groove 12 is opened in a direction perpendicular to the extending direction of the observation portion 11.

For convenient observation, the whole observation container 1 is made of a highly transparent material, such as acrylic, quartz glass, etc.

The illuminator in this embodiment is a ring-shaped light source 2, which may be an LED aperture, as shown in fig. 4, that fits over the viewing portion 11 to form a ring-shaped illumination. During the installation, be provided with light source 2 alignment jig and controller, the controller is used for controlling the bright of light source 2 and goes out, and light source 2 alignment jig is used for fixed light source 2's position to adjust the position according to the demand.

Specifically, the light source 2 adjusting frame comprises two telescopic columns 5, the bottoms of the columns 5 are connected with the fixing surface in an installing mode through bolts, and therefore the height of the light source 2 can be adjusted, and the installation position of the light source 2 and the fixing surface can also be adjusted. Further, the stand 5 upper end of light source 2 alignment jig still is fixed with horizontally connecting rod 51, and connecting rod 51 passes through the bolt with light source 2 and realizes releasable connection, and what supply the bolt to pass on the connecting rod 51 is horizontally waist type hole, and the position between light source 2 and the connecting rod 51 also can change like this, in order to realize the regulation of light source 2 in diversified position through above-mentioned setting.

It should be emphasized that the ring light source 2 is only one embodiment, and different illumination modes as described above may also be adopted according to illumination requirements of different imaging modes, and correspondingly, different installation forms and structures also need to be changed, which is not described herein again.

The beam splitting device can adopt a three-dimensional beam splitter 4 or a beam splitter plate or a dichroic mirror, and can be replaced and selected among three types according to different optical parameters. If three optical parameters of the same observation target 3 need to be adjusted, two stereoscopic beam splitters 4 arranged along the optical axis direction can be provided, so that three optical paths can be formed, and three imaging units 7 are correspondingly provided.

Of course, if more optical parameters are to be adjusted, a larger number of beam splitters may be added, and a corresponding number of imaging units 7 may be added simultaneously, so that clustered images with different parameters can be obtained.

As for the imaging unit 7, a camera, a telecentric lens and a lens holder are employed in the present embodiment, and the telecentric lens is placed on the lens holder. When the above-described apparatus is mounted, the observation portion 11 of the observation vessel 1, the stereoscopic beam splitter 4, and the imaging unit 7 are all mounted along the axis of the optical path.

Because of the numerous optical parameters, different observation targets need to change different imaging optical parameters, an adjusting device is also provided, which comprises at least one of the following structures:

a position parameter adjusting structure provided at the imaging unit 7 and/or the beam splitting device to adjust a distance between each of the imaging units 7 and the beam splitting device;

an imaging unit adjustment structure, the imaging unit 7 comprising a camera and a lens/lenses, the imaging unit adjustment structure being configured to switch positions of the lenses in the imaging unit 7;

the beam splitting device adjusting structure is one or more stereoscopic beam splitters 4 or beam splitting plates or dichroic mirrors, and is used for adjusting the number or types of the beam splitting devices.

In the present embodiment, the adjusting means includes a positional parameter adjusting structure, an optical element adjusting structure, an imaging unit adjusting structure, and a beam splitting means adjusting structure. Specifically, the position parameter adjusting structure includes an adjusting rail arranged in the optical axis direction of the outgoing light beam of the beam splitting apparatus, and the imaging unit 7 whose distance position needs to be adjusted is slidably fitted on the adjusting rail to adjust the distance from the beam splitting apparatus.

The optical element adjusting structure comprises an installation part, wherein the installation part is detachably/replaceably connected with one or more polaroids, attenuation sheets and optical filters, and the polaroids, the attenuation sheets and the optical filters are sequentially arranged on a light path between each imaging unit 7 and the fraction device. When the optical fiber polarizer optical filter is installed, the installation part comprises an installation frame, vertical installation disks are rotatably connected to the installation frame, a plurality of polarizers or attenuation sheets or optical filters with different parameters or quantities are detachably fixed to each installation disk, and different optical parameters of the same optical path can be switched by rotating the installation disks.

The imaging unit adjustment structure is similar to the mounting structure of the optical element adjustment structure, and a plurality of cameras with different magnifications or gains or exposure times are mounted on a rotatable mounting plate, and by rotating the mounting plate, switching of cameras with different parameters in the same imaging unit 7 can be realized.

The regulation of beam splitting device, it is mainly realized by the cylinder, beam splitting device regulation structure includes many cylinders promptly, the telescopic link end connection of every cylinder has the mounting panel, it has three-dimensional beam splitter 4 or beam splitter or dichroic mirror to fix a position on the mounting panel, when needing to adjust beam splitting device, because the tip driven beam splitting device of many cylinders is different, thereby drive different cylinders, can make up the beam splitting device that forms different quantity or kind, only place single three-dimensional beam splitter 4 if in a certain light path, also can place two three-dimensional beam splitter 4 in proper order, also can replace three-dimensional beam splitter 4 with the dichroic mirror.

Embodiment 2 is an apparatus for constructing an image pair, an image cluster, or a video pair, and is different from embodiment 1 in that embodiment 1 is a general-purpose type of the apparatus, and can satisfy most of optical parameter adjustments, and has a wide applicability, but the structure is also complicated, and a part of the structure is simplified in this embodiment, and is a customized type that is made for a specific observation situation.

In the present embodiment, the observation of plankton is taken as an example, and the purpose is to obtain an image pair or an image cluster or a video pair of plankton in the sample solution. As shown in fig. 1, the apparatus comprises an illuminator, an observation container 1, a beam splitting apparatus, an imaging apparatus and an adjusting apparatus, wherein the structure of the illuminator and the observation container 1 is the same as that of embodiment 1, the beam splitting apparatus is a stereo beam splitter 4, and the imaging apparatus comprises two sets of telecentric lenses, a camera and a lens holder.

Adjusting device (not shown in the figure) includes position parameter adjustment structure and imaging unit adjustment structure, and position parameter adjustment structure is for laying the regulation track of each imaging device below, and imaging unit adjustment structure includes a fixed stand 5, rotates on the stand 5 and is connected with a slice fixed disk, then fixed mounting has the different camera lenses of a plurality of magnifications on the fixed disk.

During imaging, the position of one of the two paths of lenses and the camera can be adjusted through the adjusting track, the distance between the lens and the beam splitter is changed, and paired images with different defocusing degrees are obtained; the lenses with different magnifications can be switched by rotating a fixed disk in the imaging unit adjusting structure, so that paired images with different magnifications can be obtained. The original image may be provided for subsequent image alignment processing.

In another embodiment, two stereoscopic beam splitters 4 arranged in sequence may be further provided, and the splitting ratio of the first-stage beam splitter is 1:1, thus dividing the optical signal emitted from the observation target 3 into three paths, the corresponding imaging device also includes three sets of imaging units 7. The adjusting device comprises a position parameter adjusting structure and an imaging unit adjusting structure, wherein the position parameter adjusting structure is an adjusting track laid below each imaging device; the imaging unit adjusting structure comprises a fixed upright post 5, a fixed disc is rotatably connected to the upright post 5, a plurality of lenses with different magnifications are fixedly mounted on the fixed disc, and different magnifications are realized by rotating the fixed disc. Therefore, three images can be obtained in one-time imaging, and the three images can form two pairs of images, so that the method is more convenient. Of course, the three optical paths can also be used for adjusting the magnification or the distance at the same time, so that three clustered images with different magnifications or three clustered images with different defocusing degrees can be obtained.

Data pairs with different defocusing degrees and image pairs with different optical resolutions are acquired in a dark field imaging mode, so that the depth of field of an imaging system is expanded by a deep learning method, the optical resolution is improved, and the imaging with high contrast, large spatial range and high spatial resolution is met.

In other embodiments, the optical parameters to be adjusted are different according to the different observation targets 3, so that the types and the number of the beam splitting devices and the number of the imaging units 7 can be adjusted according to actual requirements, and the composition and the installation position of the adjusting devices can be changed correspondingly.

Embodiment 3, an image construction method, which uses the apparatus in embodiment 1 or embodiment 2, includes the specific steps of:

firstly, putting a solution of an observation sample into an observation container 1, installing and fixing an illuminator, then opening the observation container, dividing information of the observation sample into a plurality of signal lights through a beam splitting device, and emitting the signal lights, wherein each light emitted by the beam splitting device forms a target image containing the information through different imaging units 7.

The optical parameter is at least one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

And step two, changing optical parameters of different light beams emitted by the beam splitting device during respective imaging to enable different imaging units 7 to simultaneously form target images with different optical parameters, and controlling cameras in the multi-path imaging units 7 to simultaneously image to obtain paired images or clustered images of the same observation sample.

In order to realize that the cameras in the multi-path imaging unit 7 can simultaneously image, the present embodiment provides the following control modes:

if the cameras of a two-way or multi-way imaging system are operated in different modes, i.e. some of the cameras are operated in an external trigger mode, another part of the cameras are operated in a free-running mode, and vice versa. The camera working in the free running mode outputs an exposure state signal, and triggers the camera and the illuminator working in the external trigger mode to acquire an image pair of a static or moving target;

if the cameras of the multi-path imaging system work in the external trigger mode at the same time, the trigger signal can be generated by one external signal source, and the two cameras and the illuminator are triggered to work at the same time to acquire an image pair. The external signal source may be a manually triggered shutter button or an automatically recognized lamp-on signal.

In the two-path or multi-path optical path, at least the following method can be adopted when adjusting the optical parameters:

when obtaining the paired images or clustered images with different colors, at least one pair of imaging units 7 adopts lenses with the same magnification, wherein one imaging unit 7 adopts a color camera, the other imaging unit 7 adopts a monochrome camera, the focal planes of the two imaging units 7 are adjusted to be coincident, and imaging is carried out simultaneously;

when obtaining paired images or clustered images with different brightness, at least one pair of imaging units 7 adopts lenses with the same magnification, the exposure time and the gain of the camera of one imaging unit 7 are adjusted, and then the two imaging units 7 are adjusted to simultaneously image after imaging clearly;

when paired images or clustered images of fluorescence and scattered light are obtained, at least one pair of imaging units 7 adopt lenses with the same magnification, light beams irradiating the observation sample are adjusted to be monochromatic light, the beam splitting device is replaced by a dichroic mirror, and the two imaging units 7 are adjusted to image simultaneously after imaging is clear;

According to the operation method, if the adopted device is three or more imaging units 7, the plurality of optical parameters can be simultaneously adjusted aiming at one observation target 3, and a plurality of images are obtained at one time to form a cluster image; if two imaging units 7 are used for imaging, only one optical parameter can be adjusted at a time to obtain a pair of images.

And step three, aligning a plurality of images of the same observation target 3 to obtain an image pair or an image cluster or a video pair of each observation target 3. Due to errors of assembly and processing of parts of the multi-path imaging part, inconsistent magnification of the imaging part and the like, images of the same target in an image space of an image pair or an image cluster or a video pair collected by the multi-path imaging part are not completely overlapped, so that a specific method of alignment processing comprises the following steps:

firstly, performing basic preprocessing, such as color correction, background subtraction, CLAHE contrast enhancement and the like, on paired images or clustered images obtained by camera imaging in the second step;

then, one image in the paired images or the clustered images is subjected to target 3 observation detection by using a commonly used target detection algorithm in the prior art to obtain a plurality of target Regions (ROI) of the targets 3 to be observed, wherein if the paired images or the clustered images contain images with different magnifications, the images with high magnifications are selected for target detection;

then, the image detected by the observation target 3 is judged in definition, either manually or by means of algorithm, a clear target area of a judgment result is reserved, the target area with the fuzzy judgment result is discarded, and if the target area with the fuzzy judgment result is not judged in one image, the previous step is directly returned to reselect the image;

for a target area with a clear judgment result, extracting coordinates of four vertexes of the target area in an original image when observing target detection is carried out, and transforming the coordinates of the four vertexes by using a registration model to obtain coordinates of the four vertexes of the same target area on other images in a pair image or a cluster image;

at this time, the imaged coordinates of an observed target 3 in all images are known, and the observed target 3 can be cut and stored according to the coordinates, so that an image pair or an image cluster or a video pair of the observed target 3 can be obtained.

In the third step, in order to realize the coordinate transformation of the target region, a registration model is required, in this embodiment, the registration model is mainly constructed by relying on the target 6 to assist in acquisition, the target 6 provides a plurality of planar/spatial features which can be detected or identified, and coordinate information is given to each of the features, the size of the target 6 is based on the principle that the target can completely cover the field of view of the imaging unit 7 after being placed in the observation container 1, and the distribution mode and the number of the features are determined according to the resolution and the field of view.

According to different illumination modes, the target 6 can adopt different forms and structures, such as dark field imaging illumination in embodiment 2, at this time, the target 6 is a three-dimensional structure and comprises a vertical flat plate, a plurality of uniformly distributed quadrangular prisms are convexly arranged on the flat plate, and each corner point of each prism can be detected or identified; in the case of bright field illumination, the target 6 may be a two-dimensional grid, each intersection of which may be detected or identified. Thus, with the aid of the target 6, a plurality of features distributed according to a certain spatial rule can be constructed in a view field, a registration model can be established through a series of standard processes of image registration such as feature extraction, feature matching and least square solution, and coordinate information of the same observation target 3 in other images can be obtained through conversion after four-point coordinate information of a target area is obtained.

In another embodiment, another image alignment processing method is further provided, and the specific processing method is as follows:

selecting one image of the pair images or cluster images with the same magnification to perform target detection and cut out a target area (ROI), and selecting one image with the maximum magnification to perform target detection and cutting out the pair images or cluster images with the different magnification;

for the paired images or clustered images with the same magnification, a clear target area is used as a template to be matched with other images in the paired images or clustered images, the area with the highest matching degree in the images is cut off, and the images or image clusters or video pairs are stored with the target area images in the previous step; and for the pair images or the cluster images with inconsistent magnifications, performing template matching on a clear target area by taking downsampling L (L = magnification of a high magnification image/magnification of a low magnification image) as a template with other images of the pair images or the cluster images, cutting an area with the highest matching degree in the images, and storing the target area images in the last step to form an image pair or an image cluster or a video pair.

Embodiment 4, an image construction method, which uses the apparatus in embodiment 2, is different from embodiment 3 in that two stereoscopic beam splitters 4 are sequentially arranged, a splitting ratio of a first-stage beam splitter is 1:1, thus dividing the optical signal emitted from the observation target 3 into three paths, the corresponding imaging device also includes three sets of imaging units 7.

In the case of three-way imaging, the control method is as follows:

when image clusters with different magnifications are obtained, an imaging unit adjusting structure is adopted to change lenses of the three imaging units 7 to form three different magnifications, and three clustered images with different magnifications of the same observation target 3 can be obtained after simultaneous imaging;

when acquiring clustered images with different defocus degrees, three telecentric lenses with the same magnification are used, through a position parameter adjusting structure, one path of imaging unit 7 moves towards the positive direction along the optical axis in a fixed step length to reduce the distance between the imaging unit and the beam splitter, the other path of imaging unit 7 moves towards the negative direction along the optical axis in a fixed step length to increase the distance between the imaging unit and the beam splitter, the position of the third path of imaging unit 7 is unchanged, and imaging is performed at different positions at the same time, so that three clustered images with different defocus degrees of the same observation target 3 are obtained;

when acquiring clustered images with different brightness levels, using three telecentric lenses with the same magnification, and setting cameras in the three imaging parts to adopt different exposure time or gain, so that the brightness of targets in the images acquired by the three imaging parts is inconsistent and respectively corresponds to three levels of over-dark, normal and over-exposure, thereby acquiring clustered images with various brightness levels at one time;

when acquiring image clusters of different wave bands, the three imaging parts adopt telecentric lenses with the same magnification, band-pass filters of different wave bands are inserted into two paths of the three imaging parts through a mounting part in an optical element adjusting structure, the rest paths are not added with optical elements, and a camera acquires images according to a fixed frame rate to acquire clustered images of the same target containing different wave bands;

when cluster images of fluorescence and scattered light of different wave bands are obtained, the three-way imaging system adopts telecentric lenses with the same magnification, the first beam splitter and the second beam splitter both adopt dichroic mirrors, but the reflection wavelengths and the transmission wavelengths of the two dichroic mirrors are different, a camera acquires images according to a certain frame rate, and the cluster images of the fluorescence and the scattered light of the same target containing different wave bands can be obtained through simultaneous imaging;

when acquiring non-polarized images and clustered images with different polarization angles, the three imaging parts adopt telecentric lenses with the same magnification, polarized light is used for illumination, polarizing plates are added into the two imaging parts through the installation part in the optical element adjusting structure, the angles of the polarizing plates in the two imaging parts are adjusted, the included angles of the polarizing plates between the two imaging parts are different, and the images are acquired at the same time, so that the clustered images with the same target containing the non-polarized images and the clustered images with different polarization angles can be obtained.

In another embodiment, different optical parameters may also be combined for three-way imaging, such as:

two paths of the three imaging systems adopt telecentric lenses with the same magnification, the last path adopts telecentric lenses with different magnifications for imaging, the adjusting track is moved to ensure that focal planes of the three imaging parts are superposed, then one path of the two imaging systems with the same magnification is adjusted through the adjusting track, and imaging is carried out after each movement, so that clustered images of the same target containing different magnifications and different defocusing degrees are obtained simultaneously;

the three imaging parts adopt telecentric lenses with the same magnification, a band-pass filter is inserted into one path of the three imaging parts through a mounting part in an optical element adjusting structure, a polaroid is inserted into the other path of the three imaging parts, the optical element is not added into the rest path of the three imaging parts, and a camera acquires images according to a fixed frame rate to acquire clustered images of the same target, wherein the clustered images comprise different wave bands and different polarization states.

It should be noted that, in other embodiments, during three-way imaging, the aforementioned multiple optical parameters may be combined according to actual requirements to form multiple different imaging systems, and in other embodiments, there may be more imaging optical paths, so that there are also more adjustable optical parameters, which are not exhaustive here.

In addition, as shown in fig. 11, the images of plankton obtained in the applicant's experiment under high magnification and low magnification are shown, and in fig. 12, the images of target 6 with high resolution and low resolution are shown.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims

1. The image construction method is characterized by comprising the following steps:

2. The method of claim 1, wherein the beam splitting device emits two beams, and the two beams respectively pass through an imaging unit to form an image containing the information, and the optical parameter is one of optical resolution, defocus distance, brightness, polarization state, color, spectrum, and fluorescence.

3. The method of claim 1, wherein the beam splitting device emits three or more beams, each beam emitted by the beam splitting device is passed through an imaging unit to form an image containing the information, and the optical parameters include at least one or more of optical resolution, defocus distance, brightness, polarization, color, spectrum, and fluorescence.

4. A method according to any one of claims 1 to 3, wherein when obtaining pairs or clusters of images of different colours, at least one pair of imaging units is made to use lenses of the same magnification, one imaging unit being a colour camera and the other imaging unit being a monochrome camera, and the focal planes of the two imaging units are adjusted to coincide and imaged simultaneously.

5. The method according to any one of claims 1 to 3, wherein when obtaining paired images or clustered images of different wavelength bands and spectral components, at least one pair of imaging units is made to use lenses of the same magnification, one or more filters are inserted between one of the imaging units and the beam splitting device, and light beams of different wavelength bands are selected to enter the imaging unit for imaging, and then both imaging units are adjusted to image simultaneously after imaging is clear.

6. The method according to any one of claims 1-3, wherein when obtaining paired images or clustered images with different brightness, at least one pair of imaging units is made to use lenses with the same magnification, the exposure time and gain of one of the cameras of the imaging units are adjusted, and then both imaging units are adjusted to image simultaneously after imaging clearly.

7. A method according to any of claims 1-3, characterized in that when obtaining pairs of images or clusters of images of different resolutions, at least one pair of imaging units is made to use lenses of different magnifications, and then the focal planes of the two imaging units are adjusted to coincide and the depth of field is adjusted to be within the illumination range of the observed sample, and imaging is performed simultaneously.

8. A method according to any one of claims 1-3, characterized in that when obtaining paired images or clustered images with different defocus degrees, at least one pair of imaging units is made to use lenses of the same magnification to adjust the focal planes of the two imaging units to be coincident, wherein one imaging unit is fixed, the other imaging unit is moved along the optical axis direction of the light beam, and the two imaging units are imaged at different positions simultaneously.

9. The method according to any one of claims 1 to 3, wherein when paired images or clustered images of the fluorescent light and the scattered light are obtained, at least one pair of imaging units is made to use lenses with the same magnification, the light beam irradiating the observation sample is adjusted to be monochromatic light, the beam splitting device is replaced by a dichroic mirror, and after the two imaging units are adjusted to be clear, the images are formed simultaneously.

10. A method according to any one of claims 1 to 3, wherein polarized and unpolarized paired images or clustered images are obtained by passing the beams illuminating the sample under observation through a polariser and passing one of the beams from the beam splitting means through a polariser and the other beam through a polariser, the two beams being imaged simultaneously by different ones of the imaging units.

11. The method according to claim 1, wherein the information of the observed sample is divided into a plurality of signal lights by a beam splitting device, and the signal lights are emitted from the beam splitting device, specifically: the illuminator emits a light beam, and the light beam irradiates the observation sample and then emits to the beam splitting device through scattering.

12. The method of claim 11, wherein the illuminator is located on a side of the viewing container, and wherein a light beam of the illuminator is emitted along an optical axis of incident light of the beam splitting device.

13. The method of claim 11, wherein the illuminator is disposed on a side of the viewing container, and a light beam of the illuminator is angled more than 90 ° from an optical axis of incident light of the beam splitting device.

14. The method of claim 11, wherein the illuminator is disposed between the viewing container and a beam splitting device, and wherein a beam of the illuminator is angled less than 90 ° from an optical axis of incident light from the beam splitting device.

15. The method of claim 11, wherein the illuminator is disposed around the viewing container, and wherein the light beam emitted from the illuminator is perpendicular to the optical axis of the incident light from the beam splitting device.

16. The method of claim 1, wherein the means employed in constructing the pair or cluster of images comprises:

a sample observation device which emits sample information in the form of an optical signal;

17. The method of claim 16, wherein the adjustment device comprises at least one of:

the position parameter adjusting structure is arranged at the imaging units and/or the beam splitting devices so as to adjust the distance between each imaging unit and the beam splitting devices;

an imaging unit adjustment structure, the imaging unit including a camera and a lens/lenses, the imaging unit adjustment structure for switching positions of the lenses in the imaging unit;

18. The method according to claim 17, wherein the position parameter adjusting structure includes an adjusting rail arranged along an optical axis direction of an outgoing light beam of the beam splitting device, and each of the imaging units is slidably fitted on the adjusting rail to adjust a distance from the beam splitting device.

19. The method of claim 16, wherein the optical element adjustment structure comprises a mount to which at least one of one or more polarizers, attenuators, and filters are removably/replaceably attached.

20. The method of claim 1, wherein the image registration process comprises:

21. The method according to claim 20, wherein aligning the target region judged to be clear with the target regions contained in the remaining images in the pair of images or the cluster of images comprises: and extracting the coordinate information of the target area with clear judgment result, converting the coordinate information to obtain the coordinate information of the same target area contained in the rest images in the paired images or clustered images, and cutting a plurality of target areas of the same observation target according to the respective coordinate information.

22. The method according to claim 20, wherein aligning the target region judged to be clear with the target regions contained in the remaining images in the pair of images or the cluster of images comprises: and (4) performing template matching with other images in the paired images or the clustered images by using the target area with a clear result as a template, wherein the area with the highest matching degree is the same target area, and cutting and storing to obtain an image pair or an image cluster or a video pair.

23. The method according to claim 21, wherein extracting the target area coordinate information whose determination result is clear is: and extracting four vertex coordinates of the target area with clear judgment results, and transforming the coordinates of the four vertices to obtain the coordinates of the four vertices of the corresponding target area in other images in the paired images or the clustered images.

24. The method of claim 23, wherein the transforming the coordinates of the four vertices is assisted by a registration model comprising a target providing a plurality of planar/spatial features that can be detected or identified.

25. The method of claim 24, wherein the target is a grid of planes, and wherein each intersection of the grid is detected or identified.

26. The method of claim 24, wherein the target is a flat plate with a plurality of uniformly spaced prisms protruding therefrom, each corner point of the prisms being detectable or identifiable.