CN107194874B - Super-resolution imaging system and method based on bias image stabilization - Google Patents
Super-resolution imaging system and method based on bias image stabilization Download PDFInfo
- Publication number
- CN107194874B CN107194874B CN201710383337.9A CN201710383337A CN107194874B CN 107194874 B CN107194874 B CN 107194874B CN 201710383337 A CN201710383337 A CN 201710383337A CN 107194874 B CN107194874 B CN 107194874B
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- super
- imaging
- resolution imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
Abstract
The invention discloses a super-resolution imaging system and a method based on bias image stabilization, and the super-resolution imaging system based on bias image stabilization comprises: the system comprises a system imaging lens group, a spectroscope, a high frame frequency detector, a high resolution imaging detector and a data processor; the parallel light is converged by an imaging lens of the system and then is divided into two paths on the surface of the spectroscope, and one path is imaged by a high-frame-frequency detector; the data processor calculates a displacement compensation value according to the image data; and the driving device drives the imaging image surface of the high-resolution imaging detector to move the displacement compensation value.
Description
Technical Field
The invention relates to the field of space-based imaging, in particular to a super-resolution imaging system and method based on bias image stabilization.
Background
With the increasing demand of the imaging resolution of the earth, besides the technical development on the design of an optical system, a super-resolution imaging technology is considered, and the overall imaging resolution of the earth of the satellite is further improved on the design of the system.
The super-resolution reconstruction technology utilizes redundancy, similarity and a certain priori knowledge among low-resolution image (LR) sequences to carry out data fusion so as to reconstruct an image (HR) with higher spatial resolution, and meanwhile, image noise, blurring and the like are removed. The reconstruction-based method is to perform inverse degradation on a degraded image by using a certain mathematical theory according to an image degradation model to obtain a super-resolution reconstructed image, such as a frequency domain de-aliasing super-resolution reconstruction method, a set theory-based super-resolution reconstruction method, a probability statistics theory-based super-resolution reconstruction method and the like.
Since Tsai and Huang in 1984 firstly proposed a frequency domain aliasing solution method for HR reconstruction by using a multi-frame LR image with sub-pixel displacement, many scholars at home and abroad studied the super-resolution reconstruction method and obtained fruitful results. All these methods hope to obtain higher resolution images from a series of multi-frame undersampled low-resolution visual images of the same scene, and a resampling imaging model with no noise, no degradation, and identical size and radiance is adopted, and a corresponding spectrum aliasing formula is used to provide an aliasing-removing method. On the basis, a model under the condition of mixed noise and fuzzy degradation of an image is researched, and two antialiasing methods of weighted iteration and regularization iteration are provided. These methods utilize the information provided by the relative displacement between images and the different point spread functions, and do not utilize the information provided by the difference in image resolution. Therefore, in further development, people adopt new methods for using statistical prior knowledge, such as using a MAP estimator based on gaussian smooth prior knowledge to enhance the satellite remote sensing image.
At present, a typical system applying the digital super-resolution imaging technology to a satellite-borne platform mainly includes an SPOT satellite and a SkySat satellite, wherein the SPOT satellite mainly adopts a sub-pixel sampling mode to improve resolution, and the SkySat satellite mainly adopts an optical-digital processing combined design technology and a sub-pixel offset multi-frame exposure sampling mode to improve resolution. The super-resolution imaging technology of the satellite in China is not reported.
Disclosure of Invention
The invention solves the problems that the prior image stabilization imaging system has complex structure, is not beneficial to use and has low imaging resolution; in order to solve the problems, the invention provides a super-resolution imaging system and a super-resolution imaging method based on bias image stabilization.
The invention provides a super-resolution imaging system based on bias image stabilization, which comprises: the system comprises a system imaging lens group, a spectroscope, a high frame frequency detector, a high resolution imaging detector and a data processor; the parallel light is converged by an imaging lens of the system and then is divided into two paths on the surface of the spectroscope, and one path is imaged by a high-frame-frequency detector; the data processor calculates a displacement compensation value according to the image data; and the driving device drives the imaging image surface of the high-resolution imaging detector to move the displacement compensation value.
The invention provides a super-resolution imaging method based on bias image stabilization, which comprises the following steps:
step one, imaging by a high frame frequency detector, and calculating detection displacement data by a data processor according to image data to form a displacement compensation value;
driving an imaging image surface of the high-resolution imaging detector to move by a driving device, wherein the movement amount is the displacement compensation value;
thirdly, the high-resolution imaging detector acquires the current frame and stores the current frame into a cache, and judges whether the number of images in the cache reaches a preset amount, if so, the images in the cache are subjected to super-resolution processing; if not, entering the step four;
driving an imaging image surface of the high-resolution imaging detector to move, wherein the moving amount is the sum of the displacement compensation value and the fixed offset; the high-resolution imaging detector acquires the current frame image again;
judging whether the number of the images in the cache reaches a preset amount again, and if so, distinguishing the images in the cache; if not, the step returns to the step four.
Further, the resolving process includes:
step 5.1, carrying out image registration on the images in the buffer to form low-score sequence images;
and 5.2, performing super-resolution reconstruction on the low-resolution sequence image.
Further, the step 5.1 comprises: taking any frame in the image sequence in the buffer as an original reference image, and taking other frames as images to be registered; establishing a digital pyramid to perform down-sampling processing on the data to obtain a new image sequence; and sequentially performing feature extraction, feature matching, similarity calculation and geometric correction on the new image sequence to complete registration of the image to be registered and form a low-score sequence image.
Further, the step 5.2 comprises: processing the low-resolution sequence image by adopting an MAP (MAP) and Bayesian super-resolution reconstruction algorithm, and judging whether a convergence condition is reached; if so, performing deblurring and denoising processing to obtain a high-resolution image; if the convergence condition is not reached, steps 5.1 and 5.2 are repeated, or only step 5.2 is repeated until the convergence condition is reached.
Further, the fixed bias is a 0.5 pel bias.
The advantages of the invention include:
the bias image stabilization based super-resolution imaging system and method provided by the invention comprehensively consider the characteristics of a system camera and the constraint of a super-resolution method, and adopt the fixed bias of the sub-pixels in image stabilization, so that the sub-pixel sampling is realized in the process of acquiring a sequence image by the system, a stable sub-pixel change is acquired, and the motion blur caused by the satellite attitude change can be effectively reduced, thereby providing a high-quality image for the next super-resolution image reconstruction.
The invention fully utilizes the image stabilizing device, adds the fixed sub-pixel offset to obtain the dislocation sampling image, and carries out super-resolution processing on the image by the image processing technology at the later stage, thereby improving the image resolution and saving the system resources.
Drawings
FIG. 1 is a schematic structural diagram of a super-resolution imaging system based on offset image stabilization according to an embodiment of the present invention;
fig. 2 is a schematic image registration flow diagram of a bias image stabilization-based super-resolution imaging method according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a super-resolution imaging method based on bias image stabilization according to an embodiment of the present invention.
Detailed Description
The spirit and substance of the present invention will be further described below with reference to the accompanying drawings and examples.
As shown in fig. 1, a super-resolution imaging system based on offset image stabilization provided by an embodiment of the present invention includes: the system comprises a system imaging group mirror 01, a spectroscope 02, a high frame frequency detector 03, a high resolution imaging detector 04 and a data processor; the parallel light is converged by a system imaging lens group 01 and then is divided into two paths on the surface of a spectroscope 02, and one path is imaged by a high-frame-frequency detector 03; the data processor calculates a displacement compensation value according to the image data; and the driving device drives the detection surface of the high-resolution imaging detector 04 to move the displacement compensation value.
The driving device can adopt piezoelectric ceramics, and in the moving process, 0.5 pixel offset can be added in cooperation with exposure time to obtain a sub-sampling image.
The super-resolution imaging method based on the bias image stabilization provided by the embodiment of the invention comprises the following steps:
firstly, sending an instruction of image stabilization bias hyper-resolution imaging from the ground, and resetting an on-satellite cache; and a data processor acquires images to calculate detection displacement data, wherein the calculation of the detection displacement data refers to the calculation of the position offset of the imaging system by the data processor according to the generated images, and the technology is well known to those skilled in the art and is not described in detail herein.
Secondly, the on-satellite data processor forms a displacement compensation value according to the current displacement data, feeds the displacement compensation value back to the high-resolution imaging detector, and drives the high-resolution imaging image plane to move through the piezoelectric motor; the displacement compensation data is used for compensating optical blurring and motion blurring caused by the position offset of the imaging system; methods for calculating the displacement compensation data are well known to those skilled in the art and will not be described in detail herein.
Thirdly, the high-resolution detector acquires a current frame image f1 and stores the current frame image in a cache; judging the number of images in the cache, if the number of images is less than a preset amount, adding the displacement into a fixed offset by the on-board data processor, and driving the high-resolution imaging image plane to move by the piezoelectric motor;
the fixed offset is an empirical value for further compensating motion blur, and is 0.5 pixel in the embodiment, and in other embodiments, the fixed offset is selected by a person skilled in the art according to actual conditions, and can also be selected as 0; the preset amount is 4 in this embodiment, and in other embodiments, the selection is performed by a person skilled in the art according to actual conditions.
Fourthly, the high-resolution imaging detector acquires the current frame image f2 again; judging the number of the images in the cache, and returning to the step two if the number of the images is less than a preset amount;
when the number of the images is equal to the preset amount, the on-board processor performs super-resolution processing on the images in the cache to obtain high-definition super-resolution images, stores the images into storage equipment of the camera, and empties the images in the cache;
and step six, if the task stopping instruction is not sent, returning to the step two to carry out the imaging and super-resolution processes again.
The super-resolution processing includes:
step 5.1, carrying out image registration on the images in the buffer to form low-score sequence images;
image registration refers to the process of best matching two or more images of the same region. Image registration is a necessary premise to solve the image super-resolution of the system. Based on the characteristics of the satellite-borne camera and the requirements of satellite-borne remote sensing image data processing, the image registration scheme needs to have the characteristics of high processing speed, high registration precision and strong algorithm robustness.
As shown in fig. 2, image registration includes: after the original image sequence is processed by the image pyramid, the operations of feature extraction, feature matching, similarity calculation and the like are sequentially carried out, and after the original image sequence is mapped to the original image position, geometric correction is carried out to obtain a registration image. The processing such as image pyramid, integral image and BOX filtering greatly improves the defect of low processing speed of the traditional method and ensures the registration precision.
In the embodiment, an integral image and an increasing box filter template are adopted to obtain a response image, so that a point of interest (namely a brighter or darker position in the image) can be quickly found out and used as an image feature for feature matching. After the features are extracted from the image to be registered, feature matching is carried out according to the descriptors of the image feature points, the descriptors are associated with the feature points, the integral image is released, an iteration module can be added, global matching consistency is further checked, and unmatched feature pairs are eliminated. The similarity calculation is an index for estimating the degree of similarity between images to determine whether the extracted feature points can be matched. The geometric correction means that an optimal geometric transformation model which can fit the change between two images is selected according to the geometric distortion between the image to be registered and the reference image, the optimal transformation parameter between the two images is obtained according to an optimization strategy, and the image to be registered is processed through coordinate transformation and interpolation to obtain a corrected image, wherein the image is a low-resolution sequence image. The image registration can be achieved by a person skilled in the art from said description and will not be described in detail here.
And 5.2, performing super-resolution reconstruction on the low-resolution sequence image, performing reconstruction-based super-resolution reconstruction on the image, and optimizing a degradation model of the imaging system by using the process so as to enable the degradation model to better accord with the actual imaging situation. The super-resolution reconstruction includes: and restoring and reconstructing the low-resolution sequence image obtained by registering the images by utilizing an MAP and Bayesian super-resolution reconstruction algorithm in sequence. The registration and reconstruction steps are iterated independently or jointly until the convergence condition is satisfied. And then, a reconstructed high-resolution image is obtained through post-processing of image denoising and deblurring. MAP, bayesian super-resolution reconstruction algorithms are well known to those skilled in the art and will not be described in detail herein.
The method provided by the invention is already used for models, and the resolution of the formed image is improved.
Although the present invention has been described with reference to the preferred embodiments, it is not intended to limit the present invention, and those skilled in the art can make variations and modifications of the present invention without departing from the spirit and scope of the present invention by using the methods and technical contents disclosed above.
Claims (5)
1. A super-resolution imaging method of a super-resolution imaging system based on bias image stabilization is characterized in that the super-resolution imaging system comprises: the system comprises a system imaging lens group, a spectroscope, a high frame frequency detector, a high resolution imaging detector and a data processor; the parallel light is converged by an imaging lens of the system and then is divided into two paths on the surface of the spectroscope, and one path is imaged by a high-frame-frequency detector; the data processor calculates a displacement compensation value according to the image data; the driving device drives the imaging image surface of the high-resolution imaging detector to move the displacement compensation value; the method comprises the following steps:
step one, imaging by a high frame frequency detector, and calculating detection displacement data by a data processor according to image data to form a displacement compensation value;
driving an imaging image surface of the high-resolution imaging detector to move by a driving device, wherein the movement amount is the displacement compensation value; the displacement compensation value is used for compensating optical blurring and motion blurring caused by the position offset of the imaging system;
thirdly, the high-resolution imaging detector acquires the current frame and stores the current frame into a cache, and judges whether the number of images in the cache reaches a preset amount, if so, the images in the cache are subjected to super-resolution processing; if not, entering the step four;
driving an imaging image surface of the high-resolution imaging detector to move, wherein the moving amount is the sum of the displacement compensation value and the fixed offset; the high-resolution imaging detector acquires the current frame image again;
step five, judging whether the number of the images in the cache reaches a preset amount again, and if so, performing super-resolution processing on the images in the cache; and if not, returning to the step four or the step two.
2. The super-resolution imaging method according to claim 1, characterized in that the super-resolution process comprises:
step 5.1, carrying out image registration on the images in the buffer to form low-score sequence images;
and 5.2, performing super-resolution reconstruction on the low-resolution sequence image.
3. Super-resolution imaging method according to claim 2, characterized in that said step 5.1 comprises: taking any frame in the image sequence in the buffer as an original reference image, and taking other frames as images to be registered; establishing a digital pyramid to perform down-sampling processing on the data to obtain a new image sequence; and sequentially performing feature extraction, feature matching, similarity calculation and geometric correction on the new image sequence to complete registration of the image to be registered and form a low-score sequence image.
4. Super-resolution imaging method according to claim 2, characterized in that said step 5.2 comprises: processing the low-resolution sequence image by adopting an MAP (MAP) and Bayesian super-resolution reconstruction algorithm, and judging whether a convergence condition is reached; if so, performing deblurring and denoising processing to obtain a high-resolution image; if the convergence condition is not reached, steps 5.1 and 5.2 are repeated, or only step 5.2 is repeated until the convergence condition is reached.
5. The super resolution imaging method according to claim 2, characterized in that the fixed bias is 0.5 pel bias.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710383337.9A CN107194874B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging system and method based on bias image stabilization |
| CN202010960660.XA CN112017122B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710383337.9A CN107194874B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging system and method based on bias image stabilization |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010960660.XA Division CN112017122B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107194874A CN107194874A (en) | 2017-09-22 |
| CN107194874B true CN107194874B (en) | 2020-10-02 |
Family
ID=59874948
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710383337.9A Active CN107194874B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging system and method based on bias image stabilization |
| CN202010960660.XA Active CN112017122B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging method |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010960660.XA Active CN112017122B (en) | 2017-05-26 | 2017-05-26 | Super-resolution imaging method |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN107194874B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108090869B (en) * | 2017-11-27 | 2021-07-09 | 中国空间技术研究院 | On-satellite super-resolution reconstruction method based on area array CMOS optical camera |
| CN108897003B (en) * | 2018-05-03 | 2021-05-04 | 北京理工大学 | Dual-mode control phased array laser radar system and method |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7602997B2 (en) * | 2005-01-19 | 2009-10-13 | The United States Of America As Represented By The Secretary Of The Army | Method of super-resolving images |
| CN101980291B (en) * | 2010-11-03 | 2012-01-18 | 天津大学 | Random micro-displacement-based super-resolution image reconstruction method |
| US8750647B2 (en) * | 2011-02-03 | 2014-06-10 | Massachusetts Institute Of Technology | Kinetic super-resolution imaging |
| CN103047972B (en) * | 2012-12-25 | 2014-11-19 | 中国科学院长春光学精密机械与物理研究所 | Method for improving resolution of geostationary orbit satellite platform area array gazing camera |
| CN104410789A (en) * | 2014-12-08 | 2015-03-11 | 西安电子科技大学 | Staring super-resolution imaging device and method |
| CN104394309B (en) * | 2014-12-16 | 2017-05-10 | 中国科学院国家天文台 | Super-resolution imaging system and method with stable images |
| CN105069748B (en) * | 2015-07-16 | 2017-11-10 | 哈尔滨工业大学 | A kind of method that high-definition picture is obtained based on microsatellite object space scanning technique |
| CN106525238B (en) * | 2016-10-27 | 2018-08-03 | 中国科学院光电研究院 | A kind of satellite-borne multispectral imaging system design method based on super-resolution rebuilding |
-
2017
- 2017-05-26 CN CN201710383337.9A patent/CN107194874B/en active Active
- 2017-05-26 CN CN202010960660.XA patent/CN112017122B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112017122B (en) | 2024-02-09 |
| CN112017122A (en) | 2020-12-01 |
| CN107194874A (en) | 2017-09-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107025632B (en) | Image super-resolution reconstruction method and system | |
| CN109102462B (en) | Video super-resolution reconstruction method based on deep learning | |
| CN110599400B (en) | EPI-based light field image super-resolution method | |
| US11720999B2 (en) | Method, device and non-transitory computer-readable storage medium for increasing the resolution and dynamic range of a sequence of respective top view images of a same terrestrial location | |
| CN110211169B (en) | Reconstruction method of narrow baseline parallax based on multi-scale super-pixel and phase correlation | |
| CN111986134B (en) | Remote sensing imaging method and device for area-array camera | |
| CN112837245A (en) | Dynamic scene deblurring method based on multi-mode fusion | |
| CN107194874B (en) | Super-resolution imaging system and method based on bias image stabilization | |
| CN104574338B (en) | Remote sensing image super-resolution reconstruction method based on multi-angle linear array CCD sensors | |
| CN106920213B (en) | Method and system for acquiring high-resolution image | |
| JP5566199B2 (en) | Image processing apparatus, control method therefor, and program | |
| CN112184549B (en) | Super-resolution image reconstruction method based on space-time transformation technology | |
| CN109615584B (en) | SAR image sequence MAP super-resolution reconstruction method based on homography constraint | |
| Al Ismaeil et al. | Dynamic super resolution of depth sequences with non-rigid motions | |
| Vrigkas et al. | On the improvement of image registration for high accuracy super-resolution | |
| Zhang et al. | Video superresolution reconstruction using iterative back projection with critical-point filters based image matching | |
| CN113506212B (en) | Improved hyperspectral image super-resolution reconstruction method based on POCS | |
| Mohan | Adaptive super-resolution image reconstruction with lorentzian error norm | |
| JP2013069012A (en) | Multi-lens imaging apparatus | |
| Seema et al. | Multi-frame image super-resolution by interpolation and iterative backward projection | |
| CN108665412B (en) | Method for performing multi-frame image super-resolution reconstruction by using natural image priori knowledge | |
| Zhu et al. | A new method for superresolution image reconstruction based on surveying adjustment | |
| Seyid et al. | Hardware implementation of real-time multiple frame super-resolution | |
| Patel Shreyas | Novel Iterative Back Projection Approach | |
| Zhang et al. | A robust video super-resolution algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220516 Address after: 201306 building C, No. 888, Huanhu West 2nd Road, Lingang New Area, Pudong New Area, Shanghai Patentee after: Shanghai Zhongkechen New Satellite Technology Co.,Ltd. Address before: 201210 No. 99 Haike Road, Pudong New Area, Shanghai (building 4-5) Patentee before: SHANGHAI ENGINEERING CENTER FOR MICROSATELLITES |