CN104394309B - Super-resolution imaging system and method with stable images - Google Patents
Super-resolution imaging system and method with stable images Download PDFInfo
- Publication number
- CN104394309B CN104394309B CN201410779707.7A CN201410779707A CN104394309B CN 104394309 B CN104394309 B CN 104394309B CN 201410779707 A CN201410779707 A CN 201410779707A CN 104394309 B CN104394309 B CN 104394309B
- Authority
- CN
- China
- Prior art keywords
- image
- super
- undersampled
- resolution
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Studio Devices (AREA)
Abstract
The invention provides a super-resolution imaging system and method with stable images. The method comprises the following steps: firstly, placing an imaging optical element group, an oscillating mirror, a spectroscope, an imaging lens group and an imaging camera sequentially in the imaging optical path direction of a propagation path of an observed aiming beam; connecting an input end of a super-resolution image synthesis unit, which is used for synthesizing a plurality of undersampled images transmitted by a receiving and imaging camera into super-resolution images, with an output end of the imaging camera; placing the imaging optical element group, the oscillating mirror, the spectroscope, a detecting lens group and a detecting camera sequentially in a detecting optical path direction of the propagation path of the observed aiming beam; connecting an output end of the detecting camera with an input end of a computing control unit; connecting an input end of an oscillating mirror driver with an output end of the computing control unit; connecting an output end of the oscillating mirror driver, which is used for amplifying the voltage analog quantity sent by the computing control unit into the driving voltage analog quantity of the oscillating mirror and outputting the driving voltage analog quantity to the oscillating mirror, with a driving end of the oscillating mirror.
Description
Technical field
The invention belongs to high-resolution imaging field, specifically, it is related to a kind of super-resolution imaging of image stabilization
System and method for.
Background technology
Photoelectronic imaging is the important technical that the mankind obtain the outer multispectral image information of visible red, is widely used in state
People's economy and the every field of national defense construction.The resolution ratio of photo electric imaging system refer generally to image after system it is distinguishable most
Little details, is the important technology index for characterizing system detectivity, affects the factor of the index and mainly has the aperture of imaging len
Geometric parameter of size and imageing sensor etc..Limited by physical condition, the resolution ratio that people obtain image is relatively low, and these are low
Image in different resolution can not well meet and be actually needed, and especially in fields such as space flight, remote sensing, military surveillances, then people wish
Prestige is super to rebuild using the information that the pixel intrinsic displacement existed between several low-resolution images contains original high-resolution image
Image in different resolution.
Current Super-resolution Reconstruction technology includes two kinds of implementations of micro scanning and sub-pixed mapping technology.Micro scanning is considered as
It is an over-sampling process, image formed by optical system is both horizontally and vertically being carried out sub- picture by it using micro scanning device
Plain displacement, obtains multiframe sub- sampling image, and Digital Image Processor by these images according to by way of obtaining image and
Order carries out cross reconstruction into a two field picture, and so as to reach final realization high-resolution purpose is put forward.Two grades of micro scanning technologies are
Synthesize a panel height image in different resolution with four width undersampled images, as shown in figure 1, wherein 1,2,3,4 represent micro- swept by two grades respectively
The serial number of the four width undersampled images that order is obtained is retouched, the high resolution graphics of final synthesis seems this four width undersampled image picture
The result of element fusion, contained pixel increased four times.Sub-pixed mapping technology is by sampled imaging systems routine focal plane
On row's detector lines array make some detector lines arrays of Heterogeneous Permutation on linear array direction and vertical linear array direction into,
On linear array column direction by dislocation, in vertical linear array side upwardly through improving or not improving the means of time sampling frequency improving
A kind of method of object space resolution ratio.Two kinds of implementations of all things considered micro scanning and sub-pixed mapping technology are all that camera is continuous
The mutual sequence for collecting less than a series of discrete image of a pixel distance, close by numerical software integration technology by after
Become a high-resolution image.
In the application of the fields such as actual space flight, remote sensing, military surveillance, using the sub-pixed mapping of the multiple detectors of array arrangement
Technology realizes that super-resolution rebuilding increased the financial cost of project, and detector generates with the presence in detector gap and owes to adopt
Sample noise, makes the image that system is obtained can not completely reproduce observed scene.Super-resolution is realized using micro scanning technology
Reconstruction can effectively reduce undersampling method, improve systemic resolution, improve image quality.Realize micro scanning technology
Form is varied, and by the difference of type of drive Motor drive and the class of Piezoelectric Ceramic two can be substantially divided into.This two class is micro-
Scanning is rotated, changes the normal direction of its optical surface so that optical system by the optical element in control optical system
The scene image that is observed in image planes produces minute movement.No matter but which class micro scanning technology can not all overcome system platform appearance
Flating caused by the residual jitter institute of state control in the detector sample integration time, i.e. system platform is shaken and is directly affected
A series of resolution ratio of undersampled image sequences that micro scanning is obtained, and then affect to be observed the Super-resolution Reconstruction of visual field.
The content of the invention
(1) technical problem to be solved
To improve the resolution ratio of the space electro-optical system in the fields such as space flight, remote sensing, military surveillance, system platform attitude is overcome
Impact of the residual jitter of control to single width undersampled image resolution ratio, the stable undersampled image of a series of images is synthesized
One width super-resolution image, the present invention propose it is a kind of can stable single image achievable super-resolution imaging again it is flexible many
Super-resolution imaging system and method.
(2) technical scheme
A first aspect of the present invention, there is provided a kind of super-resolution imaging system of image stabilization, the system contains imaging
Road and detection light path, it is described including imaging optic element group, spectroscope, detecting lenses group, detection camera, calculation control unit,
Pendulum mirror driver, pendulum mirror, imaging lens group, image camera, super-resolution image synthesis unit, wherein：
First propagation path of the target beam being observed be imaging optical path direction be sequentially placed imaging optic element group,
Pendulum mirror, spectroscope, imaging lens group and image camera, the input of super-resolution image synthesis unit and the output of image camera
End connection；Come from the target beam being observed incide be again incident on after imaging optic element group put mirror, Jing pendulum mirror reflecting surface
Spectroscope is incided after reflection, a part of light energy Jing spectroscope in target beam is transmitted to imaging lens group and forms target
Image planes, image camera is located in target image planes, image camera, is formed simultaneously for carrying out opto-electronic conversion to the incident optical signal for receiving
With wirelessly or non-wirelessly form transmit several undersampled images, super-resolution image synthesis unit, for by receive several owe
Sampled images synthesize super-resolution image；
Second propagation path of the target beam being observed be detection optical path direction be sequentially placed imaging optic element group,
Pendulum mirror, spectroscope, detecting lenses group and detection camera, the output end for detecting camera is connected with the input of calculation control unit,
The input of pendulum mirror driver is connected with the output end of calculation control unit, puts the output end of mirror driver and the drive end of pendulum mirror
Connection, come from the target beam being observed incide be again incident on after imaging optic element group put mirror, Jing pendulum mirror reflecting surface it is anti-
Spectroscope is incided after penetrating, another part light energy Jing dichroic mirror in target beam is to detecting lenses group and forms detection
Image planes, detection camera is located in detection image planes；The optical signal incided on detection camera forms and transmits spy Jing after opto-electronic conversion
Video in window is surveyed, the detection window image of reception is carried out calculating calculation control unit the position control electricity for generating and exporting pendulum mirror
Pressing mold analog quantity；Pendulum mirror driver, for the position control voltage analog of the pendulum for receiving mirror to be zoomed into into pendulum mirror driving voltage mould
Analog quantity output is to pendulum mirror.
A second aspect of the present invention, there is provided a kind of super-resolution imaging method of image stabilization, the method is using the present invention
The step of super-resolution imaging system of image stabilization is realized is as follows：
Step S1:Super-resolution system initialization to the steady picture of image, into image stabilization super-resolution imaging Working mould
Formula；
Step S2:Detection camera is imaged to target scene, obtains detection window image；
Step S3:Calculation control unit stores the detection window image for obtaining for the first time as reference picture, and to current
Detection window image carries out the position that cross-correlation function is calculated correlation peak, that is, obtaining detection window image with reference picture
Side-play amount in detection image planes, further according to the geometric optics relation between detection light path and imaging optical path, obtains target image planes
Image shift amount；Calculate the deviation between target image planes image shift amount and the setting value of target image planes image shift amount；
Step S4:Target image planes image shift amount deviation is converted to put in imaging optical path mirror rotation again by calculation control unit
Position control amount；
Step S5:Pendulum mirror driver drives the pendulum mirror deflection in imaging optical path according to position control amount；Repeat step S2～
Step S4, until image camera completes the undersampled image imaging of an image stabilization；
Step S6:Judge whether image camera completes all undersampled image imagings, if not completing all lack samplings
Image is imaged, then execution step S7, if having completed all undersampled image imagings, execution step S8；
Step S7:The setting value of target image planes image shift amount when next time image camera is imaged, execution step are set
S2；
Step S8:Super-resolution image synthesis unit carries out image registration to several undersampled images for obtaining, and calculates and owes
Side-play amount between sampled images；
Step S9:Super-resolution image synthesis unit is multiple using image co-registration according to the side-play amount between undersampled image
Former method is obtained the super-resolution image of a width target by several undersampled images.
(3) beneficial effect
Present invention advantage compared with prior art is as follows：
(1) super-resolution image that the present invention is obtained is higher than the resolution ratio of the super-resolution image that prior art is obtained.
The present invention increased detection camera, calculation control unit in existing imaging optical device, put mirror driver and active controllable
Pendulum mirror, system platform gesture stability residual jitter is avoided to single width undersampled image by their real-time detection and correction
The impact of resolution ratio, it is ensured that the resolution ratio of single width undersampled image in super-resolution imaging, and then ensure the oversubscription for finally giving
Resolution image is the composite result of the undersampled image of many web stabilizations, and this is that existing super-resolution imaging system is beyond one's reach.
(2) present invention is more flexible compared with prior art multiplex.Present system can have four kinds of mode of operations：One is not carry out
Image stabilization super-resolution imaging works, now the pendulum mirror transfixion in present system, with existing imaging optical device
Imaging mode it is identical, be equal to existing imaging optical device；Two is the image forming job of image stabilization, now present system
In optical jitter of the pendulum mirror only in the real time correction image camera time of integration, the image point of the single image that the system is obtained
Resolution is high compared with traditional imaging systems image resolution ratio；Three is that super-resolution imaging works, the now pendulum mirror only conduct in the present invention
Micro scanning super-resolution rebuilding technology obtains the micro scanning device of several undersampled images, no longer carries out the real-time school of optical jitter
Just, it is equal to traditional micro scanning super-resolution imaging device；Four be image stabilization super-resolution imaging work, now this
Pendulum mirror in bright is both that the means for correcting and micro scanning super-resolution rebuilding technology of real time correction imaging optical path optical jitter are obtained
The micro scanning device of several undersampled images is obtained, makes the image in target image planes produce sub-pixed mapping skew.What the system was obtained
The resolution ratio of super-resolution image illustrates the high resolution of the image that prior art is obtained compared with Fig. 5.Above-mentioned four kinds of mode of operations can
Controlled by the software of calculation control unit, switching flexibly, be easily achieved.
Description of the drawings
Fig. 1 is two grades of micro scanning concept maps of super-resolution imaging；
Fig. 2 is the composition frame chart of present system；
Fig. 3 is the flow chart of the inventive method；
Fig. 4 is that the undersampled image of the present invention obtains precedence diagram；
Fig. 5 is the image that prior art imaging system is obtained.
The four width undersampled images that Fig. 6 is obtained for the embodiment of the present invention.
The super-resolution image that Fig. 7 is obtained for the embodiment of the present invention.
Specific embodiment
To make the object, technical solutions and advantages of the present invention become more apparent, below in conjunction with specific embodiment, and reference
Accompanying drawing, the present invention is described in more detail.
Super-resolution (Super-Resolution) improves the resolution ratio of original image by the method for hardware or software.
Super-resolution imaging is exactly to make camera continuous acquisition to mutual sequence less than the one of a pixel distance using active controlled member
Series of discrete image, by image co-registration restored method to this series of discrete image reconstruction, exports a high-resolution figure
Picture.
The super-resolution imaging system of the image stabilization of the present invention shown in Fig. 2 is referred to, containing imaging optical path and detection light
Road, the imaging optical path and detection light path include：Imaging optic element group 10, spectroscope 11, detecting lenses group 12, detection camera
13rd, calculation control unit 14, pendulum mirror driver 15, pendulum mirror 16, imaging lens group 17, image camera 18 and super-resolution image are closed
Into unit 19, wherein：
First propagation path of the target beam being observed is to be sequentially placed imaging optic element group in imaging optical path direction
10th, put mirror 16, spectroscope 11, imaging lens group 17 and image camera 18, the input of super-resolution image synthesis unit 19 with
The output end connection of image camera 18；Come from the target beam being observed and incide and be again incident on after imaging optic element group 10
Pendulum mirror 16, incides spectroscope 11, a part of light energy Jing spectroscope in target beam Jing after the reflective surface of pendulum mirror 16
11 are transmitted to imaging lens group 17 and form target image planes, and image camera 18 is located in target image planes, image camera 18, for right
Several undersampled images that the incident optical signal of reception carries out opto-electronic conversion formation and transmits in wirelessly or non-wirelessly form, super-resolution
Rate image composing unit 19, for several undersampled images for receiving to be synthesized into super-resolution image；
Second propagation path of the target beam being observed is to be sequentially placed imaging optic element group in detection optical path direction
10th, mirror 16, spectroscope 11, detecting lenses group 12 and detection camera 13 are put, the output end and calculation control unit of camera 13 is detected
14 input connection, the input for putting mirror driver 15 is connected with the output end of calculation control unit 14, puts mirror driver 15
Output end with pendulum mirror 16 drive end be connected, come from the target beam being observed and incide after imaging optic element group 10 again
Pendulum mirror 16 is incided, spectroscope 11 is incided Jing after the reflective surface of pendulum mirror 16, another part light energy in target beam
Jing spectroscopes 11 reflex to detecting lenses group 12 and form detection image planes, and detection camera 13 is located in detection image planes；Incide spy
The optical signal surveyed on camera 13 forms and transmits detection window image, the spy that calculation control unit 14 will be received Jing after opto-electronic conversion
Surveying video in window carries out calculating the position control voltage analog for generating and exporting pendulum mirror 16；Pendulum mirror driver 15, for connecing
Receive pendulum mirror 16 position control voltage analog zoom into pendulum mirror driving voltage analog quantity after Jing high-tension cables output to pendulum mirror
16。
When system proposed by the present invention carries out the super-resolution imaging of image stabilization, being observed for the collection of camera 13 is detected
The detection window image of target, calculation control unit 14 is calculated the motion of target image planes epigraph by detection window image
Direction and size, are obtained putting the controlled quentity controlled variable of the reality of mirror 16 with servo control algorithm, drive pendulum mirror 16 to deflect by pendulum mirror driver 15,
The imaging optical path light of mirror-reflection for making Jing pendulum mirrors 16 does not deflect, when now putting mirror only real time correction image camera and integrating
Interior optical jitter, for improving the image resolution ratio of single image, so as to reach within the time of integration of image camera 18
The purpose of stable image, makes image camera 18 obtain the undersampled image of a web stabilization.Calculation control unit 14 is swept further according to micro-
Undersampled image acquisition order and the corresponding target image planes image shift amount setting value needed for super-resolution rebuilding is retouched, control is driven
Movable pendulum mirror 16 deflects fixed angle, and real time correction target image planes image shakes within the integration time for exposure of undersampled image next time
It is dynamic, it is ensured that to obtain stable undersampled image, and there is side-play amount set in advance with the last undersampled image for obtaining, from
And several undersampled images are obtained, then all undersampled images are synthesized into a width super-resolution image.
Imaging optic element group 10, the optical system of present example is hoped using classical Cassegrain (Cassegrain)
Remote mirror system, so imaging optic element group includes that the primary mirror that a face shape is the paraboloid of revolution and a face shape are rotation hyperbolic
The secondary mirror in face；
Spectroscope 11, using a half-reflecting half mirror；
Detecting lenses group 12, using a cemented doublet, makes detection light path focal length be FVisit；
Detection camera 13, is imaged, using high-speed cmos (complementary metal for detection viewing field during stable objects image planes image
Oxide semiconductor) camera or CCD (charge coupled cell) camera, detection camera 13 resolution ratio is 64 × 64 pixels, output frame
In the 2000 frames/more than second, pixel dimension is 32 microns to frequency, and the data image signal for detecting is transferred to calculating by detection camera 13
Control unit 14, and receive the control instruction of detection camera 13 from calculation control unit 14；
Calculation control unit 14 be by field programmable gate array (FPGA) and data processor (DSP) realize from
Product processed.Calculation control unit 14 is connected respectively with detection camera 13 and pendulum mirror driver 15, defeated for receiving detection camera 13
The detection viewing field image for going out, correlation computations is carried out to detection viewing field image and obtains detecting the amount of exercise of image planes epigraph, and then is obtained
To the position control digital quantity of pendulum mirror 16, and it is converted into putting the output of the position control voltage analog of mirror 16 to pendulum mirror driver 15；
Pendulum mirror driver 15, the pendulum mirror position control voltage analog that calculation control unit 14 is exported is enlarged into pendulum mirror and drives
Pendulum mirror 16 is arrived in dynamic voltage analog output, using the pendulum mirror driver supplied on market；
Pendulum mirror 16, the pendulum mirror driving voltage analog quantity exported by pendulum mirror driver 15 drives deflection, you can reduce imaging
Road optical jitter is capable of achieving the deflection of given angle during super-resolution imaging, using on market again with stable objects image planes image
The pendulum mirror platform of supply, pendulum mirror 16 eyeglass is independently developed product, and bore is determined by imaging path optics design；
Imaging lens group 17, using a cemented doublet, makes imaging optical path focal length be FInto；
Image camera 18, for obtaining the target undersampled image of image stabilization, using high-speed cmos camera or CCD phases
Machine, the resolution ratio of image camera 18 is 1024 × 1024 pixels, and in 3-10 frames/second, pixel dimension is 14 microns to output frame, into
As camera 18 is by target undersampled image signal transmission to super-resolution image synthesis unit 19, image camera 18 will detect steady picture
Control instruction is transferred to calculation control unit 14, and reception refers to from the spectrum assignment of image camera 18 of calculation control unit 14
Order；
Super-resolution image synthesis unit 19 adopts the business computer of Windows operating system, super-resolution image to close
It is connected with image camera 18 into unit 19, for carrying out image registration and number to several undersampled images that image camera 18 is obtained
According to fusion, the super-resolution image of target is obtained.
Referring to the application imaging system of the present invention shown in Fig. 3 carries out the method for super-resolution imaging including following step
Suddenly：
Step S1：Start the image stabilization control assembly of image camera 18, including：Detection camera 13, calculation control unit
14th, mirror driver 15 and pendulum mirror 16 are put, image stabilization super-resolution imaging electronics is transmitted by super-resolution image synthesis unit 19
Calculation control unit 14 is enabled signals to, the super-resolution imaging system for making image stabilization enters image stabilization super-resolution imaging
Mode of operation.
The micro scanning that the super-resolution imaging system of the image stabilization in the present invention is employed in Super-resolution Reconstruction technology is general
Read, to put mirror 16 as the micro scanning device for obtaining several undersampled images, calculation control unit 14 sets according to micro scanning order
Target image planes image shift amount setting value when the undersampled image for putting image camera 18 exposes, control pendulum mirror 16 is in each lack sampling
Holding target image planes image stabilization is near side-play amount setting value when image is imaged.With the lack sampling figure that k represents image camera 18
As imaging number of times, the target image planes image shift amount setting value when undersampled image of kth time image camera 18 is imaged is expressed as
x0k,y0k.The present embodiment adopt two grades, three-level and level Four micro scanning order as shown in figure 4, each the deficient of image camera 18 is adopted
Coordinate value in " () " that target image planes image shift amount setting value when sampled images are imaged is shown in Fig. 4.It is with two grades of micro scannings
Example, clockwise scanning obtains four width undersampled images, target image planes image shift amount setting during four undersampled image imagings
Value is respectively：x01=0, y01=0, x02=1/2, y02=0, x03=1/2, y03=1/2, x04=0, y04=1/2.
Step S2：The frame frequency of detection camera 13 is fhigh, unit is that frame is per second, 1000≤fhigh≤10000；Receive target
The optical signal of scene, target scene detection window digital picture W is exported Jing after opto-electronic conversion and Analog-digital Converter, hereinafter referred to as
For detection window image W.Detection window image W length and widths are p pixel.
Step S3:Calculation control unit 14 receives the detection window image from detection camera 13, and storage is obtained for the first time
Detection window image as reference picture, and cross-correlation function calculating is carried out to current detection window image and reference picture,
Side-play amount of the position i.e. detection window image of correlation peak in detection image planes is obtained, further according to detection light path and imaging optical path
Between geometric optics relation, obtain target image planes image shift amount, calculate target image planes image shift amount and target image planes figure
As the deviation between the setting value of side-play amount；
Calculation control unit 14 receives the detection window image from detection camera 13, and with t reception detection window figure is represented
The number of times of picture, the t time reception is referred to as current reception.The detection that the internal storage region storage of calculation control unit 14 is received for the first time
Video in window is expressed as R as reference picture, and with currently received detection window image live image is made, and is expressed as L.Calculate
Live image and cross-correlation function C (x, y) of reference picture, are shown in formula 1, x, and y is the coordinate variable of cross-correlation function C (x, y).
C (x, y)=IFFT [FFT (R) × FFT*(L)] (1)
FFT, IFFT represent respectively forward and inverse Fast Fourier Transform (FFT) in formula 1, the complex conjugate of No. * expression Fourier transformation,
× number representing matrix multiplication cross.
Find out the coordinate (x of correlation function C (x, y) maximum positionmax,ymax), xmax,ymaxIt is coordinate (xmax,
ymax) coordinate points, be constant.It is maximum with correlation function C (x, y) centered on the position of correlation function C (x, y) maximum
The correlation function value of 3 × 3 matrix areas enclosed on weekly duty carries out surface fitting, and it is correlation peak to be fitted the curved surface maximum for obtaining
Cmax, correlation peak CmaxPosition for detection window image offset delta xVisit,δyVisit, unit is the detection pixel of camera 13, is calculated
Formula is shown in formula 2.The correlation peak of all detection window images be all stored in the memory block of calculation control unit 14 in case after
It is continuous to use.
C (x in formula 2max,ymax) it is correlation function C (x, y) of detection window image in coordinate (xmax,ymax) correlation
Functional value, that is, the maximum of correlation function C (x, y) of detection window image；C(xmax-1,ymax) detection window image
Correlation function C (x, y) is in coordinate (xmax-1,ymax) correlation function value；Other cross-correlation function values by that analogy.
Pixel dimension, the pixel dimension of detection camera 13 and imaging optical path and detection light path according to image camera 18
Geometric optics relation, by offset delta x of detection window imageVisit,δyVisitObtain target image planes image shift amount δ xInto,δyInto, unit
For the pixel of image camera 18, computing formula is shown in formula 3, then is calculated target image planes image shift amount δ xInto,δyIntoWith kth time into
Target image planes image shift amount setting value x0 when being imaged as the undersampled image of camera 18k,y0kBetween deviation delta x, Δ y,
Referred to as target image planes image deviations amount Δ x, Δ y, computing formula is shown in formula 4.
F in formula 3, formula 4IntoAnd FVisitThe respectively focal length of imaging optical path and detection light path, parameter value refers to by optical design
Mark is determined；hIntoAnd hVisitRespectively image camera 18 and detection camera 13 pixel dimension.
Step S4：Calculation control unit 14 is converted to the target image planes image deviations amount Δ x obtained by step S3, Δ y
Position control amount s that mirror 16 is rotated is put in imaging optical pathx、sy(see formula 5), and position control amount s that pendulum mirror 16 is rotatedx、sy
It is stored in the memory block of calculation control unit 14.
Q in formula 5x0,qx1,qx2For three parameters of servo controller in horizontal direction calculation control unit 14, qy0,
qy1,qy2For three parameters of servo controller in vertical direction calculation control unit 14, parameter value is steady by image of the present invention
Fixed super-resolution imaging system performance indications are determined；sx、syWithIt is respectively previous cycle and the t-1 time circulation
Pendulum mirror rotate position control amount；Δx、Δy、Δx(t-1)、Δy(t-1)With Δ x(t-2)、Δy(t-2)Be respectively previous cycle,
T-1 circulation and the target image planes image deviations amount of the t-2 time circulation.
Step S5：Position control amount s that the pendulum mirror 16 obtained by step S4 is rotatedx、syExport after digital-to-analogue conversion
The analog quantity is amplified output to pendulum mirror 16 by position control voltage analog, pendulum mirror driver 15, and then is driven in imaging optical path
Pendulum mirror 16 deflect；Above-mentioned steps S2～step S4 is repeated, during the undersampled image Imagewise exposure of image camera 18
Real-time feedback control is carried out to the position of target image planes epigraph, until image camera 18 completes the lack sampling of an image stabilization
Image is imaged.
Step S6:Judge whether image camera 18 completes the imaging of all undersampled images, if not completing all to owe to adopt
Sampled images are imaged, then execution step S7, if having completed all undersampled image imagings, execution step S8.
The present embodiment is respectively adopted two grades, three-level and during level Four micro scanning scheme, the imaging of the undersampled image of image camera 18
Number of times is respectively 4 times, 9 times and 16 times.Understand according to pertinent literature research and actual experiment effect, then the micro scanning for carrying out more stages
The resolution ratio of final super-resolution composograph is also improved without more.
Step S7:The setting value of target image planes image shift amount when next time image camera 18 is imaged, concrete number are set
Value refers to the undersampled image described in step S1 and shown in Fig. 4 and obtains precedence diagram, execution step S2.
Step S8:Several undersampled images that super-resolution image synthesis unit 19 pairs is obtained carry out image registration, calculate
Side-play amount between undersampled image.
Image camera 18 completes all undersampled image imagings by micro scanning order, N width undersampled images is obtained, with n
The serial number for obtaining undersampled image is represented, the n-th width undersampled image obtained by micro scanning order is expressed as In(n=1,
2 ..., N), the length of every width undersampled image is w pixel for l pixel, width.With the first width undersampled image I1For figure
As the benchmark image of registration, I is denoted asBase, every width undersampled image all carries out image registration with benchmark image.Every width lack sampling
Translation between image and benchmark image in the horizontal and vertical directions can be expressed as linear phase place and put down in Fourier's frequency domain
Move, obtained per width by calculating the least-squares estimation of every width undersampled image and the phase difference after benchmark image Fourier transformation
Side-play amount between undersampled image and benchmark image, computing formula is shown in formula 6 and formula 7.
A (u, v)=∠ [FFT (IBase)/FFT(In)] (6)
U in formula 6 and formula 7, v are the coordinate variables of phase place angle function A (u, v), and FFT is Fast Fourier Transform (FFT), ∠
Number for calculated complex phase angle.It is respectively the dictionary of coordinate variable u, v and corresponding coordinate phase place angle function A (u, v)
Body arrangement vector, LSQ represents that least square is solved, αn,βnRepresent the n-th width undersampled image InWith benchmark image IBaseBetween in water
Side-play amount in gentle vertical direction.
Step S9:Side-play amount between the undersampled image that super-resolution synthesis unit 19 is obtained according to step S8, adopts
The image co-registration such as bicubic interpolation method restored method is obtained the super-resolution image of a width target by several undersampled images.
Refer to Fig. 4 and illustrate that the undersampled image of the present invention obtains precedence diagram, bold type numerals represent every grade of micro scanning in figure
The undersampled image serial number of acquisition, two in bracket numeral represents respectively current undersampled image and the first width lack sampling figure
As side-play amount in the horizontal and vertical directions, arrow represents micro scanning moving direction.When the present invention adopts two grades of micro scanning sides
During case, the present invention will obtain four width undersampled images, obtain horizontal direction in the direction of arrows after acquisition piece image inclined therewith
Differ from the second width image of 1/2 pixel of image camera 18, then press again arrow method obtain both horizontally and vertically with
The pixel of 1/2 image camera of all deviations 18 the 3rd width image, finally obtain in the direction of arrows in vertical direction deviation 1/2 therewith
4th width image of the pixel of image camera 18, is so far obtained by two grades of micro scanning undersampled image acquisitions order in left side in Fig. 4
Whole four width images.By that analogy, if the present invention is using three-level or level Four micro scanning scheme, need to be by the three-level in the middle of Fig. 4
Level Four micro scanning order on the right side of micro scanning order or Fig. 4 completes the acquisition of all nine width or 16 width undersampled images.
Refer to Fig. 5 and the image that existing imaging system is obtained is shown.The imaging system of prior art is not both to system
The active correction ability of platform courses residual jitter does not have super-resolution image complex functionality yet, in the time of integration of image camera
Interior, there is shake in the image in imaging optical path target image planes, and the image directly photographed after exposure is fuzzy shown in Fig. 5
Low-resolution image.
By taking the two grades of micro scannings experiment carried out using present system as an example, image camera 18 is obtained four width image stabilizations
Undersampled image, refer to shown in Fig. 6.The image on the basis of the first width undersampled image, four width images carry out therewith image
Side-play amount between the image obtained after registration be (0,0), (α2,β2),(α3,β3) and (α4,β4), by the coordinate value of four width images
The pixel value for obtaining at 0.5 pixel side-play amount using bicubic interpolation method interpolation with pixel value, obtains pixel and increased four times
Super-resolution image, refer to Fig. 7 and illustrate that super-resolution image can tell four groups of targets compared with the undersampled image in Fig. 6 more
Mark, thus knowable to the fuzzy low-resolution image shown in Fig. 5, the resolution ratio shown in Fig. 5 is far below the present invention shown in Fig. 7
The super-resolution image that embodiment is obtained.
The super-resolution imaging system of image stabilization is obtained after a width super-resolution image, and system exits the super of image stabilization
Resolution imaging pattern, into standby mode, put mirror 16 control voltage be set in put voltage, wait next time super-resolution into
As order.
The above, the only specific embodiment in the present invention, but protection scope of the present invention is not limited thereto, and appoints
What be familiar with the people of the technology disclosed herein technical scope in, it will be appreciated that the conversion expected or replacement, all should cover
The present invention include within the scope of.
Claims (8)
1. a kind of super-resolution imaging system of image stabilization, it is characterised in that the system contains imaging optical path and detection light path,
The imaging optical path and detection light path include imaging optic element group, spectroscope, detecting lenses group, detection camera, calculating control
Unit, pendulum mirror driver, pendulum mirror, imaging lens group, image camera, super-resolution image synthesis unit, wherein：
First propagation path of the target beam being observed is to be sequentially placed imaging optic element group, pendulum in imaging optical path direction
Mirror, spectroscope, imaging lens group and image camera, the input of super-resolution image synthesis unit and the output end of image camera
Connection；Come from the target beam being observed incide be again incident on after imaging optic element group put mirror, Jing pendulum mirror reflecting surface it is anti-
Spectroscope is incided after penetrating, a part of light energy Jing spectroscope in target beam is transmitted to imaging lens group and forms target picture
Face, image camera is located in target image planes；Image camera, for receive incident optical signal carry out opto-electronic conversion formation and with
Wirelessly or non-wirelessly form transmission several undersampled images, super-resolution image synthesis unit, for by receive several owe to adopt
Sampled images synthesize super-resolution image；
Second propagation path of the target beam being observed is to be sequentially placed imaging optic element group, pendulum in detection optical path direction
Mirror, spectroscope, detecting lenses group and detection camera, the output end for detecting camera is connected with the input of calculation control unit, puts
The input of mirror driver is connected with the output end of calculation control unit, and the output end and the drive end of pendulum mirror for putting mirror driver connects
Connect, come from the target beam being observed incide be again incident on after imaging optic element group put mirror, Jing pendulum mirror reflective surface
After incide spectroscope, another part light energy Jing dichroic mirror in target beam is to detecting lenses group and forms detection picture
Face, detection camera is located in detection image planes；The optical signal incided on detection camera forms and transmits detection Jing after opto-electronic conversion
The detection window image of reception is carried out calculating video in window, calculation control unit the position control voltage for generating and exporting pendulum mirror
Analog quantity；Pendulum mirror driver, for the position control voltage analog of the pendulum for receiving mirror to be zoomed into into the driving voltage simulation of pendulum mirror
Amount output is to pendulum mirror.
2. the super-resolution imaging system of image stabilization according to claim 1, it is characterised in that the system carries out image
During stable super-resolution imaging, the detection window image of collected by camera target being observed is detected, calculation control unit is by visiting
Direction and size that video in window is calculated target image planes image motion are surveyed, is obtained putting the actual control of mirror with servo control algorithm
Amount processed, by pendulum mirror driver pendulum mirror deflection is driven, and the imaging optical path light for making the mirror-reflection of Jing pendulum mirrors does not deflect, now
Optical jitter in the pendulum mirror only real time correction image camera time of integration, for improving the image resolution of single width undersampled image
Rate, so as to reach the purpose of the stable image within the time of integration of image camera, makes image camera obtain the deficient of a web stabilization and adopts
Sampled images.
3. the super-resolution imaging system of image stabilization according to claim 2, it is characterised in that the calculation control unit
The detection window image that storage is obtained for the first time is carried out mutually as reference picture to current detection window image and reference picture
Correlation function is calculated the position of correlation peak, that is, obtain side-play amount of the detection window image in detection image planes, further according to
Geometric optics relation between detection light path and imaging optical path, obtains target image planes image shift amount；Calculate the skew of target image planes
Deviation between amount and the setting value of target image planes image shift amount.
4. the super-resolution imaging system of image stabilization according to claim 3, it is characterised in that the calculation control unit
Undersampled image acquisition order and the setting of corresponding target image planes image shift amount according to needed for micro scanning super-resolution rebuilding
Value, control drives pendulum mirror deflection fixed angle, and the real time correction target picture within the integration time for exposure of undersampled image next time
Face image is rocked, it is ensured that is obtained stable undersampled image, and is had with the last undersampled image for obtaining set in advance
All undersampled images so as to obtain several undersampled images, then are synthesized a width super-resolution image by side-play amount.
5. a kind of super-resolution imaging method for realizing image stabilization, the image being applied to any one of claim 1-4
The step of stable super-resolution imaging imaging system, the method, is as follows：
Step S1:Super-resolution system initialization to the steady picture of image, into image stabilization super-resolution imaging mode of operation；
Step S2:Detection camera is imaged to target scene, obtains detection window image；
Step S3:The detection window image that calculation control unit storage is obtained for the first time is detected as reference picture to current
Video in window carries out the position that cross-correlation function is calculated correlation peak with reference picture, that is, obtain detection window image and visiting
The side-play amount surveyed in image planes, further according to the geometric optics relation between detection light path and imaging optical path, obtains target image planes image
Side-play amount；Calculate the deviation between target image planes image shift amount and the setting value of target image planes image shift amount；
Step S4:Target image planes image shift amount deviation is converted to put in imaging optical path the position that mirror is rotated again by calculation control unit
Put controlled quentity controlled variable；
Step S5:Pendulum mirror driver drives the pendulum mirror deflection in imaging optical path according to position control amount；Repeat step S2～step
S4, until image camera completes the undersampled image imaging of an image stabilization；
Step S6:Judge whether image camera completes all undersampled image imagings, if not completing all undersampled images
It is imaged, then execution step S7, if having completed all undersampled images imagings, execution step S8；
Step S7:The setting value of target image planes image shift amount when next time image camera is imaged, execution step S2 are set；
Step S8:Super-resolution image synthesis unit carries out image registration to several undersampled images for obtaining, and calculates lack sampling
Side-play amount between image；
Step S9:Super-resolution image synthesis unit according to the side-play amount between undersampled image, using image co-registration recovery side
Method is obtained the super-resolution image of a width target by several undersampled images.
6. the super-resolution imaging method of image stabilization according to claim 5, it is characterised in that the calculation control unit
Target image planes image shift amount setting value when the undersampled image for arranging image camera according to the micro scanning order of pendulum mirror exposes,
Control pendulum mirror keeps target image planes image stabilization when each undersampled image is imaged near side-play amount setting value.
7. the super-resolution imaging method of image stabilization according to claim 6, it is characterised in that according to the picture of image camera
Elemental size, the pixel dimension of detection camera and imaging optical path and the geometric optics relation for detecting light path, by detection window image
Side-play amount obtain target image planes image shift amount, then be calculated target image planes image shift amount with kth time image camera
The deviation between target image planes image shift amount setting value when undersampled image is imaged.
8. the super-resolution imaging method of image stabilization according to claim 7, it is characterised in that the image camera is by micro-
Scanning sequency completes all undersampled image imagings, obtains N width undersampled images, is represented with n and obtains the suitable of undersampled image
Sequence number, by the n-th width undersampled image that micro scanning order is obtained, the length of every width undersampled image is w for l pixel, width
Individual pixel；Benchmark image with the first width undersampled image as image registration, every width undersampled image is all carried out with benchmark image
Image registration；Translation between every width undersampled image and benchmark image in the horizontal and vertical directions is in Fourier's frequency domain
Linear phase shift is expressed as, the phase difference after the every width undersampled image of calculating with benchmark image Fourier transformation is most
A young waiter in a wineshop or an inn takes advantage of estimation to obtain the side-play amount between every width undersampled image and benchmark image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410779707.7A CN104394309B (en) | 2014-12-16 | 2014-12-16 | Super-resolution imaging system and method with stable images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410779707.7A CN104394309B (en) | 2014-12-16 | 2014-12-16 | Super-resolution imaging system and method with stable images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104394309A CN104394309A (en) | 2015-03-04 |
| CN104394309B true CN104394309B (en) | 2017-05-10 |
Family
ID=52612139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410779707.7A Active CN104394309B (en) | 2014-12-16 | 2014-12-16 | Super-resolution imaging system and method with stable images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104394309B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105069748B (en) * | 2015-07-16 | 2017-11-10 | 哈尔滨工业大学 | A kind of method that high-definition picture is obtained based on microsatellite object space scanning technique |
| CN105184762B (en) * | 2015-08-31 | 2017-09-15 | 北京理工大学 | The method that geosynchronous satellite posture lack sampling measures lower super-resolution image reconstruction |
| CN106504228B (en) * | 2016-09-30 | 2019-02-12 | 深圳市莫廷影像技术有限公司 | A kind of a wide range of high definition rapid registering method of ophthalmology OCT image |
| CN106504195B (en) * | 2016-10-27 | 2019-10-18 | 哈尔滨工业大学 | Super-resolution imaging method and aplanatism and non-aplanatism super-resolution imaging system |
| CN112017122B (en) * | 2017-05-26 | 2024-02-09 | 上海微小卫星工程中心 | Super-resolution imaging method |
| KR102589291B1 (en) * | 2017-11-17 | 2023-10-16 | 엘지이노텍 주식회사 | Camera module, optical apparatus including thereof, and image generating method performed therein |
| CN108876783B (en) | 2018-06-27 | 2021-02-05 | 上海联影医疗科技股份有限公司 | Image fusion method and system, medical equipment and image fusion terminal |
| CN108983419B (en) * | 2018-06-27 | 2022-03-15 | 湖北华中光电科技有限公司 | Optical system based on multispectral imaging |
| CN109348103B (en) * | 2018-10-26 | 2021-05-25 | 大连海事大学 | Camera time resolution multiplying method and device based on time coding |
| CN109246349B (en) * | 2018-11-14 | 2020-02-28 | 中国科学院西安光学精密机械研究所 | High-quality super-resolution area array imaging camera and imaging method |
| CN110113592A (en) * | 2019-05-05 | 2019-08-09 | 云南兆讯科技有限责任公司 | Power equipment inspection figure image collection processing system based on camera array technology |
| CN111122126B (en) * | 2019-12-31 | 2022-03-22 | 北京灵犀微光科技有限公司 | Optical system definition testing method and device |
| CN113784061A (en) * | 2020-06-08 | 2021-12-10 | 中国航空工业集团公司洛阳电光设备研究所 | Super-resolution infrared imaging system and image stabilizing method and device thereof |
| CN114302031A (en) * | 2021-11-15 | 2022-04-08 | 华能国际电力股份有限公司上海石洞口第二电厂 | Camera array transmission method and system based on 5G communication |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3770921B2 (en) * | 1993-03-31 | 2006-04-26 | 三菱電機株式会社 | Optical disk device |
| CN1555097A (en) * | 2003-12-24 | 2004-12-15 | 中国人民解放军国防科学技术大学 | Quick algorithm in planar charge coupling device array super resolution imaging technology |
| CN101217625B (en) * | 2008-01-11 | 2010-07-28 | 清华大学 | Device and method of super-resolution imaging |
| EP2495592A4 (en) * | 2009-07-13 | 2018-04-18 | Nikon Corporation | Three-dimensional directional drift control device and microscope device |
| CN104168430B (en) * | 2014-08-18 | 2017-08-25 | 西安电子科技大学 | A kind of TDI CCD focal planes code super-resolution imaging device and method |
-
2014
- 2014-12-16 CN CN201410779707.7A patent/CN104394309B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN104394309A (en) | 2015-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104394309B (en) | Super-resolution imaging system and method with stable images | |
| CN100481937C (en) | Equipment for reconstructing high dynamic image in high resolution | |
| US10353191B2 (en) | Circular scanning technique for large area inspection | |
| CN101911671B (en) | Imaging device and optical axis control method | |
| CN105069748B (en) | A kind of method that high-definition picture is obtained based on microsatellite object space scanning technique | |
| Ye et al. | Universal semantic segmentation for fisheye urban driving images | |
| JP5760559B2 (en) | Stereo camera device and parallax image generation method | |
| CN110596885B (en) | Scanning light field imaging system | |
| CN101217625B (en) | Device and method of super-resolution imaging | |
| CN113175899B (en) | Camera and galvanometer combined three-dimensional imaging model of variable sight line system and calibration method thereof | |
| JP2008109477A (en) | Image generating device and image generating method | |
| JP7312185B2 (en) | Camera module and its super-resolution image processing method | |
| CN109360139A (en) | Sub-pixel super-resolution imaging system and method based on translation adjustable wavefront coding | |
| CN108648147A (en) | A kind of super-resolution image acquisition method and system of human eye retina's mechanism | |
| CN105911692A (en) | Optimal system parameter selection method for Fourier laminated micro imaging system | |
| CN111538051B (en) | Precise processing method for swing-scanning large-width optical satellite | |
| CN104410789A (en) | Staring super-resolution imaging device and method | |
| CN115086550A (en) | Meta-imaging method and system | |
| CN107631728B (en) | Satellite-borne image aided navigation method | |
| CN103248838B (en) | CMOS camera image bias current adjusting method based on numeric field TDI algorithm | |
| Liu et al. | A dense light field reconstruction algorithm for four-dimensional optical flow constraint equation | |
| JPS63284980A (en) | Solid-state image pickup device | |
| CN111664944A (en) | Image stabilization, non-uniform correction and super-resolution system based on micro-scanning platform | |
| CN104143179B (en) | Method for enhancing moving target through multi-linear-array time difference scanning expansion sampling | |
| CN107152971B (en) | A method of improving linear array infrared exoelectron enlarged drawing level detail resolution ratio |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |