CN105096251A - Method for improving splicing image resolution by using super-resolution reconstruction technology - Google Patents
Method for improving splicing image resolution by using super-resolution reconstruction technology Download PDFInfo
- Publication number
- CN105096251A CN105096251A CN201510424151.4A CN201510424151A CN105096251A CN 105096251 A CN105096251 A CN 105096251A CN 201510424151 A CN201510424151 A CN 201510424151A CN 105096251 A CN105096251 A CN 105096251A
- Authority
- CN
- China
- Prior art keywords
- image
- resolution
- sequence
- super
- displacement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000005516 engineering process Methods 0.000 title claims abstract description 24
- 238000006073 displacement reaction Methods 0.000 claims abstract description 36
- 238000013519 translation Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 230000008878 coupling Effects 0.000 claims description 3
- 238000010168 coupling process Methods 0.000 claims description 3
- 238000005859 coupling reaction Methods 0.000 claims description 3
- 238000007500 overflow downdraw method Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method for improving a splicing image resolution by using a super-resolution reconstruction technology. The method comprises the following steps: step one, obtaining two frames of images A1 and B1 having a partially overlapped boundary as well as sequence images A2-A4 and B2-B4 having infinitesimal displacement relations with the images; step two, carrying out super-resolution reconstruction on the images A1 and B1 to obtain high-resolution images A and B; and step three, carrying out image splicing on the high-resolution images A and B to obtain a spliced image C with a wide view angle. According to the invention, the resolution of the spliced image is improved by using the super-resolution reconstruction technology; and the image super-resolution reconstruction technology and the image splicing technology are combined. Therefore, the view field is expanded and the image resolution is enhanced; and thus the rich scene information can be obtained.
Description
Technical field
The invention belongs to technical field of image processing, relate to a kind of super-resolution rebuilding technology that utilizes to improve the method for stitching image resolution.
Background technology
Image mosaic derives from the photographic knowledge of the mankind the earliest, and when obtaining the scene image in the wide visual field owing to using ordinary digital camera, the restriction of camera subject resolution, the scene of shooting is larger, and the image resolution ratio obtained is lower.And obtaining the specialised hardware equipment of panorama sketch, it is not only expensive that such as panorama camera, fish eye lens and linear push sweep camera etc., and distortion is also relatively more serious, there is the distortion of parts of images.In order to obtain ultra-wide the visual angle even panorama sketch of 360 degree under the condition not reducing image resolution ratio, utilizing computing machine to carry out image mosaic and being suggested and research and development gradually.
Image mosaic technology by the small angle image on a series of border that overlaps, can be used different Processing Algorithm to carry out coupling according to the feature of different images and aims at, thus be spliced into a fabric width multi-view image.But although image mosaic technology expands visual field and obtain the image with stable resolution, its resolution still has very large room for promotion.
Summary of the invention
Object of the present invention provides a kind of super-resolution rebuilding technology that utilizes to improve the method for stitching image resolution, the basis of image mosaic uses image super-resolution rebuilding technology obtain wide visual field, high-resolution image scene figure.
The object of the invention is to be achieved through the following technical solutions:
Utilize super-resolution rebuilding technology to improve a method for stitching image resolution, comprise the following steps:
Step one: obtain and there are two two field picture A1, the B1 on the border that partly overlaps and there is with it sequence image A2-A4, B2-B4 of micro-displacement relation respectively.
The present invention is directed to target scene, first the information utilizing camera system to obtain a part of scene obtains image A1, now camera system is moved in the horizontal direction respectively micro-displacement (sub-pixel is to tens pixels) and obtain sequence image A2, move micro-displacement in the vertical direction and obtain sequence image A3, all move micro-displacement in horizontal and vertical direction and obtain sequence image A4; Then fortune uses the same method, and obtains another part image B1 of target scene and has the sequence image B2-B4 of micro-displacement with it.Like this, the collecting work with partly overlap boundary image A1, B1 and sequence image A2-A4, B2-B4 is just completed.
Step 2: A1-A4 and B1-B4 is carried out Super-resolution Reconstruction respectively, obtains high-definition picture A, B.
For the process of reconstruction of image A, the key step of this image super-resolution rebuilding is as follows:
(1) with piece image A1 for benchmark, utilize in image the unique point being easy to realize extracted with high accuracy, calculate sequence image A2, exact shift relation between A3, A4 and A1;
(2) if the micrometric displacement amount between image sequence is all less than a pixel namely have sub-pix dislocation, so directly utilize this image sequence A1-A4 to carry out super-resolution rebuilding, namely directly perform step (4); Otherwise perform step (3);
(3) if the micrometric displacement amount between image sequence is more than a pixel, then need to carry out image translation to the displacement of Integer Pel size between sequence image, thus obtain the image sequence with sub-pix micrometric displacement, and then perform step (4);
(4) utilize image sequence A1-A4 to carry out super-resolution rebuilding and obtain high-definition picture A.Reconstruction algorithm is specific as follows:
With the summit in the image A1 upper left corner for initial point, coordinate system is set up along image coordinate system direction, so A2, A3, A4 tri-the apex coordinate in the width image upper left corner be respectively (Δ x, 0), (0, Δ y), (Δ x, Δ y), wherein Δ x and Δ y is respectively sub-pix micrometric displacement size horizontal and vertical between image sequence.Reconstruction algorithm formula is as shown in (1):
Wherein, i, j are natural number, and a=Δ x/ λ, b=Δ y/ λ, λ is the pixel size of original image.
Thus, complete the reconstruction of high-definition picture A, same method utilizes B1-B4 to obtain high-definition picture B.
Step 3: the stitching image C that image mosaic obtains wide viewing angle is carried out to high-definition picture A, B.
This merging algorithm for images key step is as follows:
(1) extraction of unique point: utilize Harris angle detector to treat stitching image A, B and extract angle point;
(2) coupling of unique point: utilize unique point neighborhood gray scale cross-correlation method to carry out Feature Points Matching;
(3) fusion of image: utilize to be fade-in and gradually go out fusion method and carry out image co-registration, obtains splicing result.
The present invention compared with prior art, has the following advantages:
First, the present invention utilizes super-resolution rebuilding technology to improve the resolution of stitching image, image super-resolution rebuilding technology is combined with image mosaic technology, while expanding visual field, also improves the resolution of image, and then obtain more abundant scene information.
The second, in the present invention for micrometric displacement amount more than two two field pictures of a pixel before rebuilding, image translation has been carried out to the displacement of Integer Pel size between sequence image, thus has obtained the image sequence with sub-pix micrometric displacement.
3rd, utilize the method for Integer Pel image translation to solve the problem of two two field picture micrometric displacement amounts more than a pixel in the present invention, super-resolution rebuilding technology is not made excessive demands in accurate micrometric displacement.
4th, utilize Harris Corner Detection Algorithm to treat in the present invention extraction that stitching image carries out unique point.Because Harris angle point is isotropic in the picture, even if image rotates the detection that also can not affect angle point like this, the detection of angle point simultaneously can not be subject to the impact of light intensity substantially.
Accompanying drawing explanation
Fig. 1 is the process flow diagram utilizing super-resolution rebuilding technology to improve stitching image resolution method;
Fig. 2 is the structural representation of image A1-A4, B1-B4;
Fig. 3 is image super-resolution rebuilding schematic diagram;
Fig. 4 is the process flow diagram of image split-joint method.
Embodiment
Below in conjunction with accompanying drawing, technical scheme of the present invention is further described; but be not limited thereto; everyly technical solution of the present invention modified or equivalent to replace, and not departing from the spirit and scope of technical solution of the present invention, all should be encompassed in protection scope of the present invention.
The invention provides a kind of method utilizing super-resolution rebuilding technology to improve stitching image resolution, as shown in Figure 1, comprise the following steps:
S1, acquisition have two two field picture A1, the B1 on the border that partly overlaps and have sequence image A2-A4, B2-B4 of micro-displacement relation respectively with it.
As shown in Figure 2, for target scene, first the information utilizing camera system to obtain a part of scene obtains image A1, now camera system is moved in the horizontal direction respectively micro-displacement (sub-pixel is to tens pixels) and obtain sequence image A2, move micro-displacement in the vertical direction and obtain sequence image A3, all move micro-displacement in horizontal and vertical direction and obtain sequence image A4; Then fortune uses the same method, and obtains another part image B1 of target scene and has the sequence image B2-B4 of micro-displacement with it.Thus, the collecting work of image A1-A4, B1-B4 is completed.
S2, A1-A4 and B1-B4 is carried out Super-resolution Reconstruction respectively, obtain high-definition picture A, B.
S21, process of reconstruction for high-definition picture A, with piece image A1 for benchmark, utilize in image the unique point being easy to realize extracted with high accuracy to calculate sequence image A2, exact shift relation between A3, A4 and A1;
If the micrometric displacement amount between S22 image sequence is all less than a pixel namely have sub-pix dislocation, so directly utilize this image sequence A1-A4 to carry out super-resolution rebuilding, namely directly perform step S24; Otherwise perform step S23;
If the micrometric displacement amount between S23 image sequence is more than a pixel, then need to carry out image translation to the displacement of Integer Pel size between sequence image, thus obtain the image sequence with sub-pix micrometric displacement, and then execution step S24 utilizes image sequence A1-A4 to carry out super-resolution rebuilding.
S24, utilize image sequence A1-A4 to carry out super-resolution rebuilding to obtain high-definition picture A.As shown in Figure 3, concrete reconstruction algorithm is as follows for process of reconstruction:
With the summit in the image A1 upper left corner for initial point, coordinate system is set up along image coordinate system direction, so A2, A3, A4 tri-the apex coordinate in the width image upper left corner be respectively (Δ x, 0), (0, Δ y), (Δ x, Δ y), wherein Δ x and Δ y is respectively sub-pix micrometric displacement size horizontal and vertical between image sequence.Algorithmic formula is as follows:
Wherein, i, j are natural number, and a=Δ x/ λ, b=Δ y/ λ, λ is the pixel size of original image.
Thus, the reconstruction of high-definition picture A is completed.
S25, profit use the same method, and B1-B4 is carried out image super-resolution rebuilding and obtains high-definition picture B.
S3, the stitching image C that image mosaic obtains wide viewing angle is carried out to high-definition picture A, B.
As shown in Figure 4, specific algorithm is mainly as follows for this image mosaic process flow diagram:
S31, feature point extraction: utilize Harris angle detector to treat stitching image A, B and extract angle point;
S32, Feature Points Matching: utilize unique point neighborhood gray scale cross-correlation method to carry out Feature Points Matching;
S33, image co-registration: utilize to be fade-in and gradually go out fusion method and carry out image co-registration, obtain splicing result.
Claims (5)
1. utilize super-resolution rebuilding technology to improve a method for stitching image resolution, it is characterized in that described method step is as follows:
Step one: obtain and there are two two field picture A1, the B1 on the border that partly overlaps and there is with it sequence image A2-A4, B2-B4 of micro-displacement relation respectively;
Step 2: A1-A4 and B1-B4 is carried out Super-resolution Reconstruction respectively, obtains high-definition picture A, B;
Step 3: the stitching image C that image mosaic obtains wide viewing angle is carried out to high-definition picture A, B.
2. the super-resolution rebuilding technology that utilizes according to claim 1 is to improve the method for stitching image resolution, it is characterized in that the concrete steps of described step one are as follows:
For target scene, first the information utilizing camera system to obtain a part of scene obtains image A1, now camera system is moved in the horizontal direction respectively micro-displacement and obtain sequence image A2, move micro-displacement in the vertical direction and obtain sequence image A3, all move micro-displacement in horizontal and vertical direction and obtain sequence image A4; Then fortune uses the same method, and obtains another part image B1 of target scene and has the sequence image B2-B4 of micro-displacement with it.
3. the super-resolution rebuilding technology that utilizes according to claim 1 is to improve the method for stitching image resolution, it is characterized in that, in described step 2, the reconstruction procedures of image A is as follows:
(1) with piece image A1 for benchmark, utilize in image the unique point being easy to realize extracted with high accuracy, calculate sequence image A2, exact shift relation between A3, A4 and A1;
(2) if the micrometric displacement amount between image sequence is all less than a pixel namely have sub-pix dislocation, so directly utilize this image sequence A1-A4 to carry out super-resolution rebuilding, namely directly perform step (4); Otherwise perform step (3);
(3) if the micrometric displacement amount between image sequence is more than a pixel, then need to carry out image translation to the displacement of Integer Pel size between sequence image, thus obtain the image sequence with sub-pix micrometric displacement, and then perform step (4);
(4) utilize image sequence A1-A4 to carry out super-resolution rebuilding and obtain high-definition picture A.
4. the super-resolution rebuilding technology that utilizes according to claim 3 is to improve the method for stitching image resolution, it is characterized in that in described step (4), the reconstruction algorithm of high-definition picture is as follows:
With the summit in the image A1 upper left corner for initial point, coordinate system is set up along image coordinate system direction, so A2, A3, A4 tri-the apex coordinate in the width image upper left corner be respectively (Δ x, 0), (0, Δ y), (Δ x, Δ y), wherein Δ x and Δ y is respectively sub-pix micrometric displacement size horizontal and vertical between image sequence.Reconstruction algorithm formula is as follows:
Wherein, i, j are natural number, and a=Δ x/ λ, b=Δ y/ λ, λ is the pixel size of original image.
5. the super-resolution rebuilding technology that utilizes according to claim 1 is to improve the method for stitching image resolution, it is characterized in that the concrete steps of described step 3 are as follows:
(1) extraction of unique point: utilize Harris angle detector to treat stitching image A, B and extract angle point;
(2) coupling of unique point: utilize unique point neighborhood gray scale cross-correlation method to carry out Feature Points Matching;
(3) fusion of image: utilize to be fade-in and gradually go out fusion method and carry out image co-registration, obtains splicing result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510424151.4A CN105096251A (en) | 2015-07-18 | 2015-07-18 | Method for improving splicing image resolution by using super-resolution reconstruction technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510424151.4A CN105096251A (en) | 2015-07-18 | 2015-07-18 | Method for improving splicing image resolution by using super-resolution reconstruction technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105096251A true CN105096251A (en) | 2015-11-25 |
Family
ID=54576604
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510424151.4A Pending CN105096251A (en) | 2015-07-18 | 2015-07-18 | Method for improving splicing image resolution by using super-resolution reconstruction technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105096251A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106161997A (en) * | 2016-06-30 | 2016-11-23 | 上海华力微电子有限公司 | Improve the method and system of cmos image sensor pixel |
| CN107123100A (en) * | 2017-05-04 | 2017-09-01 | 王蕾 | FiDrizzle multiple repairing weld Image Reconstruction Technologies |
| CN111640061A (en) * | 2020-05-12 | 2020-09-08 | 哈尔滨工业大学 | Self-adaptive image super-resolution system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101980291A (en) * | 2010-11-03 | 2011-02-23 | 天津大学 | Random micro-displacement-based super-resolution image reconstruction method |
| CN102831591A (en) * | 2012-06-27 | 2012-12-19 | 北京航空航天大学 | Gaussian filter-based real-time defogging method for single image |
| CN103150715A (en) * | 2013-03-13 | 2013-06-12 | 腾讯科技（深圳）有限公司 | Image stitching processing method and device |
| CN103489173A (en) * | 2013-09-23 | 2014-01-01 | 百年金海科技有限公司 | Video image super-resolution reconstruction method |
| WO2014079879A2 (en) * | 2012-11-21 | 2014-05-30 | Thales | Method of 3d reconstruction and 3d panoramic mosaicing of a scene |
| CN104766273A (en) * | 2015-04-20 | 2015-07-08 | 重庆大学 | Infrared image super-resolution reestablishing method based on compressed sensing theory |
-
2015
- 2015-07-18 CN CN201510424151.4A patent/CN105096251A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101980291A (en) * | 2010-11-03 | 2011-02-23 | 天津大学 | Random micro-displacement-based super-resolution image reconstruction method |
| CN102831591A (en) * | 2012-06-27 | 2012-12-19 | 北京航空航天大学 | Gaussian filter-based real-time defogging method for single image |
| WO2014079879A2 (en) * | 2012-11-21 | 2014-05-30 | Thales | Method of 3d reconstruction and 3d panoramic mosaicing of a scene |
| CN103150715A (en) * | 2013-03-13 | 2013-06-12 | 腾讯科技（深圳）有限公司 | Image stitching processing method and device |
| CN103489173A (en) * | 2013-09-23 | 2014-01-01 | 百年金海科技有限公司 | Video image super-resolution reconstruction method |
| CN104766273A (en) * | 2015-04-20 | 2015-07-08 | 重庆大学 | Infrared image super-resolution reestablishing method based on compressed sensing theory |
Non-Patent Citations (1)
| Title |
|---|
| 张平 等: "基于多次成像的分辨率提高算法研究", 《光子学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106161997A (en) * | 2016-06-30 | 2016-11-23 | 上海华力微电子有限公司 | Improve the method and system of cmos image sensor pixel |
| CN107123100A (en) * | 2017-05-04 | 2017-09-01 | 王蕾 | FiDrizzle multiple repairing weld Image Reconstruction Technologies |
| CN107123100B (en) * | 2017-05-04 | 2020-01-31 | 王蕾 | FiDrizzle multi-sampling image reconstruction technology |
| CN111640061A (en) * | 2020-05-12 | 2020-09-08 | 哈尔滨工业大学 | Self-adaptive image super-resolution system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11823363B2 (en) | Infrared and visible light fusion method | |
| CN102984453B (en) | Single camera is utilized to generate the method and system of hemisphere full-view video image in real time | |
| WO2021120407A1 (en) | Parallax image stitching and visualization method based on multiple pairs of binocular cameras | |
| CN101276465B (en) | Method for automatically split-jointing wide-angle image | |
| CN104301677B (en) | The method and device monitored towards the panoramic video of large scene | |
| CN102857739B (en) | Distributed overall view monitoring system and method thereof | |
| CN204090039U (en) | Integration large scene panoramic video monitoring device | |
| CN104463778B (en) | A kind of Panoramagram generation method | |
| CN103337094A (en) | Method for realizing three-dimensional reconstruction of movement by using binocular camera | |
| CN101658347B (en) | Method for obtaining dynamic shape of foot model | |
| CN101247513A (en) | Method for real-time generating 360 degree seamless full-view video image by single camera | |
| CN105005964A (en) | Video sequence image based method for rapidly generating panorama of geographic scene | |
| CN104618648A (en) | Panoramic video splicing system and splicing method | |
| CN101321303A (en) | Geometric and optical correction method for non-plane multi-projection display | |
| CN105894443B (en) | A kind of real-time video joining method based on improved SURF algorithm | |
| CN104809719A (en) | Virtual view synthesis method based on homographic matrix partition | |
| CN105550981A (en) | Image registration and splicing method on the basis of Lucas-Kanade algorithm | |
| CN105096251A (en) | Method for improving splicing image resolution by using super-resolution reconstruction technology | |
| CN102231806B (en) | Video-based dual-parameter inner pipeline wall panorama modeling and generation method | |
| JP2012185712A (en) | Image collation device and image collation method | |
| CN105657268A (en) | Multi-viewpoint video splicing and fusion algorithm based on multiple resolutions | |
| CN103167247B (en) | A kind of video sequence color image joining method | |
| Gao et al. | Multi-source data-based 3D digital preservation of largescale ancient chinese architecture: A case report | |
| CN105139336A (en) | Method for converting multichannel panorama images into dome-screen fish-eye movie | |
| CN104796623A (en) | Method for eliminating structural deviation of stitched video based on pyramid block matching and functional optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20151125 |