CN108074218A - Image super-resolution method and device based on optical field acquisition device - Google Patents
Image super-resolution method and device based on optical field acquisition device Download PDFInfo
- Publication number
- CN108074218A CN108074218A CN201711474559.8A CN201711474559A CN108074218A CN 108074218 A CN108074218 A CN 108074218A CN 201711474559 A CN201711474559 A CN 201711474559A CN 108074218 A CN108074218 A CN 108074218A
- Authority
- CN
- China
- Prior art keywords
- resolution
- image
- mrow
- super
- acquisition device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
- G06T3/4076—Super resolution, i.e. output image resolution higher than sensor resolution by iteratively correcting the provisional high resolution image using the original low-resolution image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a kind of image super-resolution methods and device based on optical field acquisition device, optical field acquisition device includes multiple USB cameras and camera, to form an optical field acquisition device with 3 × 3 visual angles, multiple USB cameras at side view angle with the space form regular array of square and are centered around around camera, wherein, method includes：Multiple side view angles low-resolution image and intermediate visual angle high-definition picture are obtained by gathering light field image；Super-resolution is carried out to the multiple side view angles low-resolution image collected using dictionary learning and deep learning；The depth information of scene is obtained according to the parallax between the light field image of multiple different visual angles.This method can be predicted the high frequency section for recovering input picture, and using the information of various visual angles high-definition picture, calculated the depth of scene, reduce cost of manufacture, ensure the accuracy of space and angular resolution by single-view and various visual angles information.
Description
Technical field
The present invention relates to technical field of computer vision, more particularly to a kind of Image Super-resolution based on optical field acquisition device
Rate method and device.
Background technology
Optical field acquisition and its reconstruction technique be in computer vision field one it is extremely important the problem of, carried out using light field
Three-dimensional reconstruction has great advantage compared to traditional three-dimensional rebuilding method：It is small to the dependence of hardware resource, easily on PC
Carry out real-time reconstruction；Strong applicability, scene complexity do not influence the complexity of calculating.However, use spatial digitizer
Although high accuracy three-dimensional reconstruction can be carried out, the limitation of its expensive equipment price and use occasion limits actual
Using.Light field technology is in illuminating engineering, light field renders, illumination, refocusing camera shooting, synthetic aperture imaging, 3D display, security protection are supervised again
The occasions such as control have a wide range of applications.
In correlation technique, optical field acquisition device mainly has：It is most common to have spherical camera array peace using camera array
Face/linescan camera array, generally requires using tens or a camera arrangement up to a hundred suitable position in the scene is to same
The acquisition that scene synchronizes；Using lens array, by once shooting the photo of the scene difference depth of field, scene can be realized
Any range focus on, and such light-field camera has emerged and has entered commercial application field, and harvester is often to phase
The spatial resolution requirements of machine are higher, therefore high hardware cost limits its development.
One of key problem of light field three-dimensional reconstruction is to improve the resolution ratio of light field image.Due to acquired image
Resolution ratio will directly affect the calculating of scene depth, and therefore, estimation of Depth will be improved by carrying out super-resolution to gathered image
The accuracy of precision and three-dimensional reconstruction.Using high-resolution image information three-dimensional modeling can be carried out to scene, it is basic herein
On can realize the arbitrary viewpoint of scene, the virtual image of arbitrary illumination and image segmentation, stereoscopic display etc. significantly
Using.Traditional light field image super-resolution algorithms are mainly based upon the dictionary learning method of multiple views, are regarded by extracting difference
Dictionary information in point, different resolution image, foundation correspond relation, utilize the side of high-definition picture block weighting summation
Formula carries out super-resolution to low-resolution image.Due to optical field acquisition device in order to obtain larger angular resolution, difference regards
Often parallax is bigger for the image at angle, and the method for dictionary learning can not effectively carry out super-resolution in this case, cause ghost,
It is fuzzy to wait as a result, the accuracy of information entrained by light field image is so constrained significantly, so as to further influence subsequent scenario weight
The precision built has to be solved.
The content of the invention
It is contemplated that it solves at least some of the technical problems in related technologies.
For this purpose, an object of the present invention is to provide a kind of image super-resolution method based on optical field acquisition device,
This method can greatly reduce cost of manufacture, ensure the accuracy of space and angular resolution, can realize that light-field camera is again right
It is burnt.
It is another object of the present invention to propose a kind of image super-resolution device based on optical field acquisition device.
In order to achieve the above objectives, one aspect of the present invention embodiment proposes a kind of image oversubscription based on optical field acquisition device
Resolution method, the optical field acquisition device include multiple USB cameras and camera, to form a light field with 3 × 3 visual angles
Harvester, the multiple USB camera at side view angle with the space form regular array of square and are centered around the phase
Around machine, wherein, it the described method comprises the following steps：Light field image is gathered by the optical field acquisition device, it is multiple to obtain
Side view angle low-resolution image and intermediate visual angle high-definition picture；Using dictionary learning and deep learning to the institute that is collected
It states multiple side view angles low-resolution image and carries out super-resolution；It is obtained according to the parallax between the light field image of multiple different visual angles
The depth information of scene.
The image super-resolution method based on optical field acquisition device of the embodiment of the present invention is believed by single-view and various visual angles
Breath, prediction recovers the high frequency section of input picture, and using the information of various visual angles high-definition picture, further calculates scene
Depth, available for the situation that scene rebuilding, large scene monitor, greatly reduce cost of manufacture, ensure space and angle point
The accuracy of resolution can realize that light-field camera is focused again.
In addition, the image super-resolution method according to the above embodiment of the present invention based on optical field acquisition device can also have
There is following additional technical characteristic：
Further, in one embodiment of the invention, it is described to utilize dictionary learning and deep learning to being collected
The multiple side view angle low-resolution image carry out super-resolution, further comprise：By the intermediate visual angle high resolution graphics
As down-sampled, it is carried out to the super-resolution at the visual angle using the deep learning method by trained convolutional neural networks
Rate obtains residual image, to reflect the residual error of neutral net super-resolution with down-sampled preceding image subtraction；By the multiple side
Visual angle low-resolution image and the intermediate high-definition picture extract dictionary information using image block, and do low resolution figure
As block is corresponding with the information of high-definition picture block；By each side view angle low-resolution image by the convolutional neural networks into
The preliminary super-resolution of row, using information correspondence, side view angle is transformed by the residual image, with passing through the first of neutral net
Obtained super-resolution results added is walked, to obtain final super-resolution result.
Further, in one embodiment of the invention, the corresponding residual plot in each position is obtained by the following formula
As block：
Wherein,For the high-definition picture block that side view angle is rebuild, k is high resolution graphics among closest 9 chosen
As the index of block, ωkFor k-th of weight,Among required k-th of side view angle full resolution pricture block of j-th of formation
Full resolution pricture block, R are the value on intermediate high-definition picture, and j is the index of side view angle image block.
Further, in one embodiment of the invention, the final super-resolution result is：
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) be by neutral net into
Capable image block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
Further, in one embodiment of the invention, between the light field image according to multiple different visual angles
Parallax obtains the depth information of scene, further comprises：The edge confidence degree of every light field image is obtained, to obtain edge confidence
Spend mask；Obtain being marked as the parallax value of the pixel at confidence edge according to the edge confidence degree mask；It is double by combining
Side medium filtering is filtered initial parallax figure, and the pixel and parallax confidence level for obtaining non-edge are less than predetermined threshold value
Pixel parallax value；Disparity map is generated according to the parallax value of each pixel.
In order to achieve the above objectives, another aspect of the present invention embodiment proposes a kind of image based on optical field acquisition device and surpasses
Resolution ratio device, the optical field acquisition device include multiple USB cameras and camera, to form a light with 3 × 3 visual angles
Field harvester, the multiple USB camera at side view angle with the space form regular array of square and are centered around described
Around camera, wherein, described device includes：Acquisition module, for gathering light field image by the optical field acquisition device, to obtain
Take multiple side view angles low-resolution image and intermediate visual angle high-definition picture；Super-resolution module, for utilizing dictionary learning
Super-resolution is carried out to the multiple side view angle low-resolution image collected with deep learning；Acquisition module, for root
The depth information of scene is obtained according to the parallax between the light field image of multiple different visual angles.
The image super-resolution device based on optical field acquisition device of the embodiment of the present invention is believed by single-view and various visual angles
Breath, prediction recovers the high frequency section of input picture, and using the information of various visual angles high-definition picture, further calculates scene
Depth, available for the situation that scene rebuilding, large scene monitor, greatly reduce cost of manufacture, ensure space and angle point
The accuracy of resolution can realize that light-field camera is focused again.
In addition, the image super-resolution device according to the above embodiment of the present invention based on optical field acquisition device can also have
There is following additional technical characteristic：
Further, in one embodiment of the invention, the super-resolution module, further comprises：Computing unit,
For the intermediate visual angle high-definition picture is down-sampled, it is passed through into trained convolution using the deep learning method
Neutral net carries out the super-resolution at the visual angle, residual image is obtained with down-sampled preceding image subtraction, to reflect neutral net
The residual error of super-resolution；Extraction unit, for by the multiple side view angle low-resolution image and the intermediate high resolution graphics
As extracting dictionary information using image block, and it is corresponding with the information of high-definition picture block to do low-resolution image block；First
Acquiring unit, for each side view angle low-resolution image to be carried out preliminary super-resolution, profit by the convolutional neural networks
With information correspondence, the residual image is transformed into side view angle, with the super-resolution tentatively obtained Jing Guo neutral net
Results added, to obtain final super-resolution result.
Further, in one embodiment of the invention, the corresponding residual plot in each position is obtained by the following formula
As block：
Wherein,For the high-definition picture block that side view angle is rebuild, k is high resolution graphics among closest 9 chosen
As the index of block, ωkFor k-th of weight,Among required k-th of side view angle full resolution pricture block of j-th of formation
Full resolution pricture block, R are the value on intermediate high-definition picture, and j is the index of side view angle image block.
Further, in one embodiment of the invention, the final super-resolution result is：
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) be by neutral net into
Capable image block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
Further, in one embodiment of the invention, the acquisition module, further comprises：Second acquisition unit,
For obtaining the edge confidence degree of every light field image, to obtain edge confidence degree mask；3rd acquiring unit, for according to institute
State the parallax value that edge confidence degree mask obtains being marked as the pixel at confidence edge；4th acquiring unit, for passing through connection
Bilateral medium filtering is closed to be filtered initial parallax figure, obtain the pixel of non-edge and parallax confidence level be less than it is default
The parallax value of the pixel of threshold value；Generation unit, for generating disparity map according to the parallax value of each pixel.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
It obtains substantially or is recognized by the practice of the present invention.
Description of the drawings
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments
Substantially and it is readily appreciated that, wherein：
Fig. 1 is the structure diagram according to the optical field acquisition device of one embodiment of the invention；
Fig. 2 is the flow chart according to the image super-resolution method based on optical field acquisition device of the embodiment of the present invention；
Fig. 3 is the flow according to the three-dimensional rebuilding method of the super-resolution optical field acquisition device of one embodiment of the invention
Figure；
Fig. 4 is the structural representation according to the image super-resolution device based on optical field acquisition device of the embodiment of the present invention
Figure.
Specific embodiment
The embodiment of the present invention is described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end
Same or similar label represents same or similar element or has the function of same or like element.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and is not considered as limiting the invention.
Before the image super-resolution method and device based on optical field acquisition device of the embodiment of the present invention is introduced, first
Introduce the optical field acquisition device of the embodiment of the present invention.
Optical field acquisition device as shown in Figure 1 includes multiple USB cameras and camera, has 3 × 3 visual angles to form one
Optical field acquisition device, multiple USB cameras at side view angle with the space form regular array of square and are centered around camera week
It encloses.
Specifically, optical field acquisition device can include：8 amateur low quality USB cameras, 1 high resolution camera,
1 customization aluminum frame.Wherein, 8 low quality cameras at side view angle with square regular array around, with adjacent two on one side
The distance of a camera is 60mm, and 9 photographic devices form the sparse optical field acquisition device at 3 × 3 visual angles, for high-resolution phase
Canon's 600D slr cameras may be employed in machine, the embodiment of the present invention.
Before using the optical field acquisition device of embodiment of the present invention acquisition image, first have to set the coke of all cameras
Point, and calibrate the intrinsic parameter of whole device system；By each side view angle image projection to reference plane parallel to intermediate visual angle
In the plane of image, according to calibration result, make image rectification into light field image.So obtained all side view angles image distribution in
On 3 × 3 grids with average headway.
In addition, it is necessary to be carried out to the overlapping region of all camera the image collected in subsequent image processing flow
It cuts, after cutting is handled, the resolution ratio of the intermediate multi-view image collected by high resolution camera is 2944 × 1808, side
The resolution ratio of multi-view image is 368 × 266, is the 1/8 of intermediate visual angle image resolution ratio.
To sum up, optical field acquisition device according to embodiments of the present invention, by 8 amateur quality USB cameras and 1 high score
Resolution camera forms, and realizes the various visual angles acquisition to scene.The optical field acquisition device can be gathered by correction with higher angular
The sparse 3 D light field of resolution ratio is spent, picking rate is fast, efficient, with traditional device using polyphaser array acquisition light field
It compares, the optical field acquisition device of the embodiment of the present invention is only needed using a high quality camera, and hardware cost is low, picking rate
Soon, it is efficient, and the optical field acquisition apparatus structure is simple, it is easy to use, it is applied widely.
The image super-resolution based on optical field acquisition device proposed according to embodiments of the present invention is described with reference to the accompanying drawings
Method and device describes the image oversubscription based on optical field acquisition device proposed according to embodiments of the present invention with reference to the accompanying drawings first
Resolution method.
Fig. 2 is the flow chart of the image super-resolution method based on optical field acquisition device of the embodiment of the present invention.
Comprise the following steps as shown in Fig. 2, being somebody's turn to do the image super-resolution method based on optical field acquisition device：
In step s 201, light field image is gathered by optical field acquisition device, to obtain multiple side view angle low resolution figures
Picture and intermediate visual angle high-definition picture.
It is understood that as shown in figure 3, the embodiment of the present invention is to input 1 side view angle low-resolution image S and centre
Exemplified by the high-definition picture R of visual angle, and all 8 side view angles are respectively applied to, to gather light field figure by optical field acquisition device
Picture, to obtain multiple side view angles low-resolution image and intermediate visual angle high-definition picture.
In step S202, using dictionary learning and deep learning to multiple side view angles low-resolution image for being collected
Carry out super-resolution.
Further, in one embodiment of the invention, it is more to what is collected using dictionary learning and deep learning
A side view angle low-resolution image carries out super-resolution, further comprises：Intermediate visual angle high-definition picture is down-sampled, it utilizes
It is carried out the super-resolution at the visual angle by deep learning method by trained convolutional neural networks, with down-sampled preceding image
Subtract each other to obtain residual image, to reflect the residual error of neutral net super-resolution；By multiple side view angles low-resolution image and centre
High-definition picture extracts dictionary information using image block, and does the information of low-resolution image block and high-definition picture block
It is corresponding；Each side view angle low-resolution image is subjected to preliminary super-resolution by convolutional neural networks, is corresponded to and closed using information
System, side view angle is transformed by residual image, and the super-resolution results added tentatively obtained Jing Guo neutral net, with acquisition most
Whole super-resolution result.
It is understood that the embodiment of the present invention can be by the low-resolution image at 8 visual angles around and intermediate high-resolution
Rate imagery exploitation image block extracts dictionary information, and it is corresponding with the information of high-definition picture block to do low-resolution image block.
Specifically, the dictionary information of intermediate visual angle high-definition picture R is DR={ fR,1,…,fR,N, wherein fR,i(i=
1,2 ..., N) for the first gradient and the second gradient result of image block extracted from R.Similarly, the residual image of correspondence position
RERRDictionary information can also be obtained with identical method, be denoted as { eR,1,…,eR,N}.For the low-resolution image at side view angle
In each tile location j, calculate first gradient and the second gradient, and in DRIt is middle to use L29 neighbours are calculated in norm distance
It is denoted asSo as to obtain corresponding 9 features in residual error dictionary, it is denoted as
Further, in one embodiment of the invention, the corresponding residual plot in each position is obtained by the following formula
As block：
Wherein,For the high-definition picture block that side view angle is rebuild, k is high resolution graphics among closest 9 chosen
As the index of block, ωkFor k-th of weight,Among required k-th of side view angle full resolution pricture block of j-th of formation
Full resolution pricture block, R are the value on intermediate high-definition picture, and j is the index of side view angle image block.
Further, in one embodiment of the invention, final super-resolution result is：
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) be by neutral net into
Capable image block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
It is understood that the embodiment of the present invention can be passed through for the low-resolution image at each side view angle
Convolutional neural networks carry out preliminary super-resolution, and using information correspondence, residual image is transformed into side view angle, with passing through god
The super-resolution results added tentatively obtained through network, obtains final super-resolution result.
Specifically, according to residual error candidate's dictionaryEach position j in S is obtained using weighted average
Corresponding residual image blockWherein,So as to be estimated using the method for dictionary learning
Meter has obtained the corresponding residual images of S, is denoted as SERR.Then by former side view angle low-resolution image by convolutional neural networks simultaneously
It is added with residual image, finally obtains the light field image after super-resolution, i.e.,
In step S203, the depth information of scene is obtained according to the parallax between the light field image of multiple different visual angles.
Further, in one embodiment of the invention, according to the parallax between the light field image of multiple different visual angles
The depth information of scene is obtained, is further comprised：The edge confidence degree of every light field image is obtained, is covered with obtaining edge confidence degree
Film；Obtain being marked as the parallax value of the pixel at confidence edge according to edge confidence degree mask；It is filtered by combining bilateral intermediate value
Ripple is filtered initial parallax figure, obtains the pixel of non-edge and parallax confidence level is less than the pixel of predetermined threshold value
Parallax value；Disparity map is generated according to the parallax value of each pixel.
It is understood that the light field depth information of the embodiment of the present invention calculates, following steps are specifically included：
Step 1：The edge confidence degree of every image is calculated, to obtain edge confidence degree mask；
Step 2：The parallax value of the pixel at confidence edge is marked as according to the calculating of edge confidence degree mask；
Step 3：Initial parallax figure is filtered by combining bilateral medium filtering；
Step 4:The pixel and parallax confidence level of calculating non-edge are less than the parallax value of the pixel of predetermined threshold value；
Step 5：Disparity map is generated according to the parallax value of each pixel.
The image super-resolution method based on optical field acquisition device proposed according to embodiments of the present invention, by single-view and
Various visual angles information, prediction recovers the high frequency section of input picture, and utilizes the information of various visual angles high-definition picture, further
The depth of scene is calculated, available for the situation that scene rebuilding, large scene monitor, cost of manufacture greatly reduces, ensures space
With the accuracy of angular resolution, it can realize that light-field camera is focused again.
The image super-resolution based on optical field acquisition device proposed according to embodiments of the present invention referring next to attached drawing description
Device.
Fig. 4 is the structure diagram of the image super-resolution device based on optical field acquisition device of the embodiment of the present invention.
As shown in figure 4, being somebody's turn to do the image super-resolution device 10 based on optical field acquisition device includes：Acquisition module 100, oversubscription
Resolution module 200 and acquisition module 300.
Wherein, acquisition module 100 is used to gather light field image by optical field acquisition device, to obtain multiple low point of side view angles
Resolution image and intermediate visual angle high-definition picture.Super-resolution module 200 is used for using dictionary learning and deep learning to institute
The multiple side view angles low-resolution image collected carries out super-resolution.Acquisition module 300 is used for according to multiple different visual angles
Parallax between light field image obtains the depth information of scene.The device 10 of the embodiment of the present invention can regard more by single-view and
Angle information, prediction recovers the high frequency section of input picture, and using the information of various visual angles high-definition picture, further calculates
The depth of scene reduces cost of manufacture, ensures the accuracy of space and angular resolution, can realize that light-field camera is focused again.
Further, in one embodiment of the invention, super-resolution module further comprises：Computing unit is used for
Intermediate visual angle high-definition picture is down-sampled, it is carried out by trained convolutional neural networks using deep learning method
The super-resolution at the visual angle obtains residual image, to reflect the residual of neutral net super-resolution with down-sampled preceding image subtraction
Difference；Extraction unit, for multiple side view angles low-resolution image and intermediate high-definition picture to be extracted word using image block
Allusion quotation information, and it is corresponding with the information of high-definition picture block to do low-resolution image block；First acquisition unit, for by each side
Visual angle low-resolution image carries out preliminary super-resolution by convolutional neural networks, using information correspondence, by residual image
Side view angle is transformed into, and the super-resolution results added tentatively obtained Jing Guo neutral net, to obtain final super-resolution knot
Fruit.
Further, in one embodiment of the invention, the corresponding residual plot in each position is obtained by the following formula
As block：
Wherein,For the high-definition picture block that side view angle is rebuild, k is high resolution graphics among closest 9 chosen
As the index of block, ωkFor k-th of weight,Among required k-th of side view angle full resolution pricture block of j-th of formation
Full resolution pricture block, R are the value on intermediate high-definition picture, and j is the index of side view angle image block.
Further, in one embodiment of the invention, final super-resolution result is：
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) be by neutral net into
Capable image block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
Further, in one embodiment of the invention, acquisition module further comprises：Second acquisition unit is used for
The edge confidence degree of every light field image is obtained, to obtain edge confidence degree mask；3rd acquiring unit, for being put according to edge
Reliability mask obtains being marked as the parallax value of the pixel at confidence edge；4th acquiring unit, for pass through combine it is bilateral in
Value filtering is filtered initial parallax figure, obtains the pixel of non-edge and parallax confidence level is less than the picture of predetermined threshold value
The parallax value of vegetarian refreshments；Generation unit, for generating disparity map according to the parallax value of each pixel.
It should be noted that the foregoing explanation to the image super-resolution method embodiment based on optical field acquisition device
The image super-resolution device based on optical field acquisition device of the embodiment is also applied for, details are not described herein again.
The image super-resolution based on optical field acquisition device proposed according to embodiments of the present invention by single-view and regards more
Angle information, prediction recovers the high frequency section of input picture, and using the information of various visual angles high-definition picture, further calculates
The depth of scene available for the situation that scene rebuilding, large scene monitor, reduces cost of manufacture, ensures space and angle-resolved
The accuracy of rate can realize that light-field camera is focused again.
In the description of the present invention, it is to be understood that term " " center ", " longitudinal direction ", " transverse direction ", " length ", " width ",
" thickness ", " on ", " under ", "front", "rear", "left", "right", " vertical ", " level ", " top ", " bottom " " interior ", " outer ", " up time
The orientation or position relationship of the instructions such as pin ", " counterclockwise ", " axial direction ", " radial direction ", " circumferential direction " be based on orientation shown in the drawings or
Position relationship is for only for ease of the description present invention and simplifies description rather than instruction or imply that signified device or element must
There must be specific orientation, with specific azimuth configuration and operation, therefore be not considered as limiting the invention.
In addition, term " first ", " second " are only used for description purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, " multiple " are meant that at least two, such as two, three
It is a etc., unless otherwise specifically defined.
In the present invention, unless otherwise clearly defined and limited, term " installation ", " connected ", " connection ", " fixation " etc.
Term should be interpreted broadly, for example, it may be being fixedly connected or being detachably connected or integral；Can be that machinery connects
It connects or is electrically connected；It can be directly connected, can also be indirectly connected by intermediary, can be in two elements
The connection in portion or the interaction relationship of two elements, unless otherwise restricted clearly.For those of ordinary skill in the art
For, the concrete meaning of above-mentioned term in the present invention can be understood as the case may be.
In the present invention, unless otherwise clearly defined and limited, fisrt feature can be with "above" or "below" second feature
It is that the first and second features contact directly or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature directly over second feature or oblique upper or be merely representative of
Fisrt feature level height is higher than second feature.Fisrt feature second feature " under ", " lower section " and " below " can be
One feature is immediately below second feature or obliquely downward or is merely representative of fisrt feature level height less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment of the present invention or example.In the present specification, schematic expression of the above terms is not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It is combined in an appropriate manner in one or more embodiments or example.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the different embodiments described in this specification or example and different embodiments or exemplary feature
It closes and combines.
Although the embodiment of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is impossible to limitation of the present invention is interpreted as, those of ordinary skill in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changes, replacing and modification.
Claims (10)
1. a kind of image super-resolution method based on optical field acquisition device, which is characterized in that the optical field acquisition device includes
Multiple USB cameras and camera, to form an optical field acquisition device with 3 × 3 visual angles, in the multiple of side view angle
USB camera is with the space form regular array of square and is centered around around the camera, wherein, the described method includes：
Light field image is gathered by the optical field acquisition device, it is high to obtain multiple side view angles low-resolution image and intermediate visual angle
Image in different resolution；
Super-resolution is carried out to the multiple side view angle low-resolution image collected using dictionary learning and deep learning；
And
The depth information of scene is obtained according to the parallax between the light field image of multiple different visual angles.
2. the image super-resolution method according to claim 1 based on optical field acquisition device, which is characterized in that the profit
Super-resolution is carried out to the multiple side view angle low-resolution image collected with dictionary learning and deep learning, further
Including：
The intermediate visual angle high-definition picture is down-sampled, it is passed through into trained convolution using the deep learning method
Neutral net carries out the super-resolution at the visual angle, residual image is obtained with down-sampled preceding image subtraction, to reflect neutral net
The residual error of super-resolution；
The multiple side view angle low-resolution image and the intermediate high-definition picture are believed using image block extraction dictionary
Breath, and it is corresponding with the information of high-definition picture block to do low-resolution image block；
Each side view angle low-resolution image is subjected to preliminary super-resolution by the convolutional neural networks, is corresponded to using information
The residual image is transformed into side view angle by relation, and the super-resolution results added tentatively obtained Jing Guo neutral net, with
Obtain final super-resolution result.
3. the image super-resolution method according to claim 2 based on optical field acquisition device, which is characterized in that by with
Lower formula obtains the corresponding residual image block in each position：
<mrow>
<mover>
<msub>
<mi>e</mi>
<mi>j</mi>
</msub>
<mo>^</mo>
</mover>
<mo>=</mo>
<mfrac>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mn>9</mn>
</msubsup>
<msub>
<mi>&omega;</mi>
<mi>k</mi>
</msub>
<msubsup>
<mi>e</mi>
<mrow>
<mi>R</mi>
<mo>,</mo>
<mi>k</mi>
</mrow>
<mi>j</mi>
</msubsup>
</mrow>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mn>9</mn>
</msubsup>
<msub>
<mi>&omega;</mi>
<mi>k</mi>
</msub>
</mrow>
</mfrac>
<mo>,</mo>
</mrow>
Wherein,For the high-definition picture block that side view angle is rebuild, k is high-definition picture block among closest 9 chosen
Index, ωkFor k-th of weight,To form required k-th intermediate high score of j-th of side view angle full resolution pricture block
Distinguish image block, R is the value on intermediate high-definition picture, and j is the index of side view angle image block.
4. the image super-resolution method according to claim 3 based on optical field acquisition device, which is characterized in that it is described most
Whole super-resolution result is：
<mrow>
<msup>
<mover>
<mi>S</mi>
<mo>^</mo>
</mover>
<mrow>
<mi>H</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>=</mo>
<msub>
<mi>f</mi>
<mrow>
<mi>C</mi>
<mi>N</mi>
<mi>N</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<msup>
<mi>S</mi>
<mrow>
<mi>L</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>)</mo>
</mrow>
<mo>+</mo>
<msup>
<mi>S</mi>
<mrow>
<mi>E</mi>
<mi>R</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>,</mo>
</mrow>
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) it is the figure carried out by neutral net
As block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
5. according to image super-resolution method of the claim 1-4 any one of them based on optical field acquisition device, feature exists
In the parallax between the light field image according to multiple different visual angles obtains the depth information of scene, further comprises：
The edge confidence degree of every light field image is obtained, to obtain edge confidence degree mask；
Obtain being marked as the parallax value of the pixel at confidence edge according to the edge confidence degree mask；
Initial parallax figure is filtered by combining bilateral medium filtering, obtains the pixel of non-edge and parallax confidence
Degree is less than the parallax value of the pixel of predetermined threshold value；And
Disparity map is generated according to the parallax value of each pixel.
6. a kind of image super-resolution device based on optical field acquisition device, which is characterized in that the optical field acquisition device includes
Multiple USB cameras and camera, to form an optical field acquisition device with 3 × 3 visual angles, in the multiple of side view angle
USB camera is with the space form regular array of square and is centered around around the camera, wherein, described device includes：
Acquisition module, for gathering light field image by the optical field acquisition device, to obtain multiple side view angle low resolution figures
Picture and intermediate visual angle high-definition picture；
Super-resolution module, for using dictionary learning and deep learning to the multiple side view angle low resolution that is collected
Image carries out super-resolution；And
Acquisition module obtains the depth information of scene for the parallax between the light field image according to multiple different visual angles.
7. the image super-resolution device according to claim 6 based on optical field acquisition device, which is characterized in that described super
Module resolution further comprises：
Computing unit for the intermediate visual angle high-definition picture is down-sampled, is led to using the deep learning method
The super-resolution that trained convolutional neural networks carry out the visual angle is crossed, residual image is obtained with down-sampled preceding image subtraction,
To reflect the residual error of neutral net super-resolution；
Extraction unit, for the multiple side view angle low-resolution image and the intermediate high-definition picture to be utilized image point
Block extracts dictionary information, and it is corresponding with the information of high-definition picture block to do low-resolution image block；
First acquisition unit, for each side view angle low-resolution image to be carried out preliminary oversubscription by the convolutional neural networks
The residual image using information correspondence, is transformed into side view angle by resolution, with by neutral net tentatively obtain it is super
Resolution ratio results added, to obtain final super-resolution result.
8. the image super-resolution device according to claim 7 based on optical field acquisition device, which is characterized in that by with
Lower formula obtains the corresponding residual image block in each position：
<mrow>
<mover>
<msub>
<mi>e</mi>
<mi>j</mi>
</msub>
<mo>^</mo>
</mover>
<mo>=</mo>
<mfrac>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mn>9</mn>
</msubsup>
<msub>
<mi>&omega;</mi>
<mi>k</mi>
</msub>
<msubsup>
<mi>e</mi>
<mrow>
<mi>R</mi>
<mo>,</mo>
<mi>k</mi>
</mrow>
<mi>j</mi>
</msubsup>
</mrow>
<mrow>
<msubsup>
<mi>&Sigma;</mi>
<mrow>
<mi>k</mi>
<mo>=</mo>
<mn>1</mn>
</mrow>
<mn>9</mn>
</msubsup>
<msub>
<mi>&omega;</mi>
<mi>k</mi>
</msub>
</mrow>
</mfrac>
<mo>,</mo>
</mrow>
Wherein,For the high-definition picture block that side view angle is rebuild, k is high-definition picture block among closest 9 chosen
Index, ωkFor k-th of weight,To form required k-th intermediate high score of j-th of side view angle full resolution pricture block
Distinguish image block, R is the value on intermediate high-definition picture, and j is the index of side view angle image block.
9. the image super-resolution device according to claim 8 based on optical field acquisition device, which is characterized in that it is described most
Whole super-resolution result is：
<mrow>
<msup>
<mover>
<mi>S</mi>
<mo>^</mo>
</mover>
<mrow>
<mi>H</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>=</mo>
<msub>
<mi>f</mi>
<mrow>
<mi>C</mi>
<mi>N</mi>
<mi>N</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<msup>
<mi>S</mi>
<mrow>
<mi>L</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>)</mo>
</mrow>
<mo>+</mo>
<msup>
<mi>S</mi>
<mrow>
<mi>E</mi>
<mi>R</mi>
<mi>R</mi>
</mrow>
</msup>
<mo>,</mo>
</mrow>
Wherein,For the super-resolution image block of reconstruction, HR is super-resolution, fCNN(SLR) it is the figure carried out by neutral net
As block super-resolution, SERRResidual image is obtained for down-sampled preceding image subtraction, LR represents low resolution.
10. according to image super-resolution device of the claim 6-9 any one of them based on optical field acquisition device, feature exists
In the acquisition module further comprises：
Second acquisition unit, for obtaining the edge confidence degree of every light field image, to obtain edge confidence degree mask；
3rd acquiring unit, for obtaining being marked as the parallax of the pixel at confidence edge according to the edge confidence degree mask
Value；
4th acquiring unit combines bilateral medium filtering initial parallax figure is filtered for passing through, and obtains non-edge
Pixel and parallax confidence level be less than predetermined threshold value pixel parallax value；And
Generation unit, for generating disparity map according to the parallax value of each pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711474559.8A CN108074218B (en) | 2017-12-29 | 2017-12-29 | Image super-resolution method and device based on light field acquisition device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711474559.8A CN108074218B (en) | 2017-12-29 | 2017-12-29 | Image super-resolution method and device based on light field acquisition device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108074218A true CN108074218A (en) | 2018-05-25 |
| CN108074218B CN108074218B (en) | 2021-02-23 |
Family
ID=62156067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711474559.8A Active CN108074218B (en) | 2017-12-29 | 2017-12-29 | Image super-resolution method and device based on light field acquisition device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108074218B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108876907A (en) * | 2018-05-31 | 2018-11-23 | 大连理工大学 | A kind of active three-dimensional rebuilding method of object-oriented object |
| CN109302600A (en) * | 2018-12-06 | 2019-02-01 | 成都工业学院 | A kind of stereo scene filming apparatus |
| CN109447919A (en) * | 2018-11-08 | 2019-03-08 | 电子科技大学 | In conjunction with the light field super resolution ratio reconstruction method of multi-angle of view and semantic textural characteristics |
| CN110555800A (en) * | 2018-05-30 | 2019-12-10 | 北京三星通信技术研究有限公司 | image processing apparatus and method |
| CN111630840A (en) * | 2018-08-23 | 2020-09-04 | 深圳配天智能技术研究院有限公司 | Super-resolution image acquisition method and device and image sensor |
| WO2020207172A1 (en) * | 2019-04-09 | 2020-10-15 | 深圳市视觉动力科技有限公司 | Method and system for optical monitoring of unmanned aerial vehicles based on three-dimensional light field technology |
| CN111800560A (en) * | 2019-04-08 | 2020-10-20 | 豪威科技股份有限公司 | Light field camera and method using wafer level integration processes |
| CN111818298A (en) * | 2020-06-08 | 2020-10-23 | 北京航空航天大学 | High-definition video monitoring system and method based on light field |
| CN111815513A (en) * | 2020-06-09 | 2020-10-23 | 四川虹美智能科技有限公司 | Infrared image acquisition method and device |
| CN111815515A (en) * | 2020-07-01 | 2020-10-23 | 成都智学易数字科技有限公司 | Object three-dimensional drawing method based on medical education |
| CN112037136A (en) * | 2020-09-18 | 2020-12-04 | 中国科学院国家天文台南京天文光学技术研究所 | Super-resolution imaging method based on aperture modulation |
| CN112102165A (en) * | 2020-08-18 | 2020-12-18 | 北京航空航天大学 | Light field image angular domain super-resolution system and method based on zero sample learning |
| WO2021121037A1 (en) * | 2019-12-16 | 2021-06-24 | 首都师范大学 | Method and system for reconstructing light field by applying depth sampling |
| CN113220251A (en) * | 2021-05-18 | 2021-08-06 | 北京达佳互联信息技术有限公司 | Object display method, device, electronic equipment and storage medium |
| CN113538307A (en) * | 2021-06-21 | 2021-10-22 | 陕西师范大学 | Synthetic aperture imaging method based on multi-view super-resolution depth network |
| CN113592716A (en) * | 2021-08-09 | 2021-11-02 | 上海大学 | Super-resolution method, system, terminal and storage medium for light field image spatial domain |
| WO2023231138A1 (en) * | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838568A (en) * | 2012-11-26 | 2014-06-04 | 诺基亚公司 | Method, apparatus and computer program product for generating super-resolved images |
| CN104050662A (en) * | 2014-05-30 | 2014-09-17 | 清华大学深圳研究生院 | Method for directly obtaining depth image through light field camera one-time imaging |
| CN105023275A (en) * | 2015-07-14 | 2015-11-04 | 清华大学 | Super-resolution light field acquisition device and three-dimensional reconstruction method thereof |
| CN105551050A (en) * | 2015-12-29 | 2016-05-04 | 深圳市未来媒体技术研究院 | Optical field based image depth estimation method |
-
2017
- 2017-12-29 CN CN201711474559.8A patent/CN108074218B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103838568A (en) * | 2012-11-26 | 2014-06-04 | 诺基亚公司 | Method, apparatus and computer program product for generating super-resolved images |
| CN104050662A (en) * | 2014-05-30 | 2014-09-17 | 清华大学深圳研究生院 | Method for directly obtaining depth image through light field camera one-time imaging |
| CN105023275A (en) * | 2015-07-14 | 2015-11-04 | 清华大学 | Super-resolution light field acquisition device and three-dimensional reconstruction method thereof |
| CN105551050A (en) * | 2015-12-29 | 2016-05-04 | 深圳市未来媒体技术研究院 | Optical field based image depth estimation method |
Non-Patent Citations (4)
| Title |
|---|
| VINCENT MASSELUS: "Relighting with 4D Incident Light Fields", 《ACM DIGITAL LIBRARY》 * |
| 张驰等: "光场成像技术及其在计算机视觉中的应用", 《中国图像图形学报》 * |
| 李梦娜: "全光场多视角图像超分辨率重建", 《中国优秀硕士学位论文全文数据库信息科技辑》 * |
| 肖照林: "基于相机阵列的光场成像与深度估计方法研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11967045B2 (en) | 2018-05-30 | 2024-04-23 | Samsung Electronics Co., Ltd | Image processing device and method |
| CN110555800A (en) * | 2018-05-30 | 2019-12-10 | 北京三星通信技术研究有限公司 | image processing apparatus and method |
| CN108876907A (en) * | 2018-05-31 | 2018-11-23 | 大连理工大学 | A kind of active three-dimensional rebuilding method of object-oriented object |
| CN111630840A (en) * | 2018-08-23 | 2020-09-04 | 深圳配天智能技术研究院有限公司 | Super-resolution image acquisition method and device and image sensor |
| CN109447919A (en) * | 2018-11-08 | 2019-03-08 | 电子科技大学 | In conjunction with the light field super resolution ratio reconstruction method of multi-angle of view and semantic textural characteristics |
| CN109447919B (en) * | 2018-11-08 | 2022-05-06 | 电子科技大学 | Light field super-resolution reconstruction method combining multi-view angle and semantic texture features |
| CN109302600A (en) * | 2018-12-06 | 2019-02-01 | 成都工业学院 | A kind of stereo scene filming apparatus |
| CN109302600B (en) * | 2018-12-06 | 2023-03-21 | 成都工业学院 | Three-dimensional scene shooting device |
| CN111800560B (en) * | 2019-04-08 | 2022-02-08 | 豪威科技股份有限公司 | Light field camera and method using wafer level integration processes |
| CN111800560A (en) * | 2019-04-08 | 2020-10-20 | 豪威科技股份有限公司 | Light field camera and method using wafer level integration processes |
| CN111818274A (en) * | 2019-04-09 | 2020-10-23 | 深圳市视觉动力科技有限公司 | Optical unmanned aerial vehicle monitoring method and system based on three-dimensional light field technology |
| US20220210375A1 (en) * | 2019-04-09 | 2022-06-30 | Shenzhen Vision Power Technology Co., Ltd. | Method and system for optical monitoring of unmanned aerial vehicles based on three-dimensional light field technology |
| WO2020207172A1 (en) * | 2019-04-09 | 2020-10-15 | 深圳市视觉动力科技有限公司 | Method and system for optical monitoring of unmanned aerial vehicles based on three-dimensional light field technology |
| WO2021121037A1 (en) * | 2019-12-16 | 2021-06-24 | 首都师范大学 | Method and system for reconstructing light field by applying depth sampling |
| CN111818298A (en) * | 2020-06-08 | 2020-10-23 | 北京航空航天大学 | High-definition video monitoring system and method based on light field |
| CN111818298B (en) * | 2020-06-08 | 2021-10-22 | 北京航空航天大学 | High-definition video monitoring system and method based on light field |
| CN111815513A (en) * | 2020-06-09 | 2020-10-23 | 四川虹美智能科技有限公司 | Infrared image acquisition method and device |
| CN111815515A (en) * | 2020-07-01 | 2020-10-23 | 成都智学易数字科技有限公司 | Object three-dimensional drawing method based on medical education |
| CN111815515B (en) * | 2020-07-01 | 2024-02-09 | 成都智学易数字科技有限公司 | Object three-dimensional drawing method based on medical education |
| CN112102165B (en) * | 2020-08-18 | 2022-12-06 | 北京航空航天大学 | Light field image angular domain super-resolution system and method based on zero sample learning |
| CN112102165A (en) * | 2020-08-18 | 2020-12-18 | 北京航空航天大学 | Light field image angular domain super-resolution system and method based on zero sample learning |
| CN112037136B (en) * | 2020-09-18 | 2023-12-26 | 中国科学院国家天文台南京天文光学技术研究所 | Super-resolution imaging method based on aperture modulation |
| CN112037136A (en) * | 2020-09-18 | 2020-12-04 | 中国科学院国家天文台南京天文光学技术研究所 | Super-resolution imaging method based on aperture modulation |
| CN113220251A (en) * | 2021-05-18 | 2021-08-06 | 北京达佳互联信息技术有限公司 | Object display method, device, electronic equipment and storage medium |
| CN113220251B (en) * | 2021-05-18 | 2024-04-09 | 北京达佳互联信息技术有限公司 | Object display method, device, electronic equipment and storage medium |
| CN113538307A (en) * | 2021-06-21 | 2021-10-22 | 陕西师范大学 | Synthetic aperture imaging method based on multi-view super-resolution depth network |
| CN113592716A (en) * | 2021-08-09 | 2021-11-02 | 上海大学 | Super-resolution method, system, terminal and storage medium for light field image spatial domain |
| CN113592716B (en) * | 2021-08-09 | 2023-08-01 | 上海大学 | Light field image space domain super-resolution method, system, terminal and storage medium |
| WO2023231138A1 (en) * | 2022-05-30 | 2023-12-07 | 元潼(北京)技术有限公司 | Multi-angle-of-view image super-resolution reconstruction method and apparatus based on meta-imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108074218B (en) | 2021-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108074218A (en) | Image super-resolution method and device based on optical field acquisition device | |
| Yang et al. | Dense depth posterior (ddp) from single image and sparse range | |
| CN106846463B (en) | Microscopic image three-dimensional reconstruction method and system based on deep learning neural network | |
| CN108921781B (en) | Depth-based optical field splicing method | |
| CN111343367B (en) | Billion-pixel virtual reality video acquisition device, system and method | |
| CN104376552B (en) | A kind of virtual combat method of 3D models and two dimensional image | |
| CN103337094B (en) | A kind of method of applying binocular camera and realizing motion three-dimensional reconstruction | |
| CN101443817B (en) | Method and device for determining correspondence, preferably for the three-dimensional reconstruction of a scene | |
| Jung et al. | Reconstruction of three-dimensional occluded object using optical flow and triangular mesh reconstruction in integral imaging | |
| Liang et al. | Learning non-local spatial-angular correlation for light field image super-resolution | |
| Ji et al. | SurfaceNet+: An end-to-end 3D neural network for very sparse multi-view stereopsis | |
| CN109741240A (en) | A kind of more flat image joining methods based on hierarchical clustering | |
| DE102019208216A1 (en) | Detection, 3D reconstruction and tracking of several rigid objects moving relative to one another | |
| CN113989758A (en) | Anchor guide 3D target detection method and device for automatic driving | |
| CN115082254A (en) | Lean control digital twin system of transformer substation | |
| CN112465704A (en) | Global-local adaptive optimization panoramic light field splicing method | |
| CN113436130B (en) | Intelligent sensing system and device for unstructured light field | |
| Chen et al. | Swinfsr: Stereo image super-resolution using swinir and frequency domain knowledge | |
| CN114757862A (en) | Image enhancement progressive fusion method for infrared light field equipment | |
| Chen et al. | Virtual view synthesis for 3D light-field display based on scene tower blending | |
| CN112767246A (en) | Multi-magnification spatial super-resolution method and device for light field image | |
| CN116152442A (en) | Three-dimensional point cloud model generation method and device | |
| CN114913064A (en) | Large parallax image splicing method and device based on structure keeping and many-to-many matching | |
| CN106447613A (en) | Image local registration based method and system for removing blur shadow of panorama | |
| Săftescu et al. | Look Here: Learning Geometrically Consistent Refinement of Inverse-Depth Images for 3D Reconstruction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |