WO2021185130A1 - 数字变焦方法、系统、电子设备、介质及数字成像设备 - Google Patents

数字变焦方法、系统、电子设备、介质及数字成像设备 Download PDF

Info

Publication number
WO2021185130A1
WO2021185130A1 PCT/CN2021/079929 CN2021079929W WO2021185130A1 WO 2021185130 A1 WO2021185130 A1 WO 2021185130A1 CN 2021079929 W CN2021079929 W CN 2021079929W WO 2021185130 A1 WO2021185130 A1 WO 2021185130A1
Authority
WO
WIPO (PCT)
Prior art keywords
magnification
image
zoom
multiple frames
interest
Prior art date
Application number
PCT/CN2021/079929
Other languages
English (en)
French (fr)
Inventor
冯召东
蒋彬
陈欢
Original Assignee
展讯通信（上海）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 展讯通信（上海）有限公司 filed Critical 展讯通信（上海）有限公司
Publication of WO2021185130A1 publication Critical patent/WO2021185130A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/67—Focus control based on electronic image sensor signals

Definitions

  • the present invention relates to the technical field of image processing, in particular to a digital zoom method, system, electronic equipment, storage medium and digital imaging equipment.
  • Zooming refers to the change of the focal length of the imaging lens, which is visually embodied as the zooming in or out of the imaged object, and also embodied as the zooming in or out of the imaged object.
  • the zoom capability is a very important indicator of the imaging device, which determines the shooting breadth and depth of the camera device.
  • the zoom can be divided into two types: optical zoom and digital zoom.
  • Optical zoom is the most traditional zoom method, which uses different combinations of concave and convex lenses to change the focal length.
  • Digital zoom is also called digital zoom, which is realized by cropping and interpolation algorithms, and belongs to the category of image processing.
  • optical zoom limits its application in portable photographing equipment. At present, almost all portable photographing equipment zooms are realized by digital zoom. However, digital zoom is different from optical zoom. Optical zoom is a lossy zoom, and its zoom result is often much worse than optical zoom.
  • the traditional digital zoom based on a single frame image is completely realized by interpolation algorithm.
  • the interpolation algorithm cannot recover the original information of the image, and it cannot generate more information to make the image clearer, and it will cause image noise.
  • the zoom limit of this traditional digital zoom is generally about 5 times.
  • the technical problem to be solved by the present invention is to overcome the defect of limited digital zoom capability in the prior art and provide a digital zoom method, system, electronic device, storage medium and digital imaging device.
  • a digital zoom method including:
  • the method further includes:
  • the method before the step of up-sampling the regions of interest in the multiple frames to obtain the first intermediate image, the method further includes:
  • the first magnification and the second magnification are obtained according to the zoom magnification, the zoom magnification is the product of the first magnification and the second magnification, the minimum value of the first magnification is 1, and the second magnification is An integer greater than 1;
  • the step of up-sampling the multiple frames of interest regions to obtain a first intermediate image includes:
  • first magnification as an upsampling magnification to upsample the region of interest in each frame to obtain a first intermediate image
  • the step of performing super-resolution reconstruction on the multiple frames of the first intermediate image to obtain a zoomed image includes:
  • the second magnification is used as a super-resolution reconstruction magnification to perform super-resolution reconstruction on a plurality of frames of the first intermediate image to obtain a zoomed image.
  • the step of using the second magnification as a super-resolution reconstruction magnification to perform super-resolution reconstruction on multiple frames of the first intermediate image to obtain a zoomed image includes:
  • the method further includes:
  • Iterative correction is performed on the zoomed image to obtain a final zoomed image.
  • the multiple frames of the second intermediate images are merged to obtain a zoomed image.
  • the step of up-sampling the regions of interest in multiple frames to obtain the second intermediate image includes:
  • zoom magnification as an upsampling magnification to upsample the region of interest in each frame to obtain a second intermediate image
  • the method further includes:
  • Iterative correction is performed on the zoomed image to obtain a final zoomed image.
  • the step of cropping each frame of the original image according to the zoom magnification to obtain the region of interest includes:
  • a digital zoom system including:
  • the first acquisition module is configured to continuously acquire multiple frames of original images according to a zoom instruction, where the zoom instruction includes a zoom magnification;
  • a cropping module configured to crop each frame of the original image according to the zoom magnification to obtain a region of interest
  • the first up-sampling module is configured to up-sampling the regions of interest in multiple frames to obtain the first intermediate image
  • the super-resolution reconstruction module is configured to perform super-resolution reconstruction on multiple frames of the first intermediate image to obtain a zoomed image.
  • the digital zoom system further includes:
  • a judging module for judging whether the zoom magnification is less than a preset threshold
  • the digital zoom system further includes:
  • the second acquisition module is configured to acquire a first magnification and a second magnification according to the zoom magnification, the zoom magnification is the product of the first magnification and the second magnification, and the minimum value of the first magnification is 1. ,
  • the second magnification is an integer greater than 1;
  • the first up-sampling module is specifically configured to use the first magnification as an up-sampling magnification to up-sample the region of interest in each frame to obtain a first intermediate image;
  • the super-resolution reconstruction module is specifically configured to use the second magnification as a super-resolution reconstruction magnification to perform super-resolution reconstruction on the multiple frames of the first intermediate image to obtain a zoomed image.
  • the super-resolution reconstruction module includes:
  • a determining unit configured to determine that one of the multiple frames of the first intermediate image is a reference image
  • An alignment unit configured to align the first intermediate images of the plurality of frames according to the reference image
  • An amplification unit configured to perform pixel amplification on the reference image according to the second magnification
  • a first padding unit configured to pad the pixel values of other first intermediate images except the reference image into the reference image through pixel amplification according to the alignment result of the first intermediate images of multiple frames;
  • the digital zoom system further includes:
  • the first iterative correction module is configured to iteratively correct the zoomed image acquired by the super-resolution reconstruction module to obtain a final zoomed image.
  • the digital zoom system further includes:
  • the second up-sampling module is configured to perform up-sampling on the region of interest of multiple frames to obtain a second intermediate image when the judgment module determines that it is yes;
  • the fusion module is used for fusing multiple frames of the second intermediate image to obtain a zoomed image.
  • the second up-sampling module is specifically configured to use the zoom magnification as the up-sampling magnification to up-sample the region of interest in each frame to obtain the second intermediate image;
  • the digital zoom system further includes:
  • the second iterative correction module is used to iteratively correct the zoomed image acquired by the fusion module to obtain the final zoomed image.
  • the cropping module is specifically configured to use the center of each frame of the original image as a reference point to crop each frame of the original image according to the zoom magnification to obtain the region of interest.
  • An electronic device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor.
  • the processor implements any of the above-mentioned digital zoom methods when the computer program is executed.
  • a computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned digital zoom methods are realized.
  • a digital imaging device includes any of the above-mentioned digital zoom systems.
  • the positive progress effect of the present invention is that the present invention continuously obtains multiple frames of original images according to the zoom instruction, and sequentially performs cropping, upsampling, and super-resolution reconstruction on the obtained multiple frames of original images to obtain zoomed images, rather than relying entirely on interpolation.
  • Algorithmic digital zooming of a single frame image can obtain better image resolution and better image quality, so that the present invention can achieve stronger digital zooming capabilities.
  • Fig. 1 is a flowchart of a digital zoom method according to Embodiment 1 of the present invention.
  • FIG. 2 is a specific flowchart of the digital zoom method according to Embodiment 1 of the present invention.
  • Fig. 3 is a flowchart of step S1041 in the digital zoom method according to the first embodiment of the present invention.
  • FIG. 4 is a schematic diagram of image amplification and filling in the digital zoom method according to Embodiment 1 of the present invention.
  • Fig. 5 is a flowchart of a digital zoom method according to Embodiment 2 of the present invention.
  • Fig. 6 is a schematic diagram of modules of a digital zoom system according to Embodiment 3 of the present invention.
  • FIG. 7 is a schematic diagram of modules of a digital zoom system according to Embodiment 4 of the present invention.
  • FIG. 8 is a schematic structural diagram of an electronic device according to Embodiment 5 of the present invention.
  • the digital zoom method in this embodiment includes:
  • S102 Crop each frame of the original image according to the zoom magnification to obtain a region of interest
  • S104 Perform super-resolution reconstruction on the first intermediate images of the multiple frames to obtain a zoomed image.
  • the zoom instruction includes a zoom magnification, where the minimum value of the zoom magnification is 1,
  • the device that obtains multiple frames of original images according to the zoom instruction may be, for example, a digital imaging device, and the multiple frames of original images obtained according to the zoom instruction may be It is an image acquired by a digital imaging device in real time.
  • the digital imaging device is a photographing device
  • set the zoom magnification of the photographing device, and the multiple frames of original images obtained according to the zoom instruction may be the images acquired in real time by the photographing device before the photographing action is performed, or it may be The image acquired in real time when the photographing device performs a photographing action.
  • a part of the image is cropped from the original image as the region of interest according to the zoom magnification, and the reference point for cropping the original image can be customized according to the actual application.
  • each frame of the original image can be The center of is the reference point, and each frame of the original image is cropped according to the zoom magnification to obtain the region of interest.
  • the method for up-sampling the region of interest can be customized according to actual applications. For example, interpolation (for example, bilinear interpolation, etc.), deconvolution, and up-pooling can be selected. Upsampling can be achieved by any one of them.
  • step S102 in this embodiment it may further include:
  • the zoom magnification is the product of the first magnification and the second magnification, where the minimum value of the first magnification is 1, and the second magnification is an integer greater than 1.
  • the zoom magnification is 2.5, it can be obtained
  • the first magnification is 1.25 and the second magnification is 2.
  • the second magnification obtained according to the zoom magnification is preferably the maximum value, so as to maximize the advantage of super-resolution reconstruction in integer magnification as much as possible.
  • the first magnification can be 1 and the second magnification can be 4
  • the first magnification can also be 2 and the second magnification can be 2
  • the first magnification It can also be set to 4 and the second magnification is set to 1.
  • the first magnification is preferably set to be 1, and the second magnification is set to be 4.
  • Step S103 may specifically include:
  • Step S104 may specifically include:
  • step S1041 in this embodiment may specifically include:
  • S10412 Align the first intermediate images of multiple frames according to the reference image
  • S10414 Fill the pixel values of the first intermediate images other than the reference image into the pixel-amplified reference image according to the alignment result of the multiple frames of the first intermediate image;
  • S10415 Determine whether the filled reference image includes unfilled pixels
  • step S10416 If yes, go to step S10416;
  • step S10412 multiple frames of first intermediate images are aligned, that is, image registration (Image Alignment/Image Registration) of multiple frames of first images is realized.
  • image registration Image Alignment/Image Registration
  • multiple frames of first intermediate images can be made Pixels belonging to the same object in an image correspond one-to-one.
  • step S10413 pixel amplification is performed on the reference image, where the method of pixel amplification can be customized according to the actual application. For example, zero pixel amplification can be used to achieve the reference. Specifically, the reference can be implemented based on the nearest neighbor interpolation. The image is enlarged, and then 0 is used to fill the interpolation data. Referring to Figure 4, a pixel in the reference image is enlarged to form 3*3 pixels. Among them, the pixel value of the pixel in the upper left corner is taken from the reference image. pixel.
  • the pixel values of the reference image can be compared to fill in to avoid the blur of the image caused by the motion. Further, according to the alignment result, if there are several frames of the first intermediate image pixels corresponding to the pixel amplification For the same filling area in the reference image, the filling value of the same filling area is determined by the pixel values of the pixels in the first intermediate images of the involved frames. For example, in Figure 4, the pixels of the first intermediate images All correspond to the pixel at the center of the 3*3 pixels, and then the pixel at the center can be filled according to the determined filling value. Specifically, the filling value can be calculated according to the following formula:
  • I is the padding value
  • h is the adjustable parameter
  • I k is the pixel value
  • diff k I k -I 0
  • k is the image frame number
  • k 0 is the reference image
  • step S10415 if it is judged as yes, then fill in again according to step S10416 to obtain the zoomed image. Specifically, the pixel value of the filled reference image can be compared to fill in to avoid the blur of the image caused by motion. If the judgment is no, then the zoomed image can be obtained in step S10414.
  • step S104 iterative correction may be performed on the zoomed image to obtain the final zoomed image.
  • the zoomed image obtained in step S104 can be used as the estimated value of the high-resolution image (HR), the corresponding low-resolution image (LR) is generated through the imaging model, and the error with the original data is calculated, and finally the error is minimized .
  • the number of iterations can be customized according to the calculation complexity requirements. Generally, the number of iterations can be set to a value within 10 to correct errors caused by upsampling or alignment.
  • the iterative correction implementation formula is as follows:
  • n is the number of iterations
  • Is the estimated value of the high-resolution image
  • ⁇ is the iteration step size
  • D, PSF, and Wrap k are down-sampling, the point spread function of the imaging system, and the image distortion introduced by motion, respectively.
  • the imaging model of the digital imaging device can be expressed as:
  • HR is a high-resolution image
  • Wrap ⁇ represents image deformation due to motion
  • * represents a convolution operation
  • PSF is the point spread function of the imaging system
  • D ⁇ represents the down-sampling process
  • noise is noise.
  • the first intermediate image is selected as the LR k representing the original data
  • a regular term is also introduced in the iterative correction of this embodiment To make iteration more robust, where Is the gradient operator, and ⁇ is the regularization parameter.
  • the purpose of iterative correction is to deduct the high-resolution image HR from the known multi-frame noisy low-resolution image LR:
  • multiple frames of original images are continuously acquired according to the zoom instruction, and the acquired multiple frames of original images are sequentially cropped, up-sampled, and super-resolution reconstruction is performed to obtain the zoomed image.
  • Digital zoom can achieve better image resolution and better image quality, thereby enabling stronger digital zoom capabilities.
  • this embodiment splits the zoom magnification into the first magnification and the second magnification, which not only overcomes the defect that the super-resolution reconstruction suitable for integer multiple sampling cannot be applied to the digital zoom with the zoom magnification accurate to decimal places, but also The advantages of super-resolution reconstruction in integer magnification can be fully utilized.
  • the introduction of iterative correction can also avoid image blur caused by upsampling, image alignment, and point spread function of digital imaging equipment.
  • the execution subject of the digital zoom method provided by the embodiment of the present invention may be a separate chip, a chip module or a UE (User Equipment), or may be a chip or a chip module integrated in the UE.
  • this embodiment provides a digital zoom method.
  • the digital zoom method of this embodiment may further include after step S102:
  • step S202 If yes, go to step S202; if not, go to step S105;
  • the zoom magnification in order to make full use of the super-resolution reconstruction to improve the digital zoom capability, the zoom magnification can be segmented to divide the zoom magnification into two zoom segments, a digital interpolation zoom segment and a super-resolution zoom segment.
  • the preset threshold can be customized according to actual applications.
  • the preset threshold can take a value of 2.
  • step S202 may specifically include the step of using the zoom magnification as the upsampling magnification to up-sample the region of interest in each frame to obtain the second intermediate image, that is, the size of the second intermediate image is the same as the size of the original image.
  • Step S203 may specifically include a step of aligning multiple frames of second intermediate images and a step of fusing aligned multiple frames of second intermediate images, so as to significantly reduce image noise and improve the signal-to-noise ratio of the image.
  • This embodiment may further include the step of iteratively correcting the zoomed image to obtain the final zoomed image after step S203, so as to correct errors introduced due to upsampling or alignment, and further improve the quality of the zoomed image, which is different from the first embodiment.
  • the original image is selected as the LR k representing the original data, and the regular term may not be introduced here That is:
  • the digital zoom system of this embodiment includes:
  • the first acquisition module 301 is configured to continuously acquire multiple frames of original images according to the zoom instruction;
  • the cropping module 302 is configured to crop each frame of the original image according to the zoom magnification to obtain the region of interest;
  • the first up-sampling module 303 is configured to up-sample the regions of interest in multiple frames to obtain the first intermediate image
  • the super-resolution reconstruction module 304 is configured to perform super-resolution reconstruction on the multi-frame first intermediate image to obtain a zoomed image.
  • the zoom instruction includes a zoom magnification, where the minimum value of the zoom magnification is 1,
  • the device that obtains multiple frames of original images according to the zoom instruction may be, for example, a digital imaging device, and the multiple frames of original images obtained according to the zoom instruction may be It is an image acquired by a digital imaging device in real time.
  • the digital imaging device is a photographing device
  • set the zoom magnification of the photographing device, and the multiple frames of original images obtained according to the zoom instruction may be the images acquired in real time by the photographing device before the photographing action is performed, or it may be The image acquired in real time when the photographing device performs a photographing action.
  • a part of the image is cropped from the original image as the region of interest according to the zoom magnification, and the reference point for cropping the original image can be customized according to the actual application.
  • the cropping module 302 can be set every time The center of the original frame of image is the reference point, and each frame of original image is cropped according to the zoom magnification to obtain the region of interest.
  • the method for up-sampling the region of interest can be customized according to actual applications. For example, interpolation (for example, bilinear interpolation, etc.), deconvolution, and up-pooling can be selected. Upsampling can be achieved by any one of them.
  • the digital zoom system may further include:
  • the second obtaining module 305 is configured to obtain the first magnification and the second magnification according to the zoom magnification.
  • the zoom magnification is the product of the first magnification and the second magnification, where the minimum value of the first magnification is 1, and the second magnification is an integer greater than 1.
  • the zoom magnification is 2.5, it can be obtained
  • the first magnification is 1.25 and the second magnification is 2.
  • the second magnification obtained according to the zoom magnification is preferably the maximum value, so as to maximize the advantage of super-resolution reconstruction in integer magnification as much as possible.
  • the first magnification can be 1 and the second magnification can be 4
  • the first magnification can also be 2 and the second magnification can be 2
  • the first magnification It can also be set to 4 and the second magnification is set to 1.
  • the first magnification is preferably set to be 1, and the second magnification is set to be 4.
  • the first up-sampling module 303 is specifically configured to use the first magnification as the up-sampling magnification to up-sample the region of interest in each frame to obtain the first intermediate image
  • the super-resolution reconstruction module 304 is specifically configured to use the second magnification as the The super-resolution reconstruction magnification performs super-resolution reconstruction on the first intermediate images of multiple frames to obtain a zoomed image.
  • the super-resolution reconstruction module 304 in this embodiment may specifically include:
  • the determining unit 3041 is configured to determine that one of the multiple frames of the first intermediate image is a reference image
  • the alignment unit 3042 is configured to align the first intermediate images of multiple frames according to the reference image
  • the amplification unit 3043 is configured to perform pixel amplification on the reference image according to the second magnification
  • the first padding unit 3044 is configured to pad the pixel values of the first intermediate images other than the reference image into the pixel-amplified reference image according to the alignment result of the first intermediate images of the multiple frames;
  • the judging unit 3045 is used to judge whether the filled reference image includes unfilled pixels
  • the second filling unit 3046 is called to fill the unfilled pixels according to the median filter.
  • multiple frames of first intermediate images are aligned, that is, image registration of multiple frames of first images is realized.
  • the alignment unit 3042 By aligning multiple frames of first intermediate images by the alignment unit 3042, the pixels belonging to the same object in the multiple frames of first images can be made One-to-one correspondence.
  • the amplification unit 3043 performs pixel amplification on the reference image.
  • the method of pixel amplification can be customized according to the actual application. For example, zero pixel amplification can be adopted to realize the pixel amplification.
  • the reference image can be realized first based on the nearest neighbor interpolation. Zoom in, and then use the value 0 to fill in the interpolation data.
  • the first padding unit 3044 may compare the pixel values of the reference image to perform padding to avoid blurring of the image caused by motion. Further, according to the alignment result, if there are several frames of pixels in the first intermediate image corresponding to those in the reference image If the same filling area is the same, the filling value of the same filling area is determined by the pixel values of the pixels in the first intermediate images of the several frames involved. Specifically, the filling value can be calculated according to the following formula:
  • I is the padding value
  • h is the adjustable parameter
  • I k is the pixel value
  • diff k I k -I 0
  • k is the image frame number
  • k 0 is the reference image
  • the second filling unit 3046 is called to perform filling again to obtain the zoomed image. Specifically, the pixel value of the filled reference image can be compared to perform filling to avoid image distortion caused by motion. If it is blurred, if the judgment is no, the zoomed image can be obtained after calling the second filling unit 3046.
  • the digital zoom system of this embodiment may further include a first iterative correction module 306, configured to iteratively correct the zoom image acquired by the super-resolution reconstruction module 304 to obtain a final zoom image.
  • the zoomed image acquired by the super-resolution reconstruction module 304 can be used as the estimated value of the high-resolution image (HR), the corresponding low-resolution image (LR) is generated through the imaging model, and the error with the original data is calculated. In the end, the error is minimized.
  • the number of iterations can be customized according to the calculation complexity requirements. Generally, the number of iterations can be set to a value within 10 to correct errors caused by upsampling or alignment.
  • the iterative correction implementation formula is as follows:
  • n is the number of iterations
  • Is the estimated value of the high-resolution image
  • ⁇ is the iteration step size
  • D, PSF, and Wrap k are down-sampling, the point spread function of the imaging system, and the image distortion introduced by motion, respectively.
  • the imaging model of the digital imaging device can be expressed as:
  • HR is a high-resolution image
  • Wrap ⁇ represents image deformation due to motion
  • * represents a convolution operation
  • PSF is the point spread function of the imaging system
  • D ⁇ represents the down-sampling process
  • noise is noise.
  • the first intermediate image is selected as the LR k representing the original data
  • a regular term is also introduced in the iterative correction of this embodiment To make iteration more robust, where Is the gradient operator, and ⁇ is the regularization parameter.
  • the purpose of iterative correction is to deduct the high-resolution image HR from the known multi-frame noisy low-resolution image LR:
  • multiple frames of original images are continuously acquired according to the zoom instruction, and the acquired multiple frames of original images are sequentially cropped, up-sampled, and super-resolution reconstruction is performed to obtain the zoomed image.
  • Digital zoom can achieve better image resolution and better image quality, thereby enabling stronger digital zoom capabilities.
  • this embodiment splits the zoom magnification into the first magnification and the second magnification, which not only overcomes the defect that the super-resolution reconstruction suitable for integer multiple sampling cannot be applied to the digital zoom with the zoom magnification accurate to decimal places, but also The advantages of super-resolution reconstruction in integer magnification can be fully utilized.
  • the introduction of iterative correction can also avoid image blur caused by upsampling, image alignment, and point spread function of digital imaging equipment.
  • the digital zoom system described in the embodiments of the present invention may specifically be a separate chip, a chip module, or a UE, or may be a chip or a chip module integrated in the UE.
  • the various modules/units included in the digital zoom system may be software modules/units, hardware modules/units, or part software modules/units and part hardware modules/units.
  • the various modules/units contained therein can be implemented in the form of hardware such as circuits, or at least part of the modules/units can be implemented in the form of software programs.
  • Runs on the integrated processor inside the chip, and the remaining part of the modules/units can be implemented by hardware methods such as circuits; for each device and product applied to or integrated in the chip module, each of the modules/units contained in it can use circuits, etc. It is realized by hardware, different modules/units can be located in the same component (such as chip, circuit module, etc.) or different components of the chip module, or at least part of the modules/units can be realized in the form of a software program, and the software program runs For the processor integrated inside the chip module, the remaining part of the modules/units can be implemented by hardware such as circuits; for each device and product applied to or integrated in the UE, each module/unit contained in it can be implemented by hardware such as circuits Different modules/units can be located in the same component (for example, chip, circuit module, etc.) or different components in the terminal, or at least some of the modules/units can be implemented in the form of a software program, and the software program runs on the UE The internal integrated processor, and the remaining
  • this embodiment provides a digital zoom system.
  • the digital zoom system of this embodiment may further include:
  • the judging module 401 is used to judge whether the zoom magnification is less than a preset threshold
  • call the second up-sampling module 402 If yes, call the second up-sampling module 402; if not, call the second acquisition module 305;
  • the second up-sampling module 402 is configured to up-sample the regions of interest in multiple frames to obtain a second intermediate image
  • the fusion module 403 is used for fusing multiple frames of second intermediate images to obtain a zoomed image.
  • the zoom magnification in order to make full use of the super-resolution reconstruction to improve the digital zoom capability, can be segmented to divide the zoom magnification into two zoom segments, a digital interpolation zoom segment and a super-resolution zoom segment.
  • the preset threshold can be customized according to actual applications.
  • the preset threshold can take a value of 2.
  • the second up-sampling module 402 may be specifically used to use the zoom magnification as the up-sampling magnification to up-sample the region of interest in each frame to obtain the second intermediate image, that is, the size of the second intermediate image is different from the original The dimensions of the images are the same.
  • the fusion module 403 may be specifically used to align multiple frames of second intermediate images and merge the aligned multiple frames of second intermediate images to significantly reduce image noise and improve the signal-to-noise ratio of the image.
  • the digital zoom system of this embodiment may further include a second iterative correction module 404 for iterative correction of the zoom image acquired by the fusion module 403 to obtain the final zoom image, so as to correct the problems caused by upsampling or alignment. Error, to further improve the quality of the zoomed image.
  • a second iterative correction module 404 for iterative correction of the zoom image acquired by the fusion module 403 to obtain the final zoom image, so as to correct the problems caused by upsampling or alignment. Error, to further improve the quality of the zoomed image.
  • the original image is selected as the LR k representing the original data in this embodiment, and the regular term may not be introduced here. That is:
  • the electronic device can be expressed in the form of a computing device (for example, it can be a server device), and includes a memory, a processor, and a computer program stored in the memory and running on the processor.
  • the digital zoom method provided in Embodiment 1 or 2 can be implemented when the computer program is executed.
  • FIG. 8 shows a schematic diagram of the hardware structure of this embodiment.
  • the electronic device 9 specifically includes:
  • At least one processor 91 at least one memory 92, and a bus 93 for connecting different system components (including the processor 91 and the memory 92), where:
  • the bus 93 includes a data bus, an address bus, and a control bus.
  • the memory 92 includes a volatile memory, such as a random access memory (RAM) 921 and/or a cache memory 922, and may further include a read-only memory (ROM) 923.
  • RAM random access memory
  • ROM read-only memory
  • the memory 92 also includes a program/utility tool 925 having a set of (at least one) program module 924.
  • program module 924 includes but is not limited to: an operating system, one or more application programs, other program modules, and program data. These examples Each of these or a certain combination may include the realization of a network environment.
  • the processor 91 executes various functional applications and data processing by running a computer program stored in the memory 92, such as the digital zoom method provided in Embodiment 1 or 2 of the present invention.
  • the electronic device 9 may further communicate with one or more external devices 94 (for example, keyboards, pointing devices, etc.). This communication can be performed through an input/output (I/O) interface 95.
  • the electronic device 9 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 96.
  • the network adapter 96 communicates with other modules of the electronic device 9 through the bus 93.
  • This embodiment provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the digital zoom method provided in Embodiment 1 or 2 are implemented.
  • the readable storage medium may more specifically include but not limited to: portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device or any of the above The right combination.
  • the present invention can also be implemented in the form of a program product, which includes program code.
  • program product runs on a terminal device
  • the program code is used to make the terminal device execute the implementation. Steps of the digital zoom method described in embodiment 1 or 2.
  • program code used to execute the present invention can be written in any combination of one or more programming languages, and the program code can be completely executed on the user equipment, partially executed on the user equipment, as an independent
  • the software package is executed, partly on the user’s device, partly on the remote device, or entirely on the remote device.
  • This embodiment provides a digital imaging device, which includes the digital zoom system provided in Embodiment 3 or 4.
  • the digital imaging device of this embodiment can continuously acquire multiple frames of original images according to zoom instructions, and compare the acquired multiple frames of original images. Perform cropping, up-sampling, and super-resolution reconstruction in sequence to obtain zoomed images. Compared with digital zooming of single-frame images that rely entirely on interpolation algorithms, better image resolution and better image quality can be obtained, thereby enabling Stronger digital zoom capability.
  • this embodiment splits the zoom magnification into the first magnification and the second magnification, which not only overcomes the defect that the super-resolution reconstruction suitable for integer multiple sampling cannot be applied to the digital zoom with the zoom magnification accurate to decimal places, but also The advantages of super-resolution reconstruction in integer magnification can be fully utilized.
  • the introduction of iterative correction can also avoid image blur caused by upsampling, image alignment, and point spread function of digital imaging equipment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种数字变焦方法、系统、电子设备、介质及数字成像设备。其中，数字变焦方法包括: 根据变焦指令连续获取多帧原始图像，所述变焦指令包括变焦倍率; 根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域; 对多帧感兴趣区域进行上采样以获取第一中间图像; 对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。本发明根据变焦指令连续获取多帧原始图像，并对获取到的多帧原始图像依次进行裁剪、上采样以及超分辨率重建以获取变焦图像，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得更好的图像分辨率、更好的图像质量，从而，本发明能够实现更强的数字变焦能力。

Description

数字变焦方法、系统、电子设备、介质及数字成像设备
本申请要求申请日为2020/3/19的中国专利申请2020101976172的优先权。本申请引用上述中国专利申请的全文。
技术领域
本发明涉及图像处理技术领域，尤其涉及一种数字变焦方法、系统、电子设备、存储介质及数字成像设备。
背景技术
变焦是指成像镜头焦距的改变，在视觉上体现为被成像物体的拉近或拉远，也体现为物体所成像的放大或缩小。变焦能力是成像设备的一个非常重要的指标，其决定了拍照设备的拍摄广度和深度。变焦按实现方式可分为光学变焦和数字变焦两种。光学变焦是最传统的变焦方法，其利用凹透镜与凸透镜的不同组合方式来实现焦距的改变。数字变焦也称数码变焦，其通过裁剪和插值算法实现，属于图像处理的范畴。
光学变焦的物理结构限制了其在便携式拍照设备上的应用，目前，几乎所有的便携式拍照设备的变焦都是通过数字变焦实现的。但是，数字变焦与光学变焦不同，光学变焦是一种有损变焦，其变焦结果往往比光学变焦差很多。
传统的基于单帧图像的数字变焦完全依赖插值算法实现，但是，根据采样定律，插值算法无法恢复出图像的原本信息，其亦并不能产生更多的信息使得图像更清楚，而且会导致图像噪声明显，当使用高倍率的数字变焦时还会引入明显的马赛克现象。所以，这种传统的数字变焦的变焦极限一般为5倍左右。
为了实现更好的数字变焦效果，一些基于机器学习的超分辨率算法被应用于数字变焦，但是由于学习框架不成熟或者缺乏稳定性、可靠性等原因使得基于机器学习的超分辨率算法未能广泛应用于数字变焦中。
发明内容
本发明要解决的技术问题是为了克服现有技术中数字变焦能力有限的缺陷，提供一种数字变焦方法、系统、电子设备、存储介质及数字成像设备。
本发明是通过下述技术方案来解决上述技术问题：
一种数字变焦方法，包括：
根据变焦指令连续获取多帧原始图像，所述变焦指令包括变焦倍率；
根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域；
对多帧感兴趣区域进行上采样以获取第一中间图像；
对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
较佳地，在根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域的步骤之后还包括：
判断所述变焦倍率是否小于预设阈值；
若否，则执行所述对多帧感兴趣区域进行上采样以获取第一中间图像的步骤。
较佳地，在所述多帧感兴趣区域进行上采样以获取第一中间图像的步骤之前还包括：
根据所述变焦倍率获取第一倍率和第二倍率，所述变焦倍率为所述第一倍率与所述第二倍率的乘积，所述第一倍率的最小值为1，所述第二倍率为大于1的整数；
所述多帧感兴趣区域进行上采样以获取第一中间图像的步骤包括：
将所述第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像；
所述对多帧所述第一中间图像进行与超分辨率重建以获取变焦图像的步骤包括：
将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
较佳地，所述将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像的步骤包括：
确定多帧所述第一中间图像中的一帧为参考图像；
根据所述参考图像对齐所述多帧第一中间图像；
根据所述第二倍率对所述参考图像进行像素扩增；
根据多帧第一中间图像的对齐结果将除所述参考图像之外的其他第一中间图像的像素值填补到经像素扩增的所述参考图像中；
判断经填补的所述参考图像是否包括未经填补的像素；
若是，则根据中值滤波填补未经填补的像素；
和/或，
在所述将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像的步骤之后还包括：
对所述变焦图像进行迭代校正以得到最终的变焦图像。
较佳地，当判断所述变焦倍率是否小于预设阈值的步骤判断为是时：
对多帧感兴趣区域进行上采样以获取第二中间图像；
对多帧所述第二中间图像进行融合以获取变焦图像。
较佳地，所述对多帧感兴趣区域进行上采样以获取第二中间图像的步骤包括：
将所述变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像；
和/或，
在所述对多帧所述第二中间图像进行融合以获取变焦图像的步骤之后还包括：
对所述变焦图像进行迭代校正以得到最终的变焦图像。
较佳地，所述根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域的步骤包括：
以每帧原始图像的中心为参考点根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域。
一种数字变焦系统，包括：
第一获取模块，用于根据变焦指令连续获取多帧原始图像，所述变焦指令包括变焦倍率；
裁剪模块，用于根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域；
第一上采样模块，用于对多帧感兴趣区域进行上采样以获取第一中间图像；
超分辨率重建模块，用于对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
较佳地，所述数字变焦系统还包括：
判断模块，用于判断所述变焦倍率是否小于预设阈值；
若否，则调用所述第一上采样模块。
较佳地，所述数字变焦系统还包括：
第二获取模块，用于根据所述变焦倍率获取第一倍率和第二倍率，所述变焦倍率为所述第一倍率与所述第二倍率的乘积，所述第一倍率的最小值为1，所述第二倍率为大于1的整数；
所述第一上采样模块具体用于将所述第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像；
所述超分辨率重建模块具体用于将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
较佳地，所述超分辨率重建模块包括：
确定单元，用于确定多帧所述第一中间图像中的一帧为参考图像；
对齐单元，用于根据所述参考图像对齐所述多帧第一中间图像；
扩增单元，用于根据所述第二倍率对所述参考图像进行像素扩增；
第一填补单元，用于根据多帧第一中间图像的对齐结果将除所述参考图像之外的其他第一中间图像的像素值填补到经像素扩增的所述参考图像中；
判断单元，用于判断经填补的所述参考图像是否包括未经填补的像素；
若是，则调用第二填补单元，用于根据中值滤波填补未经填补的像素；
和/或，
所述数字变焦系统还包括：
第一迭代校正模块，用于对所述超分辨率重建模块获取到的变焦图像进行迭代校正以得到最终的变焦图像。
较佳地，所述数字变焦系统还包括：
第二上采样模块，用于在所述判断模块判断为是时对多帧感兴趣区域进行上采样以获取第二中间图像；
融合模块，用于对多帧所述第二中间图像进行融合以获取变焦图像。
较佳地，所述第二上采样模块具体用于将所述变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像；
和/或，
所述数字变焦系统还包括：
第二迭代校正模块，用于对所述融合模块获取到的变焦图像进行迭代校正以得到最终的变焦图像。
较佳地，所述裁剪模块具体用于以每帧原始图像的中心为参考点根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域。
一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述任一种数字变焦方法。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一种数字变焦方法的步骤。
一种数字成像设备，包括上述任一种所述的数字变焦系统。
本发明的积极进步效果在于：本发明根据变焦指令连续获取多帧原始图像，并对获取到的多帧原始图像依次进行裁剪、上采样以及超分辨率重建以获取变焦图像，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得更好的图像分辨率、更好的图像质量，从而，本发明能够实现更强的数字变焦能力。
附图说明
图1为根据本发明实施例1的数字变焦方法的流程图。
图2为根据本发明实施例1的数字变焦方法的具体流程图。
图3为根据本发明实施例1的数字变焦方法中步骤S1041的流程图。
图4为根据本发明实施例1的数字变焦方法中图像扩增与填补的示意图。
图5为根据本发明实施例2的数字变焦方法的流程图。
图6为根据本发明实施例3的数字变焦系统的模块示意图。
图7为根据本发明实施例4的数字变焦系统的模块示意图。
图8为根据本发明实施例5的电子设备的结构示意图。
具体实施方式
下面通过实施例的方式进一步说明本发明，但并不因此将本发明限制在所述的实施例范围之中。
实施例1
本实施例提供一种数字变焦方法，参照图1，本实施例的数字变焦方法包括：
S101、根据变焦指令连续获取多帧原始图像；
S102、根据变焦倍率裁剪每帧原始图像以获取感兴趣区域；
S103、对多帧感兴趣区域进行上采样以获取第一中间图像；
S104、对多帧第一中间图像进行超分辨率重建以获取变焦图像。
在本实施例中，变焦指令包括变焦倍率，其中，变焦倍率的最小值为1，根据变焦指令获取多帧原始图像的设备例如可以是数字成像设备，根据变焦指令获取到的多帧原始图像可以是数字成像设备实时获取的图像。具体地，当数字成像设备是拍照设备时，设置拍照设备的变焦倍率，根据变焦指令获取到的多帧原始图像，既可以是拍照设备在执行拍照动作之前实时获取到的图像，也可以是在拍照设备执行拍照动作时实时获取到的图像。
在本实施例中，根据变焦倍率从原始图像中裁剪出部分图像作为感兴趣区域，其中裁剪原始图像的参考点可以根据实际应用自定义设置，例如，在本实施例中，可以每帧原始图像的中心为参考点根据变焦倍率裁剪每帧原始图像以获取感兴趣区域。此外，在本实施例中，对感兴趣区域进行上采样的方法可以根据实际应用自定义设置，例如可以选择插值(例如，双线性插值等)、反卷积(Deconvolution)、上池化中的任意一种实现上采样。
参照图2，在本实施例中步骤S102之后还可以包括：
S105、根据变焦倍率获取第一倍率和第二倍率。
在本实施例中，变焦倍率为第一倍率与第二倍率的乘积，其中，第一倍率的最小值为1，第二倍率为大于1的整数，例如，当变焦倍率为2.5时，可以获取第一倍率1.25与第二倍率2。
进一步地，在本实施例中，根据变焦倍率获取到的第二倍率优选最大值，以尽可能地最大化超分辨率重建在整数倍放大上的优势。例如，变焦倍率取值为4时，第一倍率可以取值为1并且有第二倍率取值为4，第一倍率也可以取值为2并且有第二倍率取值为2，第一倍率也可以取值为4并且有第二倍率取值为1，为了最大化超分辨率重建在整数倍放大上的优势，本实施例中优选第一倍率取值为1，第二倍率取值为4。
步骤S103具体可以包括：
S1031、将第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像。
步骤S104具体可以包括：
S1041、将第二倍率作为超分辨率重建放大倍率对多帧第一中间图像进行超分辨率重建以获取变焦图像。
参见图3，本实施例中步骤S1041具体可以包括：
S10411、确定多帧第一中间图像中的一帧为参考图像；
S10412、根据参考图像对齐多帧第一中间图像；
S10413、根据第二倍率对参考图像进行像素扩增；
S10414、根据多帧第一中间图像的对齐结果将除参考图像之外的其他第一中间图像的像素值填补到经像素扩增的参考图像中；
S10415、判断经填补的参考图像是否包括未经填补的像素；
若是，则执行步骤S10416；
S10416、根据中值滤波填补未经填补的像素。
具体地，在步骤S10412中，对齐多帧第一中间图像，也即实现多帧第一图像的图像配准(Image Alignment/Image Registration)，通过对齐多帧第一中间图像，可以使得多帧第一图像中属于同一对象的像素点一一对应起来。
在步骤S10413中，对参考图像进行像素扩增，其中，像素扩增的方式可以根据实际应用自定义设置，例如，可以采用零像素扩增来实现，具体地，可以首先基于最邻近插值实现参考图像的放大，再利用0来填充插值数据，参照图4，参考图像中的一个像素点放大后形成3*3个像素点，其中，位于左上角的像素点的像素值取自参考图像中的像素点。
在步骤S10414中，可以对比参考图像的像素值来进行填补，以避免由于运动所引起的图像的模糊，进一步地，根据对齐结果，如果存在若干帧第一中间图像的像素点对应经像素扩增的参考图像中的相同填补区域，则该相同填补区域的填补值由所涉及的若干帧第一中间图像中像素点的像素值确定，例如，在图4中，若干第一中间图像的像素点均对应到3*3个像素点中位于中心位置的像素点，继而可以根据确定的填补值来填补位于中心位置的像素点。具体地，可以根据如下公式来计算填补值：
Figure PCTCN2021079929-appb-000001
其中，I是填补值，h表示可调参数，I k表示像素值，diff k＝I k-I 0，k表示图像帧序号，k＝0表示参考图像，flag k表示该帧图像是否处于被填补位置，若是则取值为1，否则取值为0，并且，flag 0＝1。
在步骤S10415中，若判断为是，则根据步骤S10416再次进行填补以获取变焦图像，具体地，可以对比经填补的参考图像的像素值来进行填补，以避免由于运动所引起的图像的模糊，若判断为否，则步骤S10414即可获取到变焦图像。
在本实施例步骤S104之后，还可以对变焦图像进行迭代校正以得到最终的变焦图像。具体地，可以将步骤S104获取到的变焦图像作为高分辨率图像(HR)的估计值，通过成像模型生成对应的低分辨率图像(LR)，再计算与原始数据的误差，最终使得误差最小。在本实施例中，迭代次数可以依据计算复杂度要求自定义设置，通常可以将迭代次数设置为10以内的数值，以修正由于上采样或对齐所引入的误差。
具体地，迭代校正实施公式如下：
Figure PCTCN2021079929-appb-000002
其中，n是迭代次数，
Figure PCTCN2021079929-appb-000003
是高分辨率图像的估计值，β是迭代步长，D、PSF、Wrap k分别是下采样、成像系统的点扩散函数以及运动引入的图像形变。
具体地，在本实施例中，数字成像设备的成像模型可以表示为：
LR＝D{PSF*Wrap{HR}}+noise
其中，HR是高分辨图像，Wrap{}表示由于运动引入的图像变形，*表示卷积操作，PSF是成像系统的点扩散函数，D{}表示下采样过程，noise是噪声。此外，在本实施例中，选择第一中间图像作为表示原始数据的LR k，本实施例迭代校正中还引入了正则项
Figure PCTCN2021079929-appb-000004
以使得迭代更鲁棒，其中
Figure PCTCN2021079929-appb-000005
是梯度算子，λ是正则化参数。迭代校正的目的就是从已知的多帧带噪声的低分辨率图LR中反推出高分辨率图像HR：
Figure PCTCN2021079929-appb-000006
本实施例根据变焦指令连续获取多帧原始图像，并对获取到的多帧原始图像依次进行裁剪、上采样以及超分辨率重建以获取变焦图像，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得更好的图像分辨率、更好的图像质量，从而，能够实现更强的数字变焦能力。进一步地，本实施例将变焦倍率拆分为第一倍率和第二倍率，既克服了适用于整数倍采样的超分辨率重建无法应用于变焦倍率精确到小数位的数字变焦中的缺陷，又可以充分利用超分辨率重建在整数倍放大上的优势。此外，迭代校正的引入，还可以避免由于上采样、图像对齐、数字成像设备的点扩散函数等所造成的图像模糊。
需要说明的是，关于本发明实施例提供的数字变焦方法的执行主体可以为单独的芯片、芯片模组或者UE(User Equipment)，也可以是集成于UE内的芯片或者芯片模组。
实施例2
在实施例1的基础上，本实施例提供一种数字变焦方法，参照图5，本实施例的数字变焦方法在步骤S102之后还可以包括：
S201、判断变焦倍率是否小于预设阈值；
若是，则执行步骤S202；若否，则执行步骤S105；
S202、对多帧感兴趣区域进行上采样以获取第二中间图像；
S203、对多帧第二中间图像进行融合以获取变焦图像。
具体地，在本实施例中，为了充分利用超分辨率重建来提升数字变焦能力，可以对变焦倍率进行分段，以将变焦倍率分成数字插值放大段与超分辨率变焦段两个变焦段，其中，预设阈值可以根据实际应用自定义设置，例如，预设阈值可以取值为2。在本实施例中，步骤S202具体可以包括将变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像的步骤，也即，第二中间图像的尺寸与原始图像的尺寸相同。步骤S203具体可以包括对齐多帧第二中间图像的步骤以及融合对齐后的多帧第二中间图像的步骤，以显著降低图像噪声，提高图像的信噪比。
本实施例在步骤S203之后还可以包括对变焦图像进行迭代校正以得到最终的变焦图像的步骤，以修正由于上采样或对齐引入的误差，进一步提高变焦图像的质量，与实施例1中不同的是，在本实施例中选择原始图像作为表示原始数据的LR k，并且此处可以不引入正则项
Figure PCTCN2021079929-appb-000007
也即：
Figure PCTCN2021079929-appb-000008
本实施例在利用多帧图像采样信息的基础上，根据变焦倍率所处数值范围的不同而采用不同的图像处理方式，具体地，在变焦倍率较小时，采用多帧融合的方式来实现数字变焦，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得信噪比的提升，所得变焦图像的质量更好，在变焦倍率较大时，采用超分辨率重建的方式来实现数字变焦，可以获得分辨率与信噪比的提升，从而，能够实现更强的数字变焦能力。此外，迭代校正的引入，还可以避免由于上采样、图像对齐、数字成像设备的点扩散函数等所造成的图像模糊，以进一步提高变焦图像的质量。
实施例3
本实施例提供一种数字变焦系统，参照图6，本实施例的数字变焦系统包括：
第一获取模块301，用于根据变焦指令连续获取多帧原始图像；
裁剪模块302，用于根据变焦倍率裁剪每帧原始图像以获取感兴趣区域；
第一上采样模块303，用于对多帧感兴趣区域进行上采样以获取第一中间图像；
超分辨率重建模块304，用于对多帧第一中间图像进行超分辨率重建以获取变焦图像。
在本实施例中，变焦指令包括变焦倍率，其中，变焦倍率的最小值为1，根据变焦指令获取多帧原始图像的设备例如可以是数字成像设备，根据变焦指令获取到的多帧原始图像可以是数字成像设备实时获取的图像。具体地，当数字成像设备是拍照设备时，设置拍照设备的变焦倍率，根据变焦指令获取到的多帧原始图像，既可以是拍照设备在执行拍照动作之前实时获取到的图像，也可以是在拍照设备执行拍照动作时实时获取到的图像。
在本实施例中，根据变焦倍率从原始图像中裁剪出部分图像作为感兴趣区域，其中裁剪原始图像的参考点可以根据实际应用自定义设置，例如，在本实施例中，裁剪模块302可以每帧原始图像的中心为参考点根据变焦倍率裁剪每帧原始图像以获取感兴趣区域。此外，在本实施例中，对感兴趣区域进行上采样的方法可以根据实际应用自定义设置，例如可以选择插值(例如，双线性插值等)、反卷积(Deconvolution)、上池化中的任意一种实现上采样。
参照图6，在本实施例数字变焦系统还可以包括：
第二获取模块305，用于根据变焦倍率获取第一倍率和第二倍率。
在本实施例中，变焦倍率为第一倍率与第二倍率的乘积，其中，第一倍率的最小值 为1，第二倍率为大于1的整数，例如，当变焦倍率为2.5时，可以获取第一倍率1.25与第二倍率2。
进一步地，在本实施例中，根据变焦倍率获取到的第二倍率优选最大值，以尽可能地最大化超分辨率重建在整数倍放大上的优势。例如，变焦倍率取值为4时，第一倍率可以取值为1并且有第二倍率取值为4，第一倍率也可以取值为2并且有第二倍率取值为2，第一倍率也可以取值为4并且有第二倍率取值为1，为了最大化超分辨率重建在整数倍放大上的优势，本实施例中优选第一倍率取值为1，第二倍率取值为4。
基于此，第一上采样模块303具体用于将第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像，超分辨率重建模块304具体用于将第二倍率作为超分辨率重建放大倍率对多帧第一中间图像进行超分辨率重建以获取变焦图像。
参见图6，本实施例中超分辨率重建模块304具体可以包括：
确定单元3041，用于确定多帧第一中间图像中的一帧为参考图像；
对齐单元3042，用于根据参考图像对齐多帧第一中间图像；
扩增单元3043，用于根据第二倍率对参考图像进行像素扩增；
第一填补单元3044，用于根据多帧第一中间图像的对齐结果将除参考图像之外的其他第一中间图像的像素值填补到经像素扩增的参考图像中；
判断单元3045，用于判断经填补的参考图像是否包括未经填补的像素；
若是，则调用第二填补单元3046，用于根据中值滤波填补未经填补的像素。
具体地，对齐多帧第一中间图像，也即实现多帧第一图像的图像配准，通过对齐单元3042对齐多帧第一中间图像，可以使得多帧第一图像中属于同一对象的像素点一一对应起来。
扩增单元3043对参考图像进行像素扩增，其中，像素扩增的方式可以根据实际应用自定义设置，例如，可以采用零像素扩增来实现，具体地，可以首先基于最邻近插值实现参考图像的放大，再利用数值0来填充插值数据。
第一填补单元3044可以对比参考图像的像素值来进行填补，以避免由于运动所引起的图像的模糊，进一步地，根据对齐结果，如果存在若干帧第一中间图像的像素点对应参考图像中的相同填补区域，则该相同填补区域的填补值由所涉及的若干帧第一中间图像中像素点的像素值确定。具体地，可以根据如下公式来计算填补值：
Figure PCTCN2021079929-appb-000009
其中，I是填补值，h表示可调参数，I k表示像素值，diff k＝I k-I 0，k表示图像帧序号，k＝0表示参考图像，flag k表示该帧图像是否处于被填补位置，若是则取值为1，否则取值为0，并且，flag 0＝1。
在判断单元3045判断为是时，则调用第二填补单元3046再次进行填补以获取变焦图像，具体地，可以对比经填补的参考图像的像素值来进行填补，以避免由于运动所引起的图像的模糊，若判断为否，则调用第二填补单元3046之后即可获取到变焦图像。
参照图6，本实施例数字变焦系统还可包括第一迭代校正模块306，用于对超分辨率重建模块304获取到的变焦图像进行迭代校正以得到最终的变焦图像。具体地，可以将超分辨率重建模块304获取到的变焦图像作为高分辨率图像(HR)的估计值，通过成像模型生成对应的低分辨率图像(LR)，再计算与原始数据的误差，最终使得误差最小。在本实施例中，迭代次数可以依据计算复杂度要求自定义设置，通常可以将迭代次数设置为10以内的数值，以修正由于上采样或对齐所引入的误差。
具体地，迭代校正实施公式如下：
Figure PCTCN2021079929-appb-000010
其中，n是迭代次数，
Figure PCTCN2021079929-appb-000011
是高分辨率图像的估计值，β是迭代步长，D、PSF、Wrap k分别是下采样、成像系统的点扩散函数以及运动引入的图像形变。
具体地，在本实施例中，数字成像设备的成像模型可以表示为：
LR＝D{PSF*Wrap{HR}}+noise
其中，HR是高分辨图像，Wrap{}表示由于运动引入的图像变形，*表示卷积操作，PSF是成像系统的点扩散函数，D{}表示下采样过程，noise是噪声。此外，在本实施例中，选择第一中间图像作为表示原始数据的LR k，本实施例迭代校正中还引入了正则项
Figure PCTCN2021079929-appb-000012
以使得迭代更鲁棒，其中
Figure PCTCN2021079929-appb-000013
是梯度算子，λ是正则化参数。迭代校正的目的就是从已知的多帧带噪声的低分辨率图LR中反推出高分辨率图像HR：
Figure PCTCN2021079929-appb-000014
本实施例根据变焦指令连续获取多帧原始图像，并对获取到的多帧原始图像依次进 行裁剪、上采样以及超分辨率重建以获取变焦图像，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得更好的图像分辨率、更好的图像质量，从而，能够实现更强的数字变焦能力。进一步地，本实施例将变焦倍率拆分为第一倍率和第二倍率，既克服了适用于整数倍采样的超分辨率重建无法应用于变焦倍率精确到小数位的数字变焦中的缺陷，又可以充分利用超分辨率重建在整数倍放大上的优势。此外，迭代校正的引入，还可以避免由于上采样、图像对齐、数字成像设备的点扩散函数等所造成的图像模糊。
关于本发明实施例描述的数字变焦系统具体可以是单独的芯片、芯片模组或者UE，也可以是集成于UE内的芯片或者芯片模组。数字变焦系统包含的各个模块/单元，其可以是软件模块/单元，也可以是硬件模块/单元，或者也可以部分是软件模块/单元，部分是硬件模块/单元。例如，对于应用于或集成于芯片的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于芯片内部集成的处理器，剩余的部分模块/单元可以采用电路等硬件方式实现；对于应用于或集成于芯片模组的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，不同的模块/单元可以位于芯片模组的同一组件(例如芯片、电路模块等)或者不同组件中，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于芯片模组内部集成的处理器，剩余的部分模块/单元可以采用电路等硬件方式实现；对于应用于或集成于UE的各个装置、产品，其包含的各个模块/单元可以都采用电路等硬件的方式实现，不同的模块/单元可以位于终端内同一组件(例如，芯片、电路模块等)或者不同组件中，或者，至少部分模块/单元可以采用软件程序的方式实现，该软件程序运行于UE内部集成的处理器，剩余的部分模块/单元可以采用电路等硬件方式实现。
实施例4
在实施例3的基础上，本实施例提供一种数字变焦系统，参照图7，本实施例的数字变焦系统还可以包括：
判断模块401，用于判断变焦倍率是否小于预设阈值；
若是，则调用第二上采样模块402；若否，则调用第二获取模块305；
第二上采样模块402，用于对多帧感兴趣区域进行上采样以获取第二中间图像；
融合模块403，用于对多帧第二中间图像进行融合以获取变焦图像。
具体地，在本实施例中，为了充分利用超分辨率重建来提升数字变焦能力，可以对变焦倍率进行分段，以将变焦倍率分成数字插值放大段与超分辨率变焦段两个变焦段，其中，预设阈值可以根据实际应用自定义设置，例如，预设阈值可以取值为2。在本实施 例中，第二上采样模块402具体可以用于将变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像，也即，第二中间图像的尺寸与原始图像的尺寸相同。融合模块403具体可以用于对齐多帧第二中间图像以及融合对齐后的多帧第二中间图像，以显著降低图像噪声，提高图像的信噪比。
参照图7，本实施例数字变焦系统还可以包括第二迭代校正模块404，用于对融合模块403获取到的变焦图像进行迭代校正以得到最终的变焦图像，以修正由于上采样或对齐引入的误差，进一步提高变焦图像的质量，与实施例3中不同的是，在本实施例中选择原始图像作为表示原始数据的LR k，并且此处可以不引入正则项
Figure PCTCN2021079929-appb-000015
也即：
Figure PCTCN2021079929-appb-000016
本实施例在利用多帧图像采样信息的基础上，根据变焦倍率所处数值范围的不同而采用不同的图像处理方式，具体地，在变焦倍率较小时，采用多帧融合的方式来实现数字变焦，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得信噪比的提升，所得变焦图像的质量更好，在变焦倍率较大时，采用超分辨率重建的方式来实现数字变焦，可以获得分辨率与信噪比的提升，从而，能够实现更强的数字变焦能力。此外，迭代校正的引入，还可以避免由于上采样、图像对齐、数字成像设备的点扩散函数等所造成的图像模糊，以进一步提高变焦图像的质量。
实施例5
本实施例提供一种电子设备，电子设备可以通过计算设备的形式表现(例如可以为服务器设备)，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，其中处理器执行计算机程序时可以实现实施例1或2提供的数字变焦方法。
图8示出了本实施例的硬件结构示意图，如图8所示，电子设备9具体包括：
至少一个处理器91、至少一个存储器92以及用于连接不同系统组件(包括处理器91和存储器92)的总线93，其中：
总线93包括数据总线、地址总线和控制总线。
存储器92包括易失性存储器，例如随机存取存储器(RAM)921和/或高速缓存存储器922，还可以进一步包括只读存储器(ROM)923。
存储器92还包括具有一组(至少一个)程序模块924的程序/实用工具925，这样的程序模块924包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
处理器91通过运行存储在存储器92中的计算机程序，从而执行各种功能应用以及 数据处理，例如本发明实施例1或2所提供的数字变焦方法。
电子设备9进一步可以与一个或多个外部设备94(例如键盘、指向设备等)通信。这种通信可以通过输入/输出(I/O)接口95进行。并且，电子设备9还可以通过网络适配器96与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。网络适配器96通过总线93与电子设备9的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备9使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理器、外部磁盘驱动阵列、RAID(磁盘阵列)系统、磁带驱动器以及数据备份存储系统等。
应当注意，尽管在上文详细描述中提及了电子设备的若干单元/模块或子单元/模块，但是这种划分仅仅是示例性的并非强制性的。实际上，根据本申请的实施方式，上文描述的两个或更多单元/模块的特征和功能可以在一个单元/模块中具体化。反之，上文描述的一个单元/模块的特征和功能可以进一步划分为由多个单元/模块来具体化。
实施例6
本实施例提供了一种计算机可读存储介质，其上存储有计算机程序，所述程序被处理器执行时实现实施例1或2所提供的数字变焦方法的步骤。
其中，可读存储介质可以采用的更具体可以包括但不限于：便携式盘、硬盘、随机存取存储器、只读存储器、可擦拭可编程只读存储器、光存储器件、磁存储器件或上述的任意合适的组合。
在可能的实施方式中，本发明还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在终端设备上运行时，所述程序代码用于使所述终端设备执行实现实施例1或2所述的数字变焦方法的步骤。
其中，可以以一种或多种程序设计语言的任意组合来编写用于执行本发明的程序代码，所述程序代码可以完全地在用户设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户设备上部分在远程设备上执行或完全在远程设备上执行。
实施例7
本实施例提供一种数字成像设备，其包括实施例3或4提供的数字变焦系统，本实施例的数字成像设备可以根据变焦指令连续获取多帧原始图像，并对获取到的多帧原始图像依次进行裁剪、上采样以及超分辨率重建以获取变焦图像，较之完全依赖插值算法的对单帧图像的数字变焦，可以获得更好的图像分辨率、更好的图像质量，从而，能够实现更强的数字变焦能力。进一步地，本实施例将变焦倍率拆分为第一倍率和第二倍率，既克服了适用于整数倍采样的超分辨率重建无法应用于变焦倍率精确到小数位的数字变 焦中的缺陷，又可以充分利用超分辨率重建在整数倍放大上的优势。此外，迭代校正的引入，还可以避免由于上采样、图像对齐、数字成像设备的点扩散函数等所造成的图像模糊。
虽然以上描述了本发明的具体实施方式，但是本领域的技术人员应当理解，这仅是举例说明，本发明的保护范围是由所附权利要求书限定的。本领域的技术人员在不背离本发明的原理和实质的前提下，可以对这些实施方式做出多种变更或修改，但这些变更和修改均落入本发明的保护范围。

Claims (17)

  1. 一种数字变焦方法，其特征在于，包括：
    根据变焦指令连续获取多帧原始图像，所述变焦指令包括变焦倍率；
    根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域；
    对多帧感兴趣区域进行上采样以获取第一中间图像；
    对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
  2. 如权利要求1所述的数字变焦方法，其特征在于，在根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域的步骤之后还包括：
    判断所述变焦倍率是否小于预设阈值；
    若否，则执行所述对多帧感兴趣区域进行上采样以获取第一中间图像的步骤。
  3. 如权利要求1-2中至少一项所述的数字变焦方法，其特征在于，在所述多帧感兴趣区域进行上采样以获取第一中间图像的步骤之前还包括：
    根据所述变焦倍率获取第一倍率和第二倍率，所述变焦倍率为所述第一倍率与所述第二倍率的乘积，所述第一倍率的最小值为1，所述第二倍率为大于1的整数；
    所述多帧感兴趣区域进行上采样以获取第一中间图像的步骤包括：
    将所述第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像；
    所述对多帧所述第一中间图像进行与超分辨率重建以获取变焦图像的步骤包括：
    将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
  4. 如权利要求3所述的数字变焦方法，其特征在于，所述将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像的步骤包括：
    确定多帧所述第一中间图像中的一帧为参考图像；
    根据所述参考图像对齐所述多帧第一中间图像；
    根据所述第二倍率对所述参考图像进行像素扩增；
    根据多帧第一中间图像的对齐结果将除所述参考图像之外的其他第一中间图像的像素值填补到经像素扩增的所述参考图像中；
    判断经填补的所述参考图像是否包括未经填补的像素；
    若是，则根据中值滤波填补未经填补的像素；
    和/或，
    在所述将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像的步骤之后还包括：
    对所述变焦图像进行迭代校正以得到最终的变焦图像。
  5. 如权利要求2所述的数字变焦方法，其特征在于，当判断所述变焦倍率是否小于预设阈值的步骤判断为是时：
    对多帧感兴趣区域进行上采样以获取第二中间图像；
    对多帧所述第二中间图像进行融合以获取变焦图像。
  6. 如权利要求5所述的数字变焦方法，其特征在于，所述对多帧感兴趣区域进行上采样以获取第二中间图像的步骤包括：
    将所述变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像；
    和/或，
    在所述对多帧所述第二中间图像进行融合以获取变焦图像的步骤之后还包括：
    对所述变焦图像进行迭代校正以得到最终的变焦图像。
  7. 如权利要求1-6中至少一项所述的数字变焦方法，其特征在于，所述根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域的步骤包括：
    以每帧原始图像的中心为参考点根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域。
  8. 一种数字变焦系统，其特征在于，包括：
    第一获取模块，用于根据变焦指令连续获取多帧原始图像，所述变焦指令包括变焦倍率；
    裁剪模块，用于根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域；
    第一上采样模块，用于对多帧感兴趣区域进行上采样以获取第一中间图像；
    超分辨率重建模块，用于对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
  9. 如权利要求8所述的数字变焦系统，其特征在于，所述数字变焦系统还包括：
    判断模块，用于判断所述变焦倍率是否小于预设阈值；
    若否，则调用所述第一上采样模块。
  10. 如权利要求8-9中至少一项所述的数字变焦系统，其特征在于，所述数字变焦系统还包括：
    第二获取模块，用于根据所述变焦倍率获取第一倍率和第二倍率，所述变焦倍率为所述第一倍率与所述第二倍率的乘积，所述第一倍率的最小值为1，所述第二倍率为大于 1的整数；
    所述第一上采样模块具体用于将所述第一倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第一中间图像；
    所述超分辨率重建模块具体用于将所述第二倍率作为超分辨率重建放大倍率对多帧所述第一中间图像进行超分辨率重建以获取变焦图像。
  11. 如权利要求10所述的数字变焦系统，其特征在于，所述超分辨率重建模块包括：
    确定单元，用于确定多帧所述第一中间图像中的一帧为参考图像；
    对齐单元，用于根据所述参考图像对齐所述多帧第一中间图像；
    扩增单元，用于根据所述第二倍率对所述参考图像进行像素扩增；
    第一填补单元，用于根据多帧第一中间图像的对齐结果将除所述参考图像之外的其他第一中间图像的像素值填补到经像素扩增的所述参考图像中；
    判断单元，用于判断经填补的所述参考图像是否包括未经填补的像素；
    若是，则调用第二填补单元，用于根据中值滤波填补未经填补的像素；
    和/或，
    所述数字变焦系统还包括：
    第一迭代校正模块，用于对所述超分辨率重建模块获取到的变焦图像进行迭代校正以得到最终的变焦图像。
  12. 如权利要求9所述的数字变焦系统，其特征在于，所述数字变焦系统还包括：
    第二上采样模块，用于在所述判断模块判断为是时对多帧感兴趣区域进行上采样以获取第二中间图像；
    融合模块，用于对多帧所述第二中间图像进行融合以获取变焦图像。
  13. 如权利要求12所述的数字变焦系统，其特征在于，所述第二上采样模块具体用于将所述变焦倍率作为上采样倍率对每帧感兴趣区域进行上采样以获取第二中间图像；
    和/或，
    所述数字变焦系统还包括：
    第二迭代校正模块，用于对所述融合模块获取到的变焦图像进行迭代校正以得到最终的变焦图像。
  14. 如权利要求8-13中至少一项所述的数字变焦系统，其特征在于，所述裁剪模块具体用于以每帧原始图像的中心为参考点根据所述变焦倍率裁剪每帧原始图像以获取感兴趣区域。
  15. 一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的 计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7中任一项所述的数字变焦方法。
  16. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7中任一项所述的数字变焦方法的步骤。
  17. 一种数字成像设备，其特征在于，包括如权利要求8-14中任一项所述的数字变焦系统。
PCT/CN2021/079929 2020-03-19 2021-03-10 数字变焦方法、系统、电子设备、介质及数字成像设备 WO2021185130A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010197617.2 2020-03-19
CN202010197617.2A CN111447359B (zh) 2020-03-19 2020-03-19 数字变焦方法、系统、电子设备、介质及数字成像设备

Publications (1)

Publication Number Publication Date
WO2021185130A1 true WO2021185130A1 (zh) 2021-09-23

Family

ID=71654153

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079929 WO2021185130A1 (zh) 2020-03-19 2021-03-10 数字变焦方法、系统、电子设备、介质及数字成像设备

Country Status (2)

Country Link
CN (1) CN111447359B (zh)
WO (1) WO2021185130A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024124816A1 (zh) * 2022-12-15 2024-06-20 浙江宇视科技有限公司 一种数字变倍方法、装置、电子设备以及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111447359B (zh) * 2020-03-19 2021-07-02 展讯通信（上海）有限公司 数字变焦方法、系统、电子设备、介质及数字成像设备
CN114390213B (zh) * 2020-10-22 2023-04-18 华为技术有限公司 一种拍摄方法及设备
CN112954195A (zh) * 2021-01-27 2021-06-11 维沃移动通信有限公司 对焦方法、装置、电子设备及介质
CN115134633B (zh) * 2021-03-26 2024-04-26 华为技术有限公司 一种远程视频方法及相关装置
CN113570531B (zh) * 2021-07-27 2024-09-06 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113935934A (zh) * 2021-10-08 2022-01-14 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101621622A (zh) * 2008-06-30 2010-01-06 索尼株式会社 超级分辨率数字变焦
CN103986867A (zh) * 2014-04-24 2014-08-13 宇龙计算机通信科技(深圳)有限公司 一种图像拍摄终端和图像拍摄方法
US20150371369A1 (en) * 2013-06-19 2015-12-24 Qualcomm Technologies, Inc. System and method for single-frame based super resolution interpolation for digital cameras
JP2017010115A (ja) * 2015-06-17 2017-01-12 日本電信電話株式会社 画像解像度変換装置、画像解像度変換方法及びコンピュータプログラム
CN108391035A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种拍摄方法、装置与设备
CN111447359A (zh) * 2020-03-19 2020-07-24 展讯通信（上海）有限公司 数字变焦方法、系统、电子设备、介质及数字成像设备

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5326802B2 (ja) * 2009-05-19 2013-10-30 ソニー株式会社 情報処理装置、画像拡大縮小方法及びそのプログラム
CN102170571A (zh) * 2010-06-22 2011-08-31 上海盈方微电子有限公司 一种支持双通道cmos传感器的数码相机架构
US8988538B2 (en) * 2012-07-02 2015-03-24 Canon Kabushiki Kaisha Image pickup apparatus and lens apparatus
US10489887B2 (en) * 2017-04-10 2019-11-26 Samsung Electronics Co., Ltd. System and method for deep learning image super resolution
CN109495681A (zh) * 2017-09-12 2019-03-19 天津三星通信技术研究有限公司 获取图像的方法和设备
CN109886875B (zh) * 2019-01-31 2023-03-31 深圳市商汤科技有限公司 图像超分辨率重建方法及装置、存储介质
CN110490196B (zh) * 2019-08-09 2022-11-15 Oppo广东移动通信有限公司 主体检测方法和装置、电子设备、计算机可读存储介质
CN110536057B (zh) * 2019-08-30 2021-06-08 Oppo广东移动通信有限公司 图像处理方法和装置、电子设备、计算机可读存储介质
CN110650295B (zh) * 2019-11-26 2020-03-06 展讯通信（上海）有限公司 图像处理方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101621622A (zh) * 2008-06-30 2010-01-06 索尼株式会社 超级分辨率数字变焦
US20150371369A1 (en) * 2013-06-19 2015-12-24 Qualcomm Technologies, Inc. System and method for single-frame based super resolution interpolation for digital cameras
CN103986867A (zh) * 2014-04-24 2014-08-13 宇龙计算机通信科技(深圳)有限公司 一种图像拍摄终端和图像拍摄方法
JP2017010115A (ja) * 2015-06-17 2017-01-12 日本電信電話株式会社 画像解像度変換装置、画像解像度変換方法及びコンピュータプログラム
CN108391035A (zh) * 2018-03-26 2018-08-10 华为技术有限公司 一种拍摄方法、装置与设备
CN111447359A (zh) * 2020-03-19 2020-07-24 展讯通信（上海）有限公司 数字变焦方法、系统、电子设备、介质及数字成像设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024124816A1 (zh) * 2022-12-15 2024-06-20 浙江宇视科技有限公司 一种数字变倍方法、装置、电子设备以及存储介质

Also Published As

Publication number Publication date
CN111447359B (zh) 2021-07-02
CN111447359A (zh) 2020-07-24

Similar Documents

Publication Publication Date Title
WO2021185130A1 (zh) 数字变焦方法、系统、电子设备、介质及数字成像设备
WO2021017811A1 (zh) 图像处理方法和装置、电子设备、计算机可读存储介质
EP2557536B1 (en) Optical transfer function based image restoration
WO2021179590A1 (zh) 视差图的处理方法、装置、计算机设备和存储介质
US9760977B2 (en) Method and apparatus for generating a super-resolved image from a single image
CN111402139B (zh) 图像处理方法、装置、电子设备和计算机可读存储介质
JP4782899B2 (ja) 視差検出装置、測距装置及び視差検出方法
CN112767290B (zh) 图像融合方法、图像融合装置、存储介质与终端设备
JP5627256B2 (ja) 画像処理装置、撮像装置および画像処理プログラム
CN110675404A (zh) 图像处理方法、图像处理装置、存储介质与终端设备
US20040091171A1 (en) Mosaic construction from a video sequence
JP2002300459A (ja) 反復法による画像復元装置、画像復元方法、プログラム及び記録媒体
KR101140953B1 (ko) 영상 왜곡 보정 장치 및 방법
WO2023070862A1 (zh) 校正广角镜头图像畸变的方法、装置及照相设备
CN112215906A (zh) 图像处理方法、装置和电子设备
CN111553841A (zh) 一种基于最佳缝合线更新的实时视频拼接算法
Jeong et al. Multi-frame example-based super-resolution using locally directional self-similarity
CN111652800B (zh) 一种单张图像超分辨的方法及计算机可读存储介质
CN111935397A (zh) 图像的处理方法、装置、电子设备和计算机可读介质
CN114972030A (zh) 一种图像拼接方法、装置、存储介质与电子设备
US20150161771A1 (en) Image processing method, image processing apparatus, image capturing apparatus and non-transitory computer-readable storage medium
CN114494258A (zh) 镜头的像差预测和图像重建方法及装置
CN114092562A (zh) 噪声模型标定方法、图像去噪方法、装置、设备和介质
JP2005045513A (ja) 画像処理装置及び歪補正方法
CN113938578B (zh) 一种图像虚化方法、存储介质及终端设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21772637

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21772637

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21772637

Country of ref document: EP

Kind code of ref document: A1