CN113438401B - Digital zooming method, system, storage medium and terminal - Google Patents
Digital zooming method, system, storage medium and terminal Download PDFInfo
- Publication number
- CN113438401B CN113438401B CN202110740104.6A CN202110740104A CN113438401B CN 113438401 B CN113438401 B CN 113438401B CN 202110740104 A CN202110740104 A CN 202110740104A CN 113438401 B CN113438401 B CN 113438401B
- Authority
- CN
- China
- Prior art keywords
- image
- ith frame
- original
- frames
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a digital zooming method, a digital zooming system, a storage medium and a terminal, wherein the method comprises the following steps: acquiring a first original tele image under an ith frame of long-focus lens and a first original wide image under an ith frame of wide-angle lens; separating an ith frame of first foreground main body image from an ith frame of first original tele image; processing the first field angle of the ith frame of the first original wide-angle image according to the first field angle of the telephoto lens and a first change field angle preset by the second field angle of the wide-angle lens to obtain an ith frame of the first background image; fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame; the processing process is executed on the N frames of first original tele images and the N frames of first original wide-angle images to obtain N frames of first zoom images, and the first zooming is completed.
Description
Technical Field
The present invention relates to the field of camera technologies, and in particular, to a digital zoom method, a digital zoom system, a storage medium, and a terminal.
Background
Hirschk zoom is originally a professional movie production technique. This effect is to adjust an angle of view through the zoom lens while the camera is close to or far from the photographic subject, thereby keeping the photographic subject the same size throughout the screen. In its classical form, when the lens is zoomed in, the camera angle pulls away from an object and vice versa. Therefore, during zooming, there is a continuous perspective distortion, and the most obvious feature is that the background appears to change size relative to the subject, creating a more science-fictive and cool shot.
In the current implementation, in order to implement the heucher zoom, a camera generally needs to be moved by a specific device, and meanwhile, the optical zoom is matched, so that the professional performance is high, and the implementation difficulty is high.
Therefore, there is a need to provide a novel digital zoom method, system, storage medium and terminal to solve the above problems in the prior art.
Disclosure of Invention
The invention aims to provide a digital zooming method, a digital zooming system, a storage medium and a terminal, which can shoot a Hirschhorn zoom video without a mobile terminal and are more convenient to operate.
In a first aspect, to achieve the above object, the digital zooming method of the present invention includes:
acquiring a first original tele image under an ith frame of telephoto lens and a first original wide image under the ith frame of wide lens, wherein the first original tele image and the first original wide image comprise N frames of images, N is a positive integer, i is more than or equal to 1 and less than or equal to N, and i is a positive integer;
separating an ith frame of first foreground main body image from the ith frame of the first original tele image;
processing a first field angle of the first original wide-angle image of the ith frame according to a preset first variable field angle to obtain a first background image of the ith frame;
fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
and executing the processing process on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frames of the first zoom image is a second field angle under a wide-angle lens to obtain N frames of the first zoom image, and finishing the first zooming.
The digital zooming method has the advantages that: the method comprises the steps of separating a first foreground main body image from a first original telephoto image, calculating to obtain an N-frame first background image according to a second field angle of a first original wide-angle image and a first field angle of the first original telephoto image, fusing the i-frame first foreground main body image and the i-frame first background image to obtain an i-frame first zoom image, and then executing the processing process on the N-frame first original telephoto image and the N-frame first original wide-angle image, so that the N-frame first zoom image can be shot to obtain a zoom image with a continuously expanded background range, videos with different requirements can be shot, and the difficulty in shooting the zoom video is reduced.
In some possible embodiments, the separating the ith frame of the first foreground subject image from the ith frame of the first raw tele image includes:
acquiring a background image and a first foreground main body image in the first original tele image of the ith frame through a depth convolution network, and calculating a mask matrix of the first original tele image according to the proportion of the background image and the first foreground main body image in the first original tele image;
the fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain the first zoom image of the ith frame, including:
and fusing the mask matrix of the first original tele image of the ith frame, the background image, the first foreground main body image and the first background images of the N frames to obtain a first zoom image of the N frames. The beneficial effects are that: and obtaining a background image and a first foreground main body image in the first original tele image through a depth convolution network to calculate a mask matrix of the first original tele image, so as to complete image zooming in a fusion mode subsequently and ensure zooming effect.
In some possible embodiments, the mask matrix of the first raw tele image satisfies the following equation: it ═ alpha ═ F + (1-alpha) × B;
wherein It is an image matrix of the first original tele image of the N frames, alpha is a mask matrix having the same size as the image matrix, F is the first foreground subject image, and B is the background image.
In some possible embodiments, the processing the field angle of the first original wide-angle image of the ith frame according to a preset first changing field angle to obtain an ith frame first background image includes:
calculating to obtain the first variable field angle according to the field angle of the telephoto lens and the field angle of the wide-angle lens;
reducing the first field angle of the first original wide-angle image of the ith frame according to the first change field angle, so that the difference value between the field angle of the first original wide-angle image of the ith frame and the field angle of the original wide-angle image of the (i-1) th frame is the first change field angle, and obtaining a first clipping image of the ith frame;
and carrying out interpolation processing on the first cutting image of the ith frame to obtain the first background image of the ith frame. The beneficial effects are that: the method comprises the steps of reducing a first change field angle of a first original wide-angle image of an ith frame according to frames to enable the difference value of the field angle of the first original wide-angle image of the ith frame and the field angle of the original wide-angle image of an ith-1 frame to be the first change field angle, ensuring that the image background changes smoothly in the zooming process, improving the display effect of the zooming image, and simultaneously ensuring that the size of a subsequently output image is the same as that of the first original tele image in an interpolation processing mode.
In some possible embodiments, the first varying field of view satisfies the following equation: x | a1-a2|/N, where X represents a first varying field of view, a1 represents a first field of view, and a2 represents a second field of view.
In some possible embodiments, the fusing the mask matrix of the first raw tele image with the N frames of the first background image to obtain the N frames of the first zoom image satisfies the following formula: if ═ alpha ═ It + (1-alpha) × Iw;
wherein If is the first zoom image, alpha is the mask matrix of the first original tele image of the ith frame, It is the image matrix of the first original tele image of the ith frame, and Iw is the first background image of the ith frame.
In some possible embodiments, after completing the first zoom, the method further comprises:
starting to execute the second zooming after reaching the preset time interval or receiving the operation requested by the user;
acquiring an L-th frame of first zoom image and an L-th frame of second original tele image under a tele lens, wherein the second original tele image comprises M frames of images, M is a positive integer, L is more than or equal to 0 and less than or equal to M, and L is a positive integer;
separating an L frame second foreground main body image from the L frame second original tele image;
processing the first zoom image of the L-th frame according to the second change field angle to obtain a second background image of the L-th frame;
fusing the second foreground main image of the L-th frame and the second background image of the L-th frame together to obtain a second zoom image of the L-th frame;
and executing the processing process on the M frames of the second original tele image and the M frames of the first zoom image until the field angle of the M frames of the second zoom image is the first field angle under the tele lens to obtain the M frames of the second zoom image, and finishing the second zooming. The beneficial effects are that: through the process, the zoomed image is zoomed for the second time, so that the effect of reverse zooming is achieved, the change process of the background view of the zoomed image from large to small is realized, and the zooming requirements of different scenes are met.
In a second aspect, the present invention discloses a digital zoom system, the system comprising:
the image acquisition module is used for acquiring a first original tele image under an ith frame of tele lens and a first original wide image under an ith frame of wide lens, wherein the first original tele image and the first original wide image comprise N frames of images, N is a positive integer, i is more than or equal to 1 and less than or equal to N, and i is a positive integer;
the separation module is used for separating the ith frame of first foreground main body image from the ith frame of first original tele image;
the background processing module is used for processing the first original wide-angle image of the ith frame according to a preset first change field angle to obtain a first background image of the ith frame;
the fusion module is used for fusing the first foreground main body image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
and the first zooming module is used for executing the processing process on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frames of the first zooming image is a second field angle under a wide-angle lens, so as to obtain N frames of first zooming images and finish the first zooming.
The digital zoom system has the advantages that: the method comprises the steps that a separation module is used for separating a first foreground main body image from a first original telephoto image, a background processing module is used for calculating to obtain N frames of first background images according to a second field angle of the first original wide-angle image and a first field angle of the first original telephoto image, and a first zooming module fuses the first foreground main body image and the N frames of first background images to obtain N frames of first zooming images, so that the N frames of first zooming images complete a zooming process, images with continuously expanded background ranges can be shot, videos with different requirements can be shot, and the difficulty in shooting the zooming videos is reduced.
In some possible embodiments, the system further comprises:
and the second zooming module is used for fusing a second foreground main image of the M frames of second original tele images and a second background image of the M frames of first zoom images together to form M frames of second zoom images after a preset time interval is reached or a user request operation is received, so that secondary zooming is completed.
In a third aspect, the invention also discloses a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the above method.
In a fourth aspect, the present invention further provides a terminal, including: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the computer program stored in the memory to cause the terminal to perform the method described above.
The beneficial effects of the third aspect and the fourth aspect are described in the first aspect and the second aspect, and are not described herein again.
Drawings
Fig. 1 is a flowchart of a first zooming of a digital zooming method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a tele image taken under a tele lens according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a wide-angle image captured under a wide-angle lens according to an embodiment of the present invention;
fig. 4 is a flowchart of a second zooming of the digital zooming method according to the embodiment of the present invention;
fig. 5 is a block diagram of a digital zoom system according to an embodiment of the present invention;
fig. 6 is a block diagram of a terminal according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. As used herein, the word "comprising" and similar words are intended to mean that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items.
Before describing the embodiments of the present invention in detail, some terms used in the embodiments of the present invention will be explained below to facilitate understanding by those skilled in the art.
Wide-angle lens: a lens with a large view field and a small focal length can shoot a scene with a wide view field;
telephoto lens: a lens with a small field of view and a large focal length can generally shoot a scene with a small field of view and rich details.
In view of the problems in the prior art, an embodiment of the present invention provides a digital zooming method, as shown in fig. 1, the method including the following steps:
s101, acquiring a first original tele image under an ith frame of tele lens and a first original wide image under an ith frame of wide lens, wherein the first original tele image and the first original wide image comprise N frames of images, N is a positive integer, i is more than or equal to 1 and less than or equal to N, and i is a positive integer;
s102, separating a first foreground main body image of an ith frame from the first original tele image of the ith frame;
s103, processing the first field angle of the first original wide-angle image of the ith frame according to the first field angle of the telephoto lens and a first change field angle preset by the second field angle of the wide-angle lens to obtain an ith frame first background image;
s104, fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
and S105, executing the processing process on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frames of the first zoom image is a second field angle under a wide-angle lens to obtain N frames of first zoom images, and finishing the first zooming.
In the above method, in order to realize image zooming during photographing, first an original tele image under an i-th frame telephoto lens and a first original wide image under an i-th frame wide-angle lens are acquired, wherein, N frames refers to completing the first zooming process after passing through N frames of images, firstly separating a first foreground main body image from an ith frame of first original tele image, then processing the field angle of the ith frame of first original wide image to obtain N frames of first background images with gradually reduced field angles, so as to carry out image fusion on the ith frame first foreground main image and the ith frame first background image, thereby obtaining an ith frame first zoom image, and executing the processes on the N frames of first original tele images and the first original wide images to obtain N frames of first zoom images, thereby finishing the process of zooming for the first time.
In the above zooming process, fig. 2 is a telephoto image taken under the telephoto lens, fig. 3 is a wide-angle image taken under the wide-angle lens, as shown in fig. 2 and 3, where an ellipse in fig. 2 and 3 represents a subject image in the taken image, and an image other than the ellipse is a background image, and since the field angle under the telephoto lens is small, the field range of the background image taken is small, but the details of the taken subject image are more clear and rich, while the field angle under the wide-angle lens is large, the field of the background image taken is larger, but the details of the taken subject image are not clear and rich, by the above zooming process, the first foreground subject image of the first original telephoto image taken under the telephoto lens and the first background image under the processed wide-angle lens are fused during the taking, so that the obtained first zoom image combines the advantages of the telephoto image and the wide-angle image at the same time, the method has the advantages that the main body image with clear and rich details can be shot, the field angle of the shot background image can be gradually increased from small to large, the effect that the field of view of the background image is gradually enlarged is achieved, the shooting process of the Hirschhorn zoom is completed, a user can conveniently shoot a zoom image with a truer scene, the shooting operation of the user is facilitated, the precision of the shot zoom image is high, and different requirements can be met.
It should be noted that, in this embodiment, the first original tele image and the first original wide image are both used to capture a portrait image, where a first foreground subject image in the first original tele image refers to a portrait image in the first original tele image, and a subject image also refers to a portrait image.
The method realizes shooting zooming, so that a zoomed video with the size of a portrait image unchanged and the background view field changing constantly can be shot, and the shooting requirements of short videos in different styles are met.
In some embodiments, said separating a first foreground subject image from said first raw tele image comprises:
acquiring a background image and a first foreground main body image in the first original tele image through a depth convolution network;
and calculating a mask matrix of the first original tele image according to the proportion of the background image and the first foreground subject image in the first original tele image.
And then fusing the mask matrix of the first original tele image of the ith frame, the background image, the first foreground main body image and the first background images of the N frames to obtain the first zoom images of the N frames.
In the process of acquiring the background image and the first foreground main body image in the first original tele image through the depth convolution network, the first foreground main body image is the portrait image in the first original tele image, specifically, the portrait image in the first original tele image can be scratched out through an automatic portrait scratching technology.
It should be noted that, in the present application, in order to obtain the background image and the first foreground subject image in the first original telephoto image, including but not limited to using an automatic portrait matting technique, any technique capable of obtaining the background image and the first foreground subject image in the image may be applied to the present application, and the present application does not relate to improvement of the technique itself for obtaining the background image and the first foreground subject image in the image, and is not described here again.
Specifically, the human image semantic and edge detail information are learned through a deep convolution network, so that a background image and a first foreground main body image in a first original tele image are identified, based on the sizes of the identified background image and first foreground subject image, and the values of the tele image matrix of the entire first original tele image, that is, a mask matrix alpha of the same size in the first original tele image can be calculated, wherein the mask matrix alpha of the first original tele image is an arbitrary value between [0,1], when the mask matrix alpha is 0, it means that all of the first original tele image are background images, when the mask matrix alpha is 1, it means that all of the first original tele image are the first foreground subject image, and the mask matrix alpha between 0 and 1 represents the blending coefficients of the first foreground subject image and the background image in the first raw tele image.
In a further embodiment, the mask matrix of the first raw tele image satisfies the following equation: it ═ alpha ═ F + (1-alpha) × B;
wherein It is an image matrix of the first original tele image of the N frames, alpha is a mask matrix having the same size as the image matrix, F is the first foreground subject image, and B is the background image.
Through the above calculation formula, the mask matrix alpha of the first original tele image can be calculated, so as to facilitate the subsequent image fusion and complete the first zooming process.
In still other embodiments, the processing the field angle of the first original wide-angle image of the ith frame according to a preset first changing field angle to obtain an ith frame first background image includes:
calculating to obtain the first variable field angle according to the field angle of the telephoto lens and the field angle of the wide-angle lens;
according to the first change field angle, reducing the first field angle of the first original wide-angle image of the ith frame, so that the difference value between the field angle of the first original wide-angle image of the ith frame and the field angle of the original wide-angle image of the (i-1) th frame is the first change field angle, and obtaining a first clipping image of the ith frame;
and carrying out interpolation processing on the first cutting image of the ith frame to obtain the first background image of the ith frame.
The purpose of the interpolation processing is to make the size of the obtained first background image consistent with the sizes of the first original tele image and the first original wide image, so as to ensure that the front and back sizes of the output video after the final zooming is finished are consistent.
Further, the interpolation processing specifically adopts biquadratic linear interpolation, bicubic linear interpolation and any other non-linear interpolation in the prior art, and the main purpose is to implement interpolation of an image, and any interpolation method capable of implementing the interpolation purpose in the present solution in the prior art can be applied to the scheme of the present application, and is not described here again.
Specifically, for the N frames of the first original wide-angle image, first, according to the first field angle under the telephoto lens and the second field angle under the wide-angle lens, since the first zooming is completed through the N frames of images, the first variable field angle to be adjusted for each frame of image can be calculated according to the first field angle and the second field angle.
After the first change field angle is obtained, the field angle of the N frames of first original wide-angle images is adjusted according to the first change field angle until the field angle of the N frames of first original wide-angle images is adjusted to the first field angle, interpolation processing is carried out on the N frames of first cut images after the N frames of first cut images are obtained, and N frames of first background images are obtained, wherein the field angles from the first frame of first background images to the N frames of first background images are gradually reduced.
In the subsequent process of image fusion, the 1 st frame first foreground main image, the 2 nd frame first foreground main image, the 1 st frame first background image, the 2 nd frame first background image, the.
In some further embodiments, the first varying field of view satisfies the following equation: x | a1-a2|/N, where X represents a first varying field of view, a1 represents a first field of view, and a2 represents a second field of view.
Through the above calculation formula, the size of the first variable field angle can be calculated, so that the field angle of the first original wide-angle image needs to be adjusted each time.
Further, the image fusion of the first foreground subject image and the N frames of first background images to obtain N frames of first zoom images includes:
and fusing the mask matrix of the first original tele image with the N frames of first background images to obtain N frames of first zoom images.
In some other embodiments, the fusing the mask matrix of the first raw tele image with the N frames of the first background image to obtain the N frames of the first zoom image satisfies the following formula: if ═ alpha ═ It + (1-alpha) × Iw;
wherein If is the first zoom image, alpha is the mask matrix of the first original tele image of the ith frame, It is the image matrix of the first original tele image of the ith frame, and Iw is the first background image of the ith frame.
Through the calculation formula, the obtained N frames of first background images can be fused with the mask matrix, the background images and the first foreground main body images of the N frames of first original tele images in sequence to obtain N frames of first zoom images.
In some embodiments, after the first zooming is completed, fig. 4 is a flowchart of a second zooming process according to an embodiment of the present invention, and as shown in fig. 4, the method further includes:
s401, starting to execute second zooming after reaching a preset time interval or receiving a user request operation;
s402, acquiring an L-th frame of first zoom image and an L-th frame of second original tele image under a tele lens, wherein the second original tele image comprises M frames of images, M is a positive integer, L is more than or equal to 0 and less than or equal to M, and L is a positive integer;
s403, separating an L frame second foreground main image from the L frame second original tele image;
s404, processing the first zoom image of the L-th frame according to a second change field angle to obtain a second background image of the L-th frame;
s405, fusing the second foreground main image of the L-th frame and the second background image of the L-th frame together to obtain a second zoom image of the L-th frame;
and S406, executing the processing procedure on the M frames of the second original tele image and the M frames of the first zoom image until the field angle of the M frames of the second zoom image is the first field angle under the tele lens to obtain M frames of the second zoom image, and finishing the second zooming.
The above-mentioned second zooming process is equivalent to the reverse operation of the first zooming process, the first zooming is to gradually increase the background field of view in the first original tele image, i.e. to gradually increase the field angle, and the second zooming is to gradually decrease the background field of view in the image, i.e. to gradually decrease the field angle, and the process is basically the same, except that for the field angle of the M-frame first zoomed image, in the process of changing according to the second changing field angle, the field angle of the M-frame first zoomed image is sequentially decreased according to the second changing field angle until the field angle of the M-frame first zoomed image is changed to the field angle of the second original tele image, so that after the M-frame first zoomed image is processed through the second changing field angle, the M-frame second background image can be obtained.
It should be noted that the second variable field angle is a ratio of a difference between the first field angle of the telephoto lens and the second field angle of the wide-angle lens to M, and a calculation process of the second variable field angle is substantially the same as a calculation process of the first variable field angle, which is not described herein again.
The process of acquiring the second background image is substantially the same as the process of acquiring the first background image, and is not described herein again.
It should be noted that the second background image is a second cropped image obtained by cropping the first zoom image to obtain M frames of views with decreasing field angles, and then performing interpolation processing on the M frames of second cropped images to obtain M frames of second background images, so as to ensure that the output sizes of the zoom videos after fusion are consistent.
And then sequentially fusing the second background image of the first zoom image of the L-th frame and the second foreground main body image of the second original tele image of the L-th frame together to form a second zoom image of the L-th frame, and executing the fusion process on the first zoom image of the M frames and the second original tele image of the M frames to obtain a second zoom image of the M frames, namely finishing the second zooming.
Because the first zoom image is obtained by zooming according to the first original tele image, the main image in the first zoom image is not changed, and the second original tele image is also obtained by shooting through a tele lens, the second foreground main image in the second original tele image is basically consistent with the first foreground main image in the first original tele image, so that the calculation process of the second foreground main image in the second original tele image is basically the same as the calculation process of the first foreground main image in the first original tele image, and the first original tele image is not described herein any more.
During specific fusion, sequentially fusing the second background image of the L-th frame first zoom image and the second foreground main body image of the L-th frame second original tele image together, and fusing the M-frame first zoom image and the M-frame second original tele image to obtain a second zoom image with an M-frame background view field which is continuously reduced, so that the finally obtained background view field of the second zoom image is continuously reduced until the background view field under the tele lens is recovered, and the process of zooming for the second time is completed.
It should be noted that, in this embodiment, after the first zooming is completed, the second zooming process may be started after the preset interval time set by the user is reached, or the second zooming process may be started after the request of the user is received, which is not described herein again.
The present invention also provides a digital zoom system, as shown in fig. 5, the system comprising:
an image obtaining module 501, configured to obtain a first original tele image under an ith frame of tele lens and a first original wide image under an ith frame of wide lens, where the first original tele image and the first original wide image include N frames of images, N is a positive integer, i is greater than or equal to 1 and less than or equal to N, and i is a positive integer;
a separating module 502, configured to separate an ith frame of a first foreground subject image from an ith frame of the first original tele image;
a background processing module 503, configured to process the first original wide-angle image of the ith frame according to a preset first variable field angle to obtain an ith frame first background image;
a fusion module 504, configured to fuse the first foreground subject image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
the first zooming module 505 is configured to perform the above processing procedure on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frame of the first zoom image is the second field angle under the wide-angle lens, so as to obtain N frames of the first zoom image, and complete the first zooming.
In some embodiments, the system further comprises:
the second zooming module 506 is configured to fuse a second foreground main image of the M frames of second original tele images with a second background image of the M frames of first zoom images to form M frames of second zoom images after a preset time interval is reached or a user request operation is received, so as to complete second zooming.
It should be noted that the structure and principle of the digital zoom system correspond to the steps in the digital zoom method one-to-one, and therefore, the details are not repeated herein.
It should be noted that the division of the modules of the above apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. For example, the selection module may be a processing element that is set up separately, or may be implemented by being integrated in a chip of the system, or may be stored in a memory of the system in the form of program code, and the function of the above x module may be called and executed by a processing element of the system. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when one of the above modules is implemented in the form of a Processing element scheduler code, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. For another example, these modules may be integrated together and implemented in the form of a System-On-a-Chip (SOC).
In other embodiments of the present application, an embodiment of the present application discloses a terminal device, and as shown in fig. 6, the terminal device 600 may include: one or more processors 601; a memory 602; a display 503; one or more application programs (not shown); and one or more computer programs 604, which may be connected via one or more communication buses 605. Wherein the one or more computer programs 604 are stored in the memory 602 and configured to be executed by the one or more processors 601, the one or more computer programs 604 comprising instructions.
The invention also discloses a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, performs the above-mentioned method.
The storage medium of the invention has stored thereon a computer program which, when being executed by a processor, carries out the above-mentioned method. The storage medium includes: a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, a usb disk, a Memory card, or an optical disk, which can store program codes.
In another embodiment of the disclosure, the present invention further provides a chip system, which is coupled to the memory and configured to read and execute the program instructions stored in the memory to perform the steps of the above method.
Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.
Each functional unit in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or make a contribution to the prior art, or all or part of the technical solutions may be implemented in the form of a software product stored in a storage medium and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: flash memory, removable hard drive, read only memory, random access memory, magnetic or optical disk, and the like.
The above description is only a specific implementation of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.
Although the embodiments of the present invention have been described in detail hereinabove, it is apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it is to be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the following claims. Moreover, the invention as described herein is capable of other embodiments and of being practiced or of being carried out in various ways.
Claims (11)
1. A digital zoom method, the method comprising:
acquiring a first original tele image under an ith frame of telephoto lens and a first original wide image under the ith frame of wide lens, wherein the first original tele image and the first original wide image comprise N frames of images, N is a positive integer, i is more than or equal to 1 and less than or equal to N, and i is a positive integer;
separating an ith frame of first foreground main body image from the ith frame of the first original tele image;
processing the first field angle of the first original wide-angle image of the ith frame according to a first field angle of the telephoto lens and a first change field angle preset by a second field angle of the wide-angle lens to obtain an ith frame first background image;
fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
and executing the processing process on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frames of the first zoom image is a second field angle under a wide-angle lens to obtain N frames of the first zoom image, and finishing the first zooming.
2. The digital zooming method of claim 1, wherein said separating the ith frame of the first foreground subject image from the ith frame of the first raw tele image comprises:
acquiring a background image and a first foreground main body image in the first original tele image of the ith frame through a depth convolution network, and calculating a mask matrix of the first original tele image according to the proportion of the background image and the first foreground main body image in the first original tele image;
the fusing the first foreground main image of the ith frame and the first background image of the ith frame together to obtain the first zoom image of the ith frame, including:
and fusing the mask matrix of the first original tele image of the ith frame, the background image, the first foreground main body image and the first background images of the N frames to obtain a first zoom image of the N frames.
3. The digital zoom method of claim 2, wherein the mask matrix of the first raw tele image satisfies the following equation: it ═ alpha ═ F + (1-alpha) × B;
wherein It is an image matrix of the first original tele image of the ith frame, alpha is a mask matrix having the same size as the image matrix, F is the first foreground subject image, and B is the background image.
4. The digital zooming method of claim 1, wherein the processing the field angle of the first original wide-angle image of the ith frame according to a preset first variable field angle to obtain an ith frame first background image comprises:
calculating to obtain the first variable field angle according to the field angle of the telephoto lens and the field angle of the wide-angle lens;
reducing the first field angle of the first original wide-angle image of the ith frame according to the first change field angle, so that the difference value between the field angle of the first original wide-angle image of the ith frame and the field angle of the original wide-angle image of the (i-1) th frame is the first change field angle, and obtaining a first clipping image of the ith frame;
and carrying out interpolation processing on the first cutting image of the ith frame to obtain the first background image of the ith frame.
5. The digital zoom method according to claim 4, wherein the first varying field angle satisfies the following formula: x | a1-a2|/N, where X represents a first varying field of view, a1 represents a first field of view, and a2 represents a second field of view.
6. The digital zooming method of claim 4, wherein the fusion of the first foreground subject image of the ith frame and the first background image of the ith frame to obtain the first zoom image of the ith frame satisfies the following formula: if ═ alpha ═ It + (1-alpha) × Iw;
wherein If is the first zoom image, alpha is the mask matrix of the first original tele image of the ith frame, It is the image matrix of the first original tele image of the ith frame, and Iw is the first background image of the ith frame.
7. The digital zooming method of claim 1, wherein after completion of the first zooming, the method further comprises:
starting to execute the second zooming after reaching the preset time interval or receiving the operation requested by the user;
acquiring an L-th frame of first zoom image and an L-th frame of second original tele image under a tele lens, wherein the second original tele image comprises M frames of images, M is a positive integer, L is more than or equal to 0 and less than or equal to M, and L is a positive integer;
separating an L frame second foreground main body image from the L frame second original tele image;
processing the first zoom image of the L-th frame according to the second change field angle to obtain a second background image of the L-th frame;
fusing the second foreground main image of the L-th frame and the second background image of the L-th frame together to obtain a second zoom image of the L-th frame;
and executing the processing process on the M frames of the second original tele image and the M frames of the first zoom image until the field angle of the M frames of the second zoom image is the first field angle under the tele lens to obtain M frames of the second zoom image and finish the second zooming.
8. A digital zoom system, the system comprising:
the image acquisition module is used for acquiring a first original tele image under an ith frame of tele lens and a first original wide image under an ith frame of wide lens, wherein the first original tele image and the first original wide image comprise N frames of images, N is a positive integer, i is more than or equal to 1 and less than or equal to N, and i is a positive integer;
the separation module is used for separating the ith frame of first foreground main body image from the ith frame of the first original tele image;
the background processing module is used for processing the first original wide-angle image of the ith frame according to a preset first change field angle to obtain a first background image of the ith frame;
the fusion module is used for fusing the first foreground main body image of the ith frame and the first background image of the ith frame together to obtain a first zoom image of the ith frame;
and the first zooming module is used for executing the processing process on the N frames of the first original tele image and the N frames of the first original wide-angle image until the field angle of the N frames of the first zooming image is a second field angle under a wide-angle lens, so as to obtain N frames of first zooming images and finish the first zooming.
9. The system of claim 8, further comprising:
and the second zooming module is used for fusing a second foreground main image of the M frames of second original tele images and a second background image of the M frames of first zoom images together to form M frames of second zoom images after a preset time interval is reached or a user request operation is received, so that secondary zooming is completed.
10. A computer-readable storage medium, having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, implements the method of any one of claims 1 to 7.
11. A terminal, comprising: a processor and a memory;
the memory is used for storing a computer program;
the processor is configured to execute the memory-stored computer program to cause the terminal to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740104.6A CN113438401B (en) | 2021-06-30 | 2021-06-30 | Digital zooming method, system, storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110740104.6A CN113438401B (en) | 2021-06-30 | 2021-06-30 | Digital zooming method, system, storage medium and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113438401A CN113438401A (en) | 2021-09-24 |
| CN113438401B true CN113438401B (en) | 2022-08-05 |
Family
ID=77758245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110740104.6A Active CN113438401B (en) | 2021-06-30 | 2021-06-30 | Digital zooming method, system, storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113438401B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114125291B (en) * | 2021-11-23 | 2023-12-05 | 北京拙河科技有限公司 | Image imaging method and device based on multi-focal-length camera and electronic equipment |
| CN116095512B (en) * | 2022-07-15 | 2024-04-02 | 荣耀终端有限公司 | Photographing method of terminal equipment and related device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107835372A (en) * | 2017-11-30 | 2018-03-23 | 广东欧珀移动通信有限公司 | Imaging method, device, mobile terminal and storage medium based on dual camera |
| CN111479065A (en) * | 2020-04-16 | 2020-07-31 | 北京迈格威科技有限公司 | Camera field angle synchronization method and device and image shooting method and device |
| CN112532808A (en) * | 2020-11-24 | 2021-03-19 | 维沃移动通信有限公司 | Image processing method and device and electronic equipment |
| CN112991242A (en) * | 2019-12-13 | 2021-06-18 | RealMe重庆移动通信有限公司 | Image processing method, image processing apparatus, storage medium, and terminal device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9350924B2 (en) * | 2014-08-25 | 2016-05-24 | John G. Posa | Portable electronic devices with integrated image/video compositing |
| US20180070010A1 (en) * | 2016-09-02 | 2018-03-08 | Altek Semiconductor Corp. | Image capturing apparatus and image zooming method thereof |
-
2021
- 2021-06-30 CN CN202110740104.6A patent/CN113438401B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107835372A (en) * | 2017-11-30 | 2018-03-23 | 广东欧珀移动通信有限公司 | Imaging method, device, mobile terminal and storage medium based on dual camera |
| CN112991242A (en) * | 2019-12-13 | 2021-06-18 | RealMe重庆移动通信有限公司 | Image processing method, image processing apparatus, storage medium, and terminal device |
| CN111479065A (en) * | 2020-04-16 | 2020-07-31 | 北京迈格威科技有限公司 | Camera field angle synchronization method and device and image shooting method and device |
| CN112532808A (en) * | 2020-11-24 | 2021-03-19 | 维沃移动通信有限公司 | Image processing method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 推推拉拉乐趣多――变焦的概念;陈皓;《数字世界》;20030531(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113438401A (en) | 2021-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113438401B (en) | Digital zooming method, system, storage medium and terminal | |
| CN110493525B (en) | Zoom image determination method and device, storage medium and terminal | |
| CN105991915B (en) | Photographing method and device and terminal | |
| CN107454332B (en) | Image processing method and device and electronic equipment | |
| CN106550184B (en) | Photo processing method and device | |
| FI117265B (en) | An electronic device and a method for processing image data in an electronic device | |
| TWI538512B (en) | Method for adjusting focus position and electronic apparatus | |
| JP5635844B2 (en) | Focus adjustment apparatus and imaging apparatus | |
| JP2020533697A (en) | Methods and equipment for image processing | |
| CN105847664A (en) | Shooting method and device for mobile terminal | |
| CN107800951B (en) | Electronic device and lens switching method thereof | |
| WO2013155804A1 (en) | Photograph shooting method and electronic device | |
| WO2018076460A1 (en) | Photographing method for terminal, and terminal | |
| CN112261387B (en) | Image fusion method and device for multi-camera module, storage medium and mobile terminal | |
| CN110177210B (en) | Photographing method and related device | |
| JP7221334B2 (en) | Photography method and device, terminal, storage medium | |
| CN105227847A (en) | A kind of camera photographic method of mobile phone and system | |
| JP2020188448A (en) | Imaging apparatus and imaging method | |
| WO2019006762A1 (en) | Image capturing apparatus and method | |
| CN114339042A (en) | Image processing method and device based on multiple cameras and computer readable storage medium | |
| CN111726528B (en) | Camera switching method, device, terminal and computer storage medium | |
| CN112585941A (en) | Focusing method and device, shooting equipment, movable platform and storage medium | |
| CN116389902A (en) | Liquid lens rapid focusing method and system | |
| Peddigari et al. | Real-time implementation issues in passive automatic focusing for digital still cameras | |
| CN109698902B (en) | Synchronous focusing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |