CN113747085B - Method and device for shooting video - Google Patents

Abstract

The application discloses a method and device for shooting video, which relate to the technical field of photographing and image processing, and can make the white balance effect of the Hitchcock zoom video better, so as to improve the quality of the Hitchcock zoom video. The method includes: collecting N+1 images including the target subject in real time for the first scene when the terminal is farther and farther away from the target subject. For the N images collected later, the white balance processing is performed based on the neural network "used to ensure the white balance consistency of adjacent images in time domain", and N optimized images are obtained. Enlarge and crop N optimized images to obtain N target images; the size of the target subject in the N target images is consistent with the size of the target subject in the first image collected, and the relative position of the target subject in the N target images is the same as The relative position of the target subject in the first image is consistent; the size of the N target images is consistent with the first image. Generate a Hitchcock zoom video based on N target images and the first image.

Description

Method and device for shooting video

This application claims the priority of the Chinese patent application submitted to the State Intellectual Property Office on May 30, 2020, with the application number 202010480536.3 and the application name "A Zooming Method and Device for Distinguishing Subject Characters and Backgrounds", the entire content of which is passed References are incorporated in this application.

technical field

The present application relates to the field of photographing technology and the field of image processing technology, and in particular to a method and device for shooting video.

Background technique

With the rapid development of mobile phone camera functions, more and more photography users are trying to add movie shooting elements to the video to increase the richness and high-level sense of the picture, and at the same time allow users to experience more cool and stunning visual effects. Hitchcock's zoom video is one of them that meets this visual effect.

Hitchcock zoom is a special video shooting technique. During a Hitchcock zoom video, the camera moves forward or backward, and changes focus while moving forward or backward, so that the subject of interest remains the same size in the captured image while the background image changes drastically. This effect can be used to reflect the rich emotions of the main character, bringing users a sense of tension and impact of space compression or expansion, and then obtain an unconventional video recording experience.

Therefore, how to realize the Hitchcock zoom has become a technical problem to be solved urgently.

Contents of the invention

The embodiments of the present application provide a video shooting method and device, which can make the obtained Hitchcock zoom video have a better white balance effect, thereby improving the quality of the Hitchcock zoom video, and further improving user experience.

In order to achieve the above object, the application adopts the following technical solutions:

In a first aspect, a method for shooting a video is provided, and the method is applied to a terminal. The method includes: collecting N+1 images in real time for the first scene, and the N+1 images all include the target subject; wherein, during the process of collecting the N+1 images, the terminal is getting closer and closer to the target subject Far. N is an integer greater than or equal to 1. For the N images collected after the N+1 images, white balance processing is performed based on the preset neural network to obtain N optimized images; the preset neural network is used to ensure the white balance consistency of adjacent images in the temporal domain. The N optimized images are enlarged and cropped to obtain N target images; wherein, the size of the target subject in each target image in the N target images is the same as the target in the first image collected in the N+1 images The size of the subject is consistent, and the relative position of the target subject in each of the N target images is consistent with the relative position of the target subject in the first image. The N target images are the same size as the first image. Generate a Hitchcock zoom video based on N target images and the first image.

In this technical solution, in the process of obtaining the Hitchcock zoom video, white balance processing is performed on the last N images of the N+1 images collected in real time, so that the processed images are consistent with the collected N+1 images. The white balance of the first image in 1 image is the same. In this way, the white balance effect of the obtained Hitchcock zoom video can be better, thereby improving the quality of the Hitchcock zoom video, thereby improving user experience.

In a possible design, the N+1 images include N1+1 images collected before and N2 images collected later, where the N1+1 images are collected by the first camera of the terminal, and the N2 images The image is collected by the second camera of the terminal; both N1 and N2 are integers greater than or equal to 1. That is to say, the technical solution provided by the embodiment of the present application can be applied to shooting a Hitchcock zoom video in a scene of switching cameras.

In a possible design, collecting N+1 images in real time for the first scene includes: obtaining the shooting magnification of the i-th image in the N+1 images; wherein, 2≤i≤N, i is an integer; If the shooting magnification of the i-th image is within the first shooting magnification range, the first camera based on the terminal captures the i+1th image among the N+1 images for the first scene; if the shooting magnification of the i-th image Within the second shooting magnification range, the i+1 th image of the N+1 images is collected for the first scene based on the second camera of the terminal. Wherein, the magnification of the first camera is a, and the magnification of the second camera is b; a<b; the first shooting magnification range is [a, b); the second shooting magnification range is greater than or equal to b.

That is to say, based on the shooting magnification of the i-th image, the terminal determines the camera that captures the i+1-th image. In this way, the terminal can enlarge the size of the target subject in the subsequent captured image by switching the camera. Compared with the traditional technology, it helps to make the obtained video with the Hitchcock zoom effect higher in definition, thereby improving user experience.

In a possible design, the shooting magnification of the i-th image is based on the zoom magnification of the size of the target subject in the i-th image relative to the size of the target subject in the first image, and the magnification of the camera that captures the first image The magnification is fixed.

In one possible design, the size of the target subject in the i-th image is characterized by at least one of the following features: the width of the target subject in the i-th image, the height of the target subject in the i-th image, the i-th The area of the target subject in the i-th image, or the number of pixels occupied by the target subject in the i-th image.

In a possible design, the method further includes: using an instance segmentation algorithm to extract the target subject from the i-th image, so as to determine the size of the target subject in the i-th image. In this way, it is helpful to improve the accuracy of determining the size of the target subject in the i-th image.

In a possible design, the method further includes: displaying first information in the current preview interface. The first information is used to instruct to stop shooting the Hitchcock zoom video. In this way, the user can know when to stop the mobile terminal, thereby improving user experience.

In a possible design, the method further includes: displaying second information in the current preview interface. The second information is used to indicate that the target subject is stationary. Since one of the requirements of the Hitchcock zoom video is that the positions of the target subjects in each image are consistent, based on this possible design, the user can know whether the current acquisition target is met during the process of acquiring the Hitchcock zoom video. Zone Kirk zooms video requirements, thereby improving user experience.

In a possible design, the method further includes: displaying third information in the current preview interface, where the third information is used to indicate that the target subject is at the center of the current preview image. In this way, the user can determine whether to move the terminal based on whether the terminal displays the third information, thereby helping to improve the quality of the Hitchcock video. Wherein, the current preview interface includes the current preview image (ie, the image captured by the camera) and information other than the current preview image (such as shooting controls, instruction information, etc.).

In a possible design, the method further includes: collecting N+1 images in real time for the first scene, including: when the target subject is in the center of the current preview image, collecting the first image of the N+1 images . In this way, it helps to improve the quality of Hitchcock's video.

In a possible design, the method further includes: displaying a user interface, where the user interface includes a first control, and the first control is used to indicate that the Hitchcock zoom video is taken from near to far. Collecting N+1 images for the first scene in real time includes: receiving an operation on the first control, and collecting N+1 images for the first scene in real time in response to the operation.

In a possible design, the moving speed of the terminal is less than or equal to a preset speed. In this way, it helps to improve the quality of the Hitchcock zoom video.

In a possible design, the preset neural network is used to predict the white balance gain of the image to be processed in combination with the feature map of the historical network layer, so as to ensure the consistency of white balance of adjacent images in the time domain; wherein, the historical network layer is the network layer used when predicting the white balance gain of the image before and temporally continuous with the image to be processed. Exemplarily, the image to be processed is one of the above N images. The white balance network fuses the network layer feature information of the current frame and the historical frame. In this way, considering the information of multiple frames helps to make the predicted value of white balance gain between frames closer, so that the white balance network is more stable, and then the white balance obtained after performing white balance processing on multiple consecutive images White balance consistency between images is better.

In a possible design, the preset neural network is trained based on preset constraint conditions; wherein, the preset constraint conditions include: the white balance gain prediction values for simulating multiple images continuous in time domain are consistent.

In a possible design, for the N images collected later in the N+1 images, white balance processing is performed based on the preset neural network to obtain N optimized images, including: the j-th image in the N+1 images input the image to the preset neural network to obtain the white balance gain prediction value of the jth image; where, 2≤j≤N-1, j is an integer. Applying the white balance gain prediction value of the jth image to the jth image to obtain an optimized image corresponding to the jth image; wherein, the N optimized images include the optimized image corresponding to the jth image.

In a second aspect, a method for shooting a video is provided, the method is applied to a terminal, and the method includes: collecting N+1 images for the first scene, and each of the N+1 images includes a target subject; wherein, during the collection of N+1 In the process of one image, the terminal is getting closer and closer to the target subject; N is an integer greater than or equal to 1. The first image of the N+1 images is collected by the first camera of the terminal, part or all of the images of the last N images of the N+1 images are collected by the second camera of the terminal, and the The magnification is smaller than that of the first camera. The size of the target subject in the N images acquired later in the N+1 images is smaller than or equal to the size of the target subject in the first image acquired in the N+1 images. For the N images collected later in the N+1 images, white balance processing is performed based on the preset neural network to obtain N optimized images; the preset neural network is used to ensure the white balance consistency of adjacent images in the temporal domain. Enlarge and crop the N optimized images to obtain N target images. Wherein, the size of the target subject in each of the N target images is consistent with the size of the target subject in the first image collected in the N+1 images, and the target in each of the N target images The relative position of the subject is consistent with the relative position of the target subject in the first image; the size of N target images is consistent with the first image. Based on the N target images and the first image, generate a Hitchcock zoom video.

Optionally, the N+1 images may be N+1 images collected continuously, that is, N+1 images collected in real time.

In this technical solution, in the scene where the terminal is getting closer and closer to the target subject, the subsequent image is collected by switching the camera with a smaller magnification, so that the size of the target subject in the post-collected image is smaller than or equal to the front-collected image The size of the target subject in the image of . Moreover, in the process of obtaining the Hitchcock zoom video, white balance processing is performed on the last N images of the collected N+1 images, so that the processed images are consistent with the collected N+1 images The white balance of the first image in is consistent. In this way, the white balance effect of the obtained Hitchcock zoom video can be better, thereby improving the quality of the Hitchcock zoom video and improving user experience.

In a possible design, the N images include N1 images collected before and N2 images collected later, where the N1 images are collected by the second camera, and the N2 images are collected by the third camera of the terminal Obtained; N1 and N2 are integers greater than or equal to 1.

In a possible design, collecting N+1 images in real time for the first scene includes: obtaining the shooting magnification of the i-th image in the N+1 images; wherein, 2≤i≤N, i is an integer; If the shooting magnification of the i-th image is within the first shooting magnification range, then based on the second camera, the i+1th image in the N+1 images is collected for the first scene; if the shooting magnification of the i-th image is at Within the second shooting magnification range, the third camera based on the terminal collects the i+1th image among the N+1 images for the first scene; wherein, the magnification of the second camera is b, and the magnification of the third camera is c; b>c; the first shooting magnification range is greater than or equal to b; the second shooting magnification range is [c, b).

That is to say, based on the shooting magnification of the i-th image, the terminal determines the camera that captures the i+1-th image. In this way, the terminal can use a camera with a smaller magnification than the camera used to capture the previous image to capture the subsequent image, that is, the target subject can be reduced by switching to a camera with a smaller magnification. The final image is "filled" to improve user experience.

In a possible design, the shooting magnification of the i-th image is based on the zoom magnification of the size of the target subject in the i-th image relative to the size of the target subject in the first image, and the magnification of the camera that captures the first image The magnification is fixed.

In one possible design, the size of the target subject in the i-th image is characterized by at least one of the following features: the width of the target subject in the i-th image, the height of the target subject in the i-th image, the i-th The area of the target subject in the i-th image, or the number of pixels occupied by the target subject in the i-th image.

In a possible design, the method further includes: using an instance segmentation algorithm to extract the target subject from the i-th image, so as to determine the size of the target subject in the i-th image. In this way, it is helpful to improve the accuracy of determining the size of the target subject in the i-th image.

In a possible design, the method further includes: displaying first information in the current preview interface, where the first information is used to indicate to stop shooting the Hitchcock zoom video. In this way, it is helpful to instruct the user to stop the mobile terminal at an appropriate time, thereby improving user experience.

In a possible design, the method further includes: displaying second information in the current preview interface, where the second information is used to indicate that the target subject is still. Based on this possible design, during the process of obtaining the Hitchcock zoom video, the user can know whether the current requirements for obtaining the Hitchcock zoom video are met, thereby improving user experience.

In a possible design, the method further includes: displaying third information in the current preview interface, where the third information is used to indicate that the target subject is at the center of the current preview image. In this way, the user can determine whether to move the terminal based on whether the terminal displays the third information, thereby helping to improve the quality of the Hitchcock video.

In a possible design, the collecting N+1 images for the first scene includes: collecting the first image when the target subject is in the center of the current preview image. In this way, it helps to improve the quality of Hitchcock's video.

In a possible design, the method further includes: displaying a user interface, where the user interface includes a second control, and the second control is used to indicate that the Hitchcock zoom video is taken from far to near. Collecting N+1 images for the first scene includes: receiving an operation on the second control, and collecting N+1 images for the first scene in response to the operation.

In a possible design, the moving speed of the terminal is less than or equal to a preset speed. In this way, it helps to improve the quality of high Hitchcock zoom videos.

In a possible design, the preset neural network is used to predict the white balance gain of the image to be processed in combination with the feature map of the historical network layer, so as to ensure the consistency of white balance of adjacent images in the time domain; wherein, the historical network layer is the network layer used when predicting the white balance gain of the image before and temporally continuous with the image to be processed. For its beneficial effects, reference may be made to related possible designs of the above-mentioned first aspect.

In a possible design, the preset neural network is trained based on preset constraint conditions; wherein, the preset constraint conditions include: the white balance gain prediction values for simulating multiple images continuous in time domain are consistent.

In a possible design, for the N images collected later in the N+1 images, white balance processing is performed based on the preset neural network to obtain N optimized images, including: the j-th image in the N+1 images input the image to the preset neural network to obtain the white balance gain prediction value of the jth image; where, 2≤j≤N-1, j is an integer. Applying the white balance gain prediction value of the jth image to the jth image to obtain an optimized image corresponding to the jth image; wherein, the N optimized images include the optimized image corresponding to the jth image.

In a third aspect, a method for shooting video is provided, which is applied to a terminal. The terminal includes a first camera and a second camera, and the magnification of the first camera is different from that of the second camera. The method includes: respectively collecting a first image and a second image for a first scene at a first moment by using a first camera and a second camera; wherein both the first image and the second image contain a target subject. Based on the preset playback duration and preset playback frame rate of the video, determine the frame number N of the image to be inserted between the first image and the second image, where N is an integer greater than or equal to 1. Based on the frame number N, the first image and the second image, N images to be inserted are determined. A video is generated based on the first image, the second image and the N images to be inserted; in the video, the size of the target subject in each image gradually becomes larger or smaller.

In this technical solution, the terminal collects multiple frames of images for the same scene at the same time through multiple cameras, and interpolates frames based on the multiple frames of images to generate a video in which the size of the target subject in each image gradually becomes larger or larger. Gradually get smaller. This helps to improve the quality of the generated video compared to conventional techniques. In addition, it helps to improve the fun of the animation effect and enhance the user's stickiness to the terminal.

In a possible design, the terminal further includes a third camera, and the magnification of the third camera is between those of the first camera and the second camera. The method further includes: collecting a third image for the first scene at the first moment by using the third camera; wherein, the third image includes the target subject. Determining the N images to be inserted based on the frame number N, the first image and the second image includes: determining the N images to be inserted based on the frame number N, the first image, the second image and the third image. In this way, it helps to further improve the quality of the video.

In a fourth aspect, a terminal is provided.

In a possible design, the terminal may be used to execute any one of the methods provided in the first aspect to the third aspect above. The present application may divide the terminal into functional modules according to any one of the methods provided in the first aspect to the third aspect. For example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. Exemplarily, the present application may divide the terminal into a collection unit, a processing unit, a display unit, and the like according to functions. For descriptions of possible technical solutions and beneficial effects performed by the above-mentioned divided functional modules, reference may be made to the corresponding technical solutions provided in the first aspect to the third aspect above, which will not be repeated here.

In another possible design, the device includes a memory and a processor, the memory is used to store computer instructions, and the processor is used to call the computer instructions to perform any of the functions provided in the first aspect to the third aspect. a way. Wherein, the collection step in any one of the methods provided by the first aspect to the third aspect may be specifically replaced by a control collection step in this possible design. In this possible design, the display step in the corresponding method above can be replaced by a control display step.

In a fifth aspect, a terminal is provided, including: a processor, a memory, and a camera. The camera is used to collect images, etc., the memory is used to store computer programs and instructions, and the processor is used to call the computer programs and instructions, and cooperate with the one or more cameras to execute the corresponding technical solutions provided by the first aspect to the third aspect above.

In a sixth aspect, a computer-readable storage medium is provided, such as a non-transitory computer-readable storage medium. A computer program (or instruction) is stored thereon, and when the computer program (or instruction) is run on the computer, the computer is made to execute any one of the methods provided in the first aspect to the third aspect above. Wherein, the collection step in any one of the methods provided by the first aspect to the third aspect may be specifically replaced by a control collection step in this possible design. In this possible design, the display step in the corresponding method above can be replaced by a control display step.

A seventh aspect provides a computer program product, which, when running on a computer, causes any one of the methods provided in the first aspect to the third aspect to be executed. Wherein, the collection step in any one of the methods provided by the first aspect to the third aspect may be specifically replaced by a control collection step in this possible design. In this possible design, the display step in the corresponding method above can be replaced by a control display step.

It can be understood that any terminal, computer storage medium, computer program product, or chip system provided above can be applied to the corresponding method provided above. Therefore, the beneficial effects that it can achieve can refer to the corresponding The beneficial effects in the method will not be repeated here.

In this application, the names of the above-mentioned terminals or functional modules do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present application, they fall within the scope of the claims of the present application and their equivalent technologies.

These or other aspects of the present application will be more clearly understood in the following description.

Description of drawings

FIG. 1 is a schematic diagram of a hardware structure of a terminal applicable to an embodiment of the present application;

FIG. 2 is a software structural block diagram of a terminal applicable to an embodiment of the present application;

FIG. 3 is a schematic diagram of interface changes for starting the Hitchcock zoom video shooting mode provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of a hardware structure of a computer device provided by an embodiment of the present application;

FIG. 5 is a schematic flowchart of a training data preparation process before training a white balance network provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a network architecture used in training a white balance network provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of another network architecture used in training the white balance network provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of a network architecture used in the prediction stage provided by the embodiment of the present application;

FIG. 9 is a schematic flowchart of a white balance gain prediction method provided by an embodiment of the present application;

FIG. 10 is a schematic flowchart of a method for shooting a video provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of interface changes for starting Hitchcock zoom video shooting from near and far modes provided by the embodiment of the present application;

Figure 12 is a schematic diagram of a set of interfaces provided by the embodiment of the present application;

Fig. 13 is another set of interface diagrams provided by the embodiment of the present application;

Fig. 14 is another set of interface diagrams provided by the embodiment of the present application;

Fig. 15 is another set of interface diagrams provided by the embodiment of the present application;

Fig. 16 is another set of interface diagrams provided by the embodiment of the present application;

Fig. 17 is a schematic diagram of another set of interfaces provided by the embodiment of the present application;

Fig. 18 is another set of interface diagrams provided by the embodiment of the present application;

FIG. 19a is a schematic flowchart of a method for capturing images by a terminal provided in an embodiment of the present application;

Fig. 19b is a schematic flowchart of a method for determining a camera for capturing an image provided by an embodiment of the present application;

FIG. 20 is a schematic diagram of instance segmentation provided by the embodiment of the present application;

FIG. 21 is a schematic diagram of a process of capturing images by a terminal provided in an embodiment of the present application;

FIG. 22a is a schematic diagram of an image collected by a terminal provided in an embodiment of the present application;

FIG. 22b is a schematic diagram of an image collected by another terminal provided in an embodiment of the present application;

Fig. 23a is a schematic diagram of a current preview interface provided by the embodiment of the present application;

Fig. 23b is a schematic diagram of another current preview interface provided by the embodiment of the present application;

Fig. 23c is a schematic diagram of another current preview interface provided by the embodiment of the present application;

Fig. 24 is a schematic diagram of zooming in and cropping a collected image provided by an embodiment of the present application;

FIG. 25 is a schematic flowchart of another method for shooting video provided in the embodiment of the present application;

Fig. 26 is a schematic diagram of the process of processing the collected images in the Hitchcock zoom video in the traditional technology;

FIG. 27 is a schematic flowchart of another method for shooting video provided in the embodiment of the present application;

FIG. 28 is a schematic diagram of an image processing process provided by an embodiment of the present application;

FIG. 29 is a schematic structural diagram of a terminal provided in an embodiment of the present application;

FIG. 30 is a schematic structural diagram of another terminal provided by an embodiment of the present application.

detailed description

In the embodiments of the present application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application shall not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

In the embodiments of the present application, the terms "first" and "second" are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

In the embodiments of the present application, the term "consistent" is only used to describe the same or similar (that is, little difference). The small difference can be reflected by the fact that the difference between the corresponding parameters is less than or equal to the threshold. For example, the same size of the target subjects means that the sizes of the target subjects are the same or the difference is less than or equal to a threshold.

The method for shooting video provided in the embodiment of the present application can be applied to a terminal, and the terminal can be a terminal with a camera, such as a smart phone, a tablet computer, a wearable device, an AR/VR device, or a personal computer (personal computer, A PC), a personal digital assistant (personal digital assistant, PDA), a netbook, or any other terminal capable of implementing the embodiments of the present application. The present application does not limit the specific form of the terminal.

In this application, the structure of the terminal may be as shown in FIG. 1 . As shown in FIG. 1, the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, Antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, A display screen 194, a subscriber identification module (subscriber identification module, SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structure shown in this embodiment does not constitute a specific limitation on the terminal 100 . In other embodiments, the terminal 100 may include more or fewer components than shown in the illustration, or combine some components, or separate some components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor ( image signal processor (ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. For example, in this application, the processor 110 may control the camera 193 to collect N+1 images for the first scene in real time, and each of the N+1 images includes the target subject. Wherein, during the process of collecting N+1 images, the camera 193 is getting farther and farther away from the target subject; N is an integer greater than or equal to 1. Then, for the N images collected after the N+1 images, the processor 110 can perform white balance processing based on the preset neural network to obtain N optimized images; the preset neural network is used to ensure the whitening of adjacent images in the time domain. Balance consistency. Next, the processor 110 can enlarge and crop the N optimized images to obtain N target images; wherein, the size of the target subject in the N target images is the same as that of the target subject in the first image collected in the N+1 images The size of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the size of the N target images is consistent with the first image. Finally, the processor 110 may generate a Hitchcock zoom video based on the N target images and the first image. For the relevant description of the technical solution, please refer to the following.

Wherein, the controller may be the nerve center and command center of the terminal 100 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuitsound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver (universal asynchronous receiver) /transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and/or A universal serial bus (universal serial bus, USB) interface, etc.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . The MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (displayserial interface, DSI), and the like. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface to realize the shooting function of the terminal 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the terminal 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB TypeC interface, and the like. The USB interface 130 can be used to connect a charger to charge the terminal 100, and can also be used to transmit data between the terminal 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other terminals, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in this embodiment is only for schematic illustration, and does not constitute a structural limitation of the terminal 100 . In other embodiments of the present application, the terminal 100 may also adopt different interface connection modes in the foregoing embodiments, or a combination of multiple interface connection modes.

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the terminal 100 can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

The terminal 100 realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light emitting diode). AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oled, quantum dot light-emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the terminal 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

A series of graphical user interfaces (graphical user interface, GUI) can be displayed on the display screen 194 of the terminal 100 , and these GUIs are the main screen of the terminal 100 . Generally, the size of the display screen 194 of the terminal 100 is fixed, and only limited controls can be displayed on the display screen 194 of the terminal 100 . A control is a GUI element, which is a software component contained in an application that controls all the data processed by the application and the interaction of these data. Users can interact with the control through direct manipulation. , so as to read or edit the relevant information of the application. Generally speaking, controls may include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and Widgets.

The terminal 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the terminal 100 may include 1 or N cameras 193, where N is a positive integer greater than 1. For example, the aforementioned camera 193 may include one or at least two types of cameras such as a main camera, a telephoto camera, a wide-angle camera, an infrared camera, a depth camera, or a black and white camera. In combination with the technical solutions provided in the embodiments of the present application, the first terminal may use one or at least two of the above-mentioned cameras to collect images, and process (such as fusion, etc.) the collected images to obtain a preview image (such as the first preview image or second preview image, etc.).

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the terminal 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. Terminal 100 may support one or more video codecs. In this way, the terminal 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the terminal 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by executing instructions stored in the internal memory 121 . For example, in this embodiment, the processor 110 may acquire the gesture of the terminal 100 by executing instructions stored in the internal memory 121 . The internal memory 121 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.) and the like. The data storage area can store data created during the use of the terminal 100 (such as audio data, phonebook, etc.) and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like. The processor 110 executes various functional applications and data processing of the terminal 100 by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal 100 may implement an audio function through an audio module 170 , a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Terminal 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the terminal 100 answers a phone call or voice information, the receiver 170B can be placed close to the human ear to listen to the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The terminal 100 may be provided with at least one microphone 170C. In some other embodiments, the terminal 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the terminal 100 can also be equipped with three, four or more microphones 170C to realize sound signal collection, noise reduction, identify sound sources, realize directional recording functions, and the like.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D may be the USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 100 determines the strength of the pressure from the change in capacitance. When a touch operation acts on the display screen 194, the terminal 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.

The gyro sensor 180B may be used to determine the motion posture of the terminal 100 . In some embodiments, the angular velocity of the terminal 100 around three axes (ie, x, y and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the terminal 100, and calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the terminal 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The terminal 100 may use the magnetic sensor 180D to detect the opening and closing of the flip holster. In some embodiments, when the terminal 100 is a clamshell machine, the terminal 100 can detect the opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the terminal 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the terminal 100 is stationary. It can also be used to recognize terminal gestures, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The terminal 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The terminal 100 emits infrared light through the light emitting diode. The terminal 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it may be determined that there is an object near the terminal 100 . When insufficient reflected light is detected, the terminal 100 may determine that there is no object near the terminal 100 . The terminal 100 can use the proximity light sensor 180G to detect that the user holds the terminal 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The terminal 100 may adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access to the application lock, take pictures with fingerprints, answer incoming calls with fingerprints, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal 100 uses the temperature detected by the temperature sensor 180J to implement a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal 100 executes reducing the performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the terminal 100 heats the battery 142 to avoid abnormal shutdown of the terminal 100 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the terminal 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal 100 , which is different from the position of the display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The terminal 100 may receive key input and generate key signal input related to user settings and function control of the terminal 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

In addition, an operating system runs on top of the above components. For example, the iOS operating system developed by Apple, the Android open source operating system developed by Google, and the Windows operating system developed by Microsoft. Applications can be installed and run on this operating system.

The operating system of the terminal 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In this embodiment of the present application, an Android system with a layered architecture is taken as an example to illustrate the software structure of the terminal 100 .

FIG. 2 is a block diagram of the software structure of the terminal 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, the application program layer, the application program framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

The application layer can consist of a series of application packages. As shown in Figure 2, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message. For example, when taking pictures, the camera application can access the camera interface management service provided by the application framework layer.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 2, the application framework layer can include window managers, content providers, view systems, phone managers, resource managers, notification managers, and so on. For example, in the embodiment of the present application, when taking a photo, the application framework layer may provide the application layer with an API related to the photo taking function, and provide the application layer with a camera interface management service to realize the photo taking function.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc.

Content providers are used to store and retrieve data and make it accessible to applications. Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide a communication function of the terminal 100 . For example, the management of call status (including connected, hung up, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the download completion, message reminder, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, emitting a prompt sound, terminal vibration, and indicator light flashing, etc.

Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

A system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

It should be noted that although the embodiment of the present application uses the Android system as an example for illustration, its basic principles are also applicable to terminals based on operating systems such as iOS or Windows.

The workflow of the software and hardware of the terminal 100 will be exemplarily described below with reference to FIG. 1 and the video shooting scene.

First, the touch sensor 180K receives a touch operation on the icon of the camera application, and reports the touch operation to the processor 110, so that the processor 110 responds to the touch operation, starts the camera application, and displays the user interface of the camera application on the display screen 194 , as shown in Figure 3a. In addition, in the embodiment of the present application, the terminal 100 can also start the camera application in other ways, and display the user interface of the camera application on the display screen 194 . For example, when the terminal 100 is on a black screen, displays a lock screen interface, or displays a certain user interface after unlocking, it may respond to a user's voice command or shortcut operation, etc., to start the camera application, and display the camera application user interface on the display screen 194. Among them, the user interface of the camera includes controls such as "night view", "portrait", "photographing", "video recording" and "more".

Secondly, the touch sensor 180K receives a touch operation on the "recording" control and reports it to the processor 110, so that the processor 110 highlights the "recording" control in response to the above touch operation, as shown in Figure 3 b, In figure b in Figure 3, the "recording" control is highlighted with a frame; and the recording function is started, and the user interface under the recording function is displayed, as shown in figure c in Figure 3 . The user interface under the recording function includes controls such as "Hitchcock Zoom Video", "Normal Video" and "More".

Next, the touch sensor 180K receives a touch operation on the "Hitchcock zoom video" control, and reports it to the processor 110, so that the processor 110 highlights the "Hitchcock zoom video" control in response to the touch operation, As shown in figure d in Figure 3, the "Hitchcock Zoom Video" control in Figure 3 d is highlighted with a border; and the Hitchcock zoom video shooting mode is used to start video recording, that is, to start shooting Hitchcock Kirk Zoom video.

In the embodiment of the present application, the terminal 100 can also be enabled to start the Hitchcock zoom video shooting mode in other ways. For example, the terminal 100 may start the Hitchcock zoom video shooting mode in response to the user's voice command or shortcut operation.

As shown in FIG. 4 , it is a schematic diagram of a hardware structure of a computer device 30 provided in the embodiment of the present application. The computer device 30 includes a processor 301 , a memory 302 , a communication interface 303 and a bus 304 . Wherein, the processor 301 , the memory 302 and the communication interface 303 may be connected through a bus 304 .

The processor 301 is the control center of the computer device 30, and may be a general-purpose CPU or other general-purpose processors. Wherein, the general-purpose processor may be a microprocessor or any conventional processor.

As an example, the processor 301 may include one or more CPUs, such as CPU 0 and CPU 1 shown in FIG. 4 .

The memory 302 may be a read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types that can store information and instructions The dynamic storage device can also be an electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), a magnetic disk storage medium or other magnetic storage devices, or can be used to carry or store instructions or data structures desired program code and any other medium that can be accessed by a computer, but not limited thereto.

In a possible implementation manner, the memory 301 may exist independently of the processor 301 . The memory 302 may be connected to the processor 301 through the bus 304 and used for storing data, instructions or program codes. When the processor 301 invokes and executes the instructions or program codes stored in the memory 302, it can realize the corresponding method of the training data preparation process before training the white balance network and the method of training the white balance network provided by the embodiment of the present application.

In another possible implementation manner, the memory 302 may also be integrated with the processor 301 .

The communication interface 303 may be any device capable of inputting parameter information, such as a communication interface, which is not limited in this embodiment of the present application. Wherein, the communication interface may include a receiving unit and a sending unit. For example, the communication interface 303 may be used to send related information (such as values of related parameters) of the trained white balance network to the terminal 100 .

The bus 304 may be an industry standard architecture (industry standard architecture, ISA) bus, a peripheral component interconnect (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 4 , but it does not mean that there is only one bus or one type of bus.

It should be pointed out that the structure shown in FIG. 4 does not constitute a limitation to the computer device 30. In addition to the components shown in FIG. 4, the computer device 30 may include more or less components than shown in the illustration, or Combining certain parts, or different arrangements of parts.

It should also be pointed out that the computer device 30 shown in FIG. 4 may specifically be any terminal 100 provided above, or any network device, such as an access network device (such as a base station).

The technical solutions provided by the embodiments of the present application are described below in conjunction with the accompanying drawings:

The method for shooting a video provided in the embodiment of the present application can be applied to shooting a Hitchcock zoom video. Moreover, in the process of shooting the Hitchcock zoom video, the white balance processing is performed on the collected images based on the white balance network, so that the white balance of adjacent images in the time domain is consistent. in:

White balance is an index that describes the accuracy of white after the three primary colors of red, green and blue are mixed. White balance is a very important concept in the field of TV photography, through which a series of problems with color tone processing can be solved.

White balance gain is a parameter to correct the white balance of the image.

A white balance network is a network used to predict the white balance gain of an image. The white balance network can be a deep learning network, such as a neural network.

White balance consistency refers to processing adjacent images in the time domain by using an approximate white balance gain, so that the white balance effects between the processed images are the same or similar. Specifically, the white balance network provided by the embodiment of the present application is used to process the temporally adjacent images, so that the white balance effects of the processed images are the same or similar.

As an example, the images continuously collected by the terminal, or the images obtained after processing based on the images continuously collected by the terminal, can be understood as adjacent images in time domain.

The following describes the white balance network provided in the embodiment of the present application:

Training data preparation process:

As shown in FIG. 5 , it is a schematic flowchart of a training data preparation process before training a white balance network provided in an embodiment of the present application. The training data preparation process before training the white balance network can be performed by the computer device 30 described above. The method shown in Figure 5 includes the following steps:

S101: The computer device acquires original images in various environments (such as indoor and outdoor environments with different color temperatures, different brightnesses, and different viewing angles) collected by multiple cameras (such as a main camera, a wide-angle camera, etc.).

S102: For each original image, the computer device extracts parameters of the gray or white part of the original image to obtain a white balance gain of the original image.

The white balance gain here is used as the prediction target of the white balance network during the training phase. Therefore, in order to distinguish the white balance gain prediction value below, the white balance gain obtained here is called the white balance gain reference value hereinafter.

Optionally, the gray or white part in the original image can be obtained based on standard color card comparison.

S103: For each original image, the computer device performs data enhancement on the original image to obtain a set of enhanced images. A set of enhanced images is used to determine an original sample.

For convenience of description, each group of enhanced images is referred to as an image group hereinafter.

Implementation 1: An image group is used as an original sample. An image group includes P images, and P is an integer greater than or equal to 2. The P images in an image group are used to simulate the P consecutive images collected by the camera in the time domain.

Optionally, the P images in an image group are used to simulate P consecutive images in time domain collected by the same camera.

Optionally, the P images in an image group are used to simulate P time-domain continuous images collected before and after switching between different cameras.

Wherein, the first image in an image group may be generated based on a set of random numbers based on the original image corresponding to the image group, of course, this embodiment of the present application is not limited thereto.

Implementation 2: An image group and the original image corresponding to the image group are used as an original sample. An image group includes Q images, and Q is an integer greater than or equal to 1. The images in an image group and the original images corresponding to the image group are used to simulate Q+1 consecutive images collected by the camera in time domain.

Optionally, the Q images in an image group and the original images corresponding to the image group are used to simulate Q+1 time-domain continuous images collected by the same camera.

Optionally, the Q images in an image group and the original images corresponding to the image group are used to simulate Q+1 time-domain continuous images collected before and after different cameras are switched.

For example, in an original sample, the original image may be used as the first image, and the images in the image group corresponding to the original image are used as the second image to the Q+1th image in the original sample.

Usually, a part of the original samples is used to simulate multiple time-domain continuous images collected by the same camera, and another part of the original samples is used to simulate multiple time-domain continuous images collected before and after switching between different cameras. In this way, it is possible to make the white balance network trained based on all original samples applicable to the scene of not switching the camera and the scene of switching the camera at the same time.

It should be noted that, for the convenience of description, an original sample is an image group (that is, the above-mentioned implementation mode 1) as an example for description below.

S104: For each original sample, the computer device converts the image in the original sample into the same color space to obtain a sample. All samples constitute the training data.

Optionally, the color space may be a red green blue (red green blue, RGB) color space, etc., of course, the specific implementation is not limited thereto.

Optionally, all samples in the training data use the same color space.

It should be noted that the images in the samples belong to the same color space in order to avoid (or eliminate) differences between different camera modules in the prediction stage. In addition, if multiple images in an original sample are in the same color space, the step of converting to the same color space may not be performed. In this case, a group of enhanced images obtained based on the same original image is called an image group.

Training phase:

As shown in FIG. 6 , it is a schematic diagram of a network architecture used in training a white balance network provided by an embodiment of the present application.

in(n) represents the nth image in the sample. n is the number of images in a sample, n is greater than or equal to 2, and n is an integer. in(n-a) represents the n-ath image in the sample. a<n, a is an integer.

out(n) indicates the output of the first subnetwork when the input of the first subnetwork is in(n). out(n-a) indicates the output of the first sub-network when the input of the first sub-network is in(n-a).

mem(n-1, 1) represents the feature map corresponding to the n-1th image in the sample to the feature map corresponding to the first image. mem(n-a-1, 1) represents the feature map corresponding to the n-a-1th image in the sample to the feature map corresponding to the first image.

Wherein, the feature map corresponding to the n-1th image is the feature map of the network layer included in the first sub-network when the input of the first sub-network is in(n-1). The embodiment of the present application does not limit which network layer or layers in the first subnetwork the network layer is specifically, nor the specific implementation manner of each network layer. In addition, the feature maps corresponding to different images in a sample may be feature maps of the same network layer or feature maps of different network layers in the first sub-network.

The loss function (loss) is used to constrain the training process, so as to achieve the training goal of "out(n), out(n-1)...out(n-a) consistent".

During training, for any sample in the training data:

First, the computer device inputs the n-a-th image to the n-th image in the sample into the network architecture shown in Figure 6, and the network architecture outputs a set of white balance gain prediction values out(n-a) to out(n) .

Secondly, the computer device uses the white balance gain reference value of the original image corresponding to the sample as supervision, so that each value in "out(n-a)...out(n)" is as close as possible to the white balance gain reference value of the original image As the target, adjust the values of the parameters in the first subnetwork.

By analogy, multiple samples are input into the network architecture in turn, and the above steps are repeated. Moreover, during the execution process, the loss function is used to constrain, so that the "out(n-a)...out(n)" corresponding to the same sample and the white balance gain reference value of the original image corresponding to the sample are consistent. When the accuracy of the network architecture reaches a preset accuracy, it means that the white balance network has been trained.

That is to say, the white balance network is trained based on the constraint condition that "the predicted values of the white balance gain for simulating consecutive images in time domain are consistent".

In FIG. 6 , the white balance gain prediction values of consecutive a+1 images in the time domain are supervised by a consistent supervision method.

In an example, if n=2 and a=1, the network architecture used when training the white balance network can be as shown in FIG. 7 .

During training, for any sample in the training data:

First, the computer device inputs the first image in the sample as in(1) and the second image as in(2) into the network architecture shown in Figure 7, and the network architecture outputs a pair of white balance gain prediction values out (1) and out (2).

Secondly, the computer equipment uses the white balance gain reference value of the original image corresponding to the sample as supervision, and combines the white balance gain prediction values out(1) and out(2) output by the network architecture to adjust the parameters of the white balance network. value.

By analogy, multiple samples are input into the network architecture in turn, and the above steps are repeated. Moreover, during the execution process, a loss function is used to constrain so that "out(1) and out(2)" corresponding to the same sample and the white balance gain reference value of the original image corresponding to the sample are consistent. When the accuracy of the network architecture reaches a preset accuracy, it means that the white balance network has been trained.

In Fig. 7, the white balance gain prediction values of two consecutive images in the time domain are supervised by a consistent supervision method.

Forecast phase:

As shown in FIG. 8 , it is a schematic diagram of a network architecture used in the prediction stage provided by the embodiment of the present application.

The first sub-network in the white balance network in Fig. 8 is the first sub-network at the end of the training phase.

in(t) represents the input of the white balance network, which is used to input the image to be predicted.

out(t) is the output of the white balance network when the input of the white balance network is in(t).

mem(t-1, t-T) represents the feature map output by the first target network layer used in the process of predicting the white balance gain of the collected t-1th image based on the white balance network (hereinafter referred to as the first The feature map corresponding to the t-1 image), to the feature map (hereinafter referred to as the feature map (hereinafter referred to as The feature map corresponding to the t-Tth image).

Optionally, the value of T can be adjusted. Generally, the larger the value of T, the smaller the overall fluctuation range of the white balance gain of multiple consecutive images in the time domain predicted by the white balance network, that is, the larger the value of T, the smaller the overall fluctuation range of the white balance gain of the multiple images in the temporal domain predicted by the white balance network. After the white balance processing is performed on multiple consecutive images in the time domain, the white balance consistency between the obtained images is better.

Which or which feature maps mem(t-1, t-T) is specifically will be updated as the value of t is updated. For example, in the case of T=3, assuming t=5, mem(t-1, t-T) represents the feature map corresponding to the fourth image collected to the feature map corresponding to the first image collected. Assuming t=6, then mem(t-1, t-T) represents the feature map corresponding to the collected fifth image to the feature map corresponding to the collected second image.

When predicting, for the tth image collected, the image is input as in(t) into the white balance network shown in Figure 8, and the white balance network is under the constraint of mem(t-1, t-T), the output The out(t) is the white balance gain prediction value.

That is to say, the white balance network is used to predict the white balance gain of the image to be processed (ie in(t)) in combination with the feature map of the historical network layer (ie mem(t-1, t-T)), so as to ensure that the time domain White balance consistency of adjacent images. Wherein, the historical network layer is the network layer used when predicting the white balance gain of the image before and temporally continuous with the image to be processed.

As shown in FIG. 9 , it is a schematic flowchart of a white balance gain prediction method provided by the embodiment of the present application. The subject of execution of this technical solution may be the computer device 30 provided above. The method shown in Figure 9 may include the following steps:

S201: The computer device acquires a first image to be predicted and an original color space of the first image to be predicted. Wherein, the original color space of the first image to be preset is the color space of the camera used when collecting the image to be predicted.

Wherein, the first image to be predicted may be any one of the non-first images among the consecutive multiple images collected by the same camera, or any one of the non-first images among the consecutive multiple images collected by different cameras image.

S202: The computer device converts the original color space of the first image to be predicted into a preset color space to obtain a second image to be predicted. Wherein, the preset color space is the color space used in the training phase.

It should be noted that, if the original color space of the first image to be predicted is the preset color space, S202 may not be performed. In this case, "the second image to be predicted" in the following steps may be replaced with "the first image to be predicted image".

S203: The computer device inputs the second image to be predicted as in(t) to the white balance network (such as the white balance network shown in FIG. 8 ), the white balance network is under the constraint of mem(t-1, t-T), The output out(t) is the white balance gain prediction value.

S204: The computer device converts the predicted value into the original color space, and applies the converted predicted value to the first image to be predicted to obtain an optimized image corresponding to the first image to be predicted.

Applying the converted predicted value to the first image to be predicted to obtain an optimized image corresponding to the first image to be predicted may include: multiplying the converted predicted value by the pixel value of each pixel in the first image to be predicted , to obtain a new pixel value, and use the pixel value as the pixel value of the pixel corresponding to the pixel in the optimized image corresponding to the first image to be predicted.

Wherein, the optimized image is an image obtained after processing the first image to be predicted based on the white balance network. In the embodiment of the present application, it is considered that: the image obtained after performing the processing of S201-S204 on each of the non-first images among the multiple consecutive images in the temporal domain is the first of the multiple consecutive images in the temporal domain There is white balance consistency between images.

The traditional white balance network usually only considers single frame information, which will cause jumps in the predicted value of white balance gain between frames, that is, the overall fluctuation range of the predicted value of white balance gain between frames larger.

However, the white balance network provided by the embodiment of the present application integrates the network layer feature (feature) information of the current frame and the historical frame (that is, the feature map, specifically mem(t-1, t-T) above). In this way, considering the information of multiple frames helps to make the white balance gain prediction values between frames closer, thus making the white balance network more stable. More specifically, in the above technical solution, the current frame and the history frame are continuous in time domain. Therefore, the white balance network provided by the above technical solution makes the white balance gain prediction values of multiple consecutive images in the time domain closer, that is, the overall fluctuation range of the white balance gain of the continuous multiple images predicted by the white balance network is smaller, so that the white balance network is more stable, that is, the white balance consistency between images obtained after performing white balance processing on multiple consecutive images is better.

It should be noted that, during the training process of the white balance network provided in the embodiment of the present application, "the white balance gain prediction values of images enhanced from the same original image are consistent" is the constraint. In fact, it cannot be guaranteed that the white balance gain prediction values of multiple consecutive images are exactly the same, but there are certain fluctuations. However, compared with the network in the prior art based on single-frame prediction of white balance gain, the white balance network provided by the embodiment of the present application helps to reduce the overall fluctuation range of white balance gain of multiple consecutive images in the time domain, thereby Improve the stability of the white balance network.

In addition, the traditional white balance network is trained based on multiple images collected by the same camera, which will cause the traditional white balance network to not be applied to multi-camera switching scenarios.

However, in the embodiment of the present application, data enhancement is performed for a multi-camera switching scene. Specifically, it can be reflected in the fact that when training the white balance network, the training data used includes samples for simulating multiple consecutive images before and after the multi-camera switching scene. Since there are generally changes in viewing angle, size, and image input statistics during multi-camera switching, the multi-camera switching scene is simulated through data enhancement during training, which helps to constrain the prediction value of the network to be consistent in the multi-camera switching scene.

The embodiment of the present application does not limit the application of the white balance network provided above.

Hereinafter, the application of the white balance network provided by the embodiment of the present application when shooting a Hitchcock zoom video is described.

The size of the target subjects in different images in the Hitchcock zoom video is consistent (eg, the same or not much different), and the relative positions of the target subjects in different images are consistent (eg, the same or not much different). Optionally, the poses of the target subjects in different images are consistent (for example, the poses are the same or the poses are similar). For example, the consistent sizes of the target subjects in different images may include: the contours (or minimum circumscribed rectangles) of the target subjects in different images are relatively consistent. For example, the consistent relative positions of the target subjects in different images may include: the consistent relative positions of the target subjects in different images with respect to the same static object in the background. For example, the center positions (or outlines or minimum enclosing rectangles) of target subjects in different images are consistent with the center positions (or outlines or minimum enclosing rectangles) of the same static object in the background. For example, similar postures may include the same overall postures (such as standing, sitting or lying postures), but differences in local postures (such as different gestures, etc.).

It should be noted that, in an example, the size of the target subject in different images in the Hitchcock zoom video is consistent, which means that the target subject in different images of the Hitchcock zoom video does not have a jump, or jumps The degree is relatively small, so that the user cannot feel the jump, or the user can accept the jump.

It should be noted that, in an example, the relative positions of the target subjects in different images in the Hitchcock zoom video are consistent, which means that the target subjects in different images of the Hitchcock zoom video are still or dynamic The degree of change is small, so that the user does not feel this dynamic change, or the user can accept this dynamic change.

As shown in FIG. 10 , it is a schematic flowchart of a method for shooting a video provided in the embodiment of the present application. This method is applied to the terminal. The terminal includes at least two cameras. The technical solution provided in this embodiment is applied to the scene of shooting Hitchcock zoom video from near to far, that is, in the process of shooting Hitchcock zoom video, the distance between the terminal and the target subject is getting farther and farther .

In this embodiment, the Hitchcock zoom video is shot under the condition that the terminal is farther and farther away from the target subject. When the camera is not switched, the size of the target subject in the image collected later is smaller than the target in the image collected before The size of the subject, and the size of the target subject in different images of the Hitchcock zoom video is consistent, so in order to realize the Hitchcock zoom video, the post-acquisition image needs to be enlarged. However, if the image is directly enlarged, the enlarged image will not be clear.

Based on this, the basic principle of the technical solution provided by this embodiment is: the terminal can use a camera with a larger magnification than the camera used to capture the previous image to capture the subsequent image, that is, by switching to a camera with a larger magnification Realize the enlargement of the target subject, so that, compared with the technical solution of "enlarging the image collected later", it is helpful to improve the definition of the image.

The method shown in Figure 10 may include the following steps:

S300: The terminal determines to shoot the Hitchcock zoom video from near to far, and determines an initial camera.

Optionally, the terminal may determine to shoot the Hitchcock zoom video from near to far under the instruction of the user.

For example, in combination with diagram d in Figure 3, in response to the touch operation on the "Hitchcock zoom video" control, the terminal can also display the "near to far mode" 401 control and the "far to near mode" control on the user interface "402 control, as shown in figure a in Figure 11. Based on this, the user can perform a touch operation on the "near to far mode" 401 control, and in response to the touch operation, the terminal highlights the "near to far mode" 401 control (for example, the border of the control is displayed in bold), as shown in As shown in figure b in FIG. 11 , start shooting Hitchcock zoom video in the near and far mode at the same time.

It should be noted that the terminal can also start shooting the Hitchcock zoom video in the near-to-far mode in other ways (such as voice command mode or shortcut key mode, etc.), which is not specifically limited in this embodiment of the present application.

Because in this embodiment, if the terminal switches the camera, it will switch to a camera with a larger magnification. Therefore, the magnification of the initial camera is usually not the camera with the largest magnification in the terminal. Usually, it can be predefined that the initial camera is a camera with a smaller magnification in the terminal (such as minimum) of a camera.

S301: The terminal collects N+1 images in real time for the first scene, and all of the N+1 images include a target subject. Wherein, during the process of collecting N+1 images, the terminal is getting farther and farther away from the target subject. N is an integer greater than or equal to 1.

Optionally, the N+1 images may be N+1 images continuously collected by the terminal.

Optionally, the first image among the N+1 images is the first image saved by the terminal when the terminal starts shooting in the "Hitchcock zoom video" mode.

Among them, the first scene can be understood as the shooting scene within the field of view of the terminal's camera or the surrounding scene when the terminal executes S301, which is related to the user's environment, the posture of the terminal, or the parameters of the camera. limited.

Wherein, the target subject may be an object, and the position of the target subject may not move during the shooting process, or may move laterally at the same depth.

Alternatively, the target subject may also include multiple objects with the same depth, and the entirety of the multiple objects may serve as the target subject. In some embodiments, when the target subject includes multiple objects, the images of the multiple objects are connected or partially overlapped. During video shooting, as the distance between the user and the target subject changes, the imaging size of objects at different depths varies in different ranges. Therefore, when the distance between the user and the target subject changes, it is difficult for objects of different depths to simultaneously keep the size of the image substantially constant. Therefore, to keep the size of the subject image substantially constant, multiple objects in the subject should have the same depth.

Wherein, the target subject may be automatically determined by the terminal or specified by the user, and the following two cases will be described respectively.

(1) The terminal automatically determines the target subject, and the target subject may include one or more objects.

In some embodiments, the target subject is a predetermined type of object. For example, objects of the preset type are people, animals, famous buildings or landmarks, and the like. The terminal determines an object of a preset type as the target subject based on the preview image.

In some other embodiments, the target subject is an object whose image is located in a central area on the preview image. The target subject that the user is interested in is usually facing the zoom camera, so the image of the target subject on the preview image is usually located in the center area.

In some other embodiments, the target subject is an object whose image on the preview image is close to the central area and whose area is larger than the preset threshold 1 . The target subject that the user is interested in usually faces the zoom camera and is relatively close to the zoom camera, so that the image of the target subject on the preview image is close to the central area and the area is larger than the preset threshold 1.

In some other embodiments, the target subject is an object of a preset type whose image is close to a central area on the preview image.

In some other embodiments, the target subject is a preset type of object whose area on the preview image is close to a central area and whose area is greater than a preset threshold.

In some other embodiments, the target subject is a preset type of object whose image is close to the central area on the preview image and has the smallest depth. When the preset type of objects near the central area of the image on the preview image includes a plurality of objects with different depths, the target object is the object with the smallest depth among them.

In some embodiments, the terminal defaults that the target subject only includes one object.

It can be understood that there may be multiple ways for the terminal to automatically determine the target subject, which is not specifically limited in this embodiment of the present application.

In some embodiments, after determining the target subject, the terminal may prompt the target subject to the user by displaying prompt information or voice broadcast.

For example, the preset type is a person, and the terminal determines that the target subject is a preset type of person 1 whose image is close to a central area on the preview image. Exemplarily, referring to (a) in FIG. 12 , the terminal may select character 1 through block 501 to prompt the user that character 1 is a target subject.

For another example, the preset type is a person, and the terminal determines that the target subject is the person 2 and the person 3 of the preset type whose images on the preview image are close to the central area and have the same depth. For example, referring to (b) in FIG. 12 , the terminal may select character 2 and character 3 through the circle 502 to prompt the user that character 2 and character 3 are target subjects.

For another example, the preset type includes characters and animals, and the terminal determines that the target subject is the preset type of people 4 and animal 1 whose images on the preview image are close to the central area and have the same depth. Exemplarily, referring to (c) in FIG. 12 , the terminal may prompt the user that the character 4 and the animal 1 are target subjects by displaying prompt information.

It can be understood that there may be multiple manners in which the terminal prompts the target subject to the user, and this embodiment of the present application does not specifically limit the manner.

In some embodiments, after the terminal automatically determines the target subject, it may also modify the target subject in response to user operations, such as switching, adding or deleting the target subject.

For example, in the situation shown in (a) in FIG. 13 , the target subject automatically determined by the terminal is person 1, and after the terminal detects that the user clicks on character 5 on the preview image, the terminal automatically clicks on character 5 on the preview image, as shown in (b) in FIG. As shown, change the target subject from character 1 to character 5.

For another example, in the situation shown in (a) in Figure 14, the target subject automatically determined by the terminal is character 1, and after the terminal detects the user's operation of dragging a box to simultaneously select character 1 and character 5, as shown in Figure 14 As shown in (b), the target subject is changed from character 1 to character 1 and character 5.

For another example, in the case shown in (a) in Figure 15, the target subjects automatically determined by the terminal are characters 1 and 5, and after the terminal detects that the user can click on character 5, the terminal will click on the character 5, as shown in (b) in Figure 15 As shown, change the target subject from Character 1 and Character 5 to Character 1.

For another example, the terminal first enters the target subject modification mode according to the user's instruction, and then modifies the target subject in response to the user's operation.

It can be understood that there may be multiple ways for the user to modify the target body, which is not specifically limited in this embodiment of the present application.

(2) The user specifies a target subject, and the target subject includes one or more objects.

After the terminal enters the Hitchcock mode, it can determine the target subject in response to the user's preset operation on the preview interface. This default operation is used to designate one or some objects as the target object. Wherein, the preset operation may be a touch operation, a voice command operation, or a gesture operation, etc., which is not limited in this embodiment of the present application. For example, the touch operation may be single click, double click, long press, pressure press, or an operation of delimiting an object, and the like.

Exemplarily, on the preview interface shown in (a) in FIG. 16 , after the terminal detects that the user double-clicks the operation of character 1 on the preview image, it determines character 1 as the target subject as shown in (b) in FIG. 16 .

In other embodiments, after the terminal enters the Hitchcock mode, the user may be prompted to specify the target subject. Exemplarily, referring to (a) in FIG. 17 , the terminal may display a prompt message: please specify a target subject so that the image size of the target subject during shooting is basically unchanged. Then, the terminal determines the target subject in response to the user's preset operation on the preview interface. For example, after the terminal detects the user's operation of circling the character 1 shown in (a) in FIG. 17 , it determines the corresponding character 1 as the target subject as shown in (b) in FIG. 17 . For another example, the terminal determines character 1 as the target subject after detecting the operation of the user's voice indicating that the character is the target subject.

As another example, in the case that the target subject is a preset object type and the preset object type is a person, referring to (a) in FIG. Subject, so that the image size of the target subject is basically unchanged during shooting? Then, after the terminal responds to the user's operation of clicking the "Yes" control, as shown in (b) in FIG. 18 , it determines that the character is the target subject.

In some embodiments, if the terminal defaults that the target subject only includes one object, when the user specifies multiple objects as the target subject, the terminal may prompt the user: please select only one object as the target subject.

Similar to after the terminal automatically determines the target subject, after the terminal determines the target subject in response to the user's preset operation, it can also prompt the target subject to the user by displaying prompt information or voice broadcast. Moreover, the terminal may also modify the target subject in response to the user's operation, such as switching, adding or deleting the target subject. I won't repeat them here.

Wherein, the terminal collects N+1 images for the first scene in real time, which means that the terminal collects N+1 images for the first scene during the shooting process, instead of the N+1 images for the first scene that have been obtained before shooting. 1 image.

Optionally, as shown in FIG. 19a, S301 may include the following steps S301a-S301d:

S301 a: The terminal uses the initial camera to collect a first image for the first scene, where the first image includes a target subject.

S301b: The terminal uses the initial camera to collect a second image for the first scene, where the second image includes the target subject.

S301c: Based on the shooting magnification of the i-th image, the terminal determines the camera that captures the i+1-th image. i≥2, i is an integer. For specific implementation, refer to the following. The i-th image includes the target subject.

S301d: The terminal collects the i+1th image by using the determined camera that collects the i+1th image.

By analogy, the terminal can collect N+1 images.

In an implementation manner, the N+1 images include N1+1 images collected before and N2 images collected later, where the N1+1 images are collected by the first camera of the terminal, and the N2 images The image is collected by the second camera of the terminal; both N1 and N2 are integers greater than or equal to 1. That is to say, the technical solution provided by the embodiment of the present application can be applied to shooting a Hitchcock zoom video in a scene of switching cameras. Of course, in actual implementation, the camera can be switched multiple times during the shooting of a Hitchcock zoom video.

In this implementation, it can be known from the above S301a-S301d that:

The scaling ratios of the second image to the N1th image in the N1+1 images relative to the first image all belong to the first shooting ratio range. The first shooting magnification range corresponds to the first camera.

The zoom ratios of the N1+1th image in the N1+1 images and the first N2-1 images in the N2 images relative to the first image all belong to the second shooting ratio range. The second shooting magnification range corresponds to the second camera.

In another implementation manner, the N+1 images are all captured by the first camera. That is to say, the technical solution provided by the embodiment of the present application can be applied to shooting a Hitchcock zoom video in a scene without switching cameras.

Optionally, as shown in FIG. 19b, S301c may include the following steps: S301c-1 to S301c-3:

S301c-1: The terminal performs anti-shake processing on the i-th image based on the first image. Specifically, the terminal determines the position of the feature point in the first image, and based on the position of the feature point in the first image, performs motion compensation for the position of the feature point in the i-th image that matches the feature point , so as to implement anti-shake processing on the i-th image.

The embodiment of the present application does not limit the anti-shake processing technology adopted when the terminal performs anti-shake processing. For example, the anti-shake processing technology may be optical anti-shake processing technology, artificial intelligence (artificial intelligence, AI) anti-shake processing technology or electronic Handle anti-shake technology, etc.

It should be noted that S301c-1 is an optional step. After S301c-1 is executed for each of the above N images, as a whole, the video (that is, the collected The next N images) jitter is weaker (ie makes the video overall better stabilized/smooth), allowing greater precision in the obtained zoom ratio.

S301c-2: The terminal acquires the shooting magnification of the i-th image. If the terminal executes S301c-1, the i-th image here is specifically the i-th image after anti-shake processing.

Optionally, the shooting magnification of the i-th image is determined based on the zooming magnification of the i-th image relative to the first image, and the magnification of the camera that captures the first image. The zoom ratio of the i-th image relative to the first image is determined based on the size of the target subject in the i-th image and the size of the target subject in the first image.

For example, ci=c1/(di/d1). where di is the size of the target subject in the i-th image, and d1 is the size of the target subject in the first image. di/d1 is the zoom ratio of the i-th image relative to the first image. c1 is the magnification of the first camera, and ci is the shooting magnification of the i-th image.

Optionally, the size of the target subject in the image is characterized by at least one of the following features 1-4:

Feature 1: The width of the target subject in this image.

Feature 2: The height of the target subject in this image.

Feature 3: The area of the target subject in the image.

Feature 4: the number of pixels occupied by the target subject in the image.

For example, taking feature 2 representing the size of the target subject in an image as an example, the shooting magnification of the i-th image can be obtained based on the formula ci=c1/(hi/h1). where hi is the height of the target subject in the i-th image and h1 is the height of the target subject in the first image. hi/h1 is the zoom ratio of the i-th image relative to the first image. c1 is the magnification of the first camera, and ci is the shooting magnification of the i-th image.

Optionally, S301c-2 may include: the terminal extracts the target subject from the i-th image, and based on the size of the target subject and the size of the target subject in the first image, determine zoom ratio.

The embodiment of the present application does not limit the specific implementation manner of extracting the target subject from the image by the terminal. For example, the terminal extracts the target subject from the image by using one or more of a subject segmentation algorithm, a subject bone point detection algorithm, and a subject contour detection algorithm.

Exemplarily, subject segmentation algorithms include instance segmentation algorithms. Specifically, the terminal uses the instance segmentation algorithm to extract the instance segmentation mask of the target subject from the i-th image, and then divides the size of the instance segmentation mask of the target subject extracted from the i-th image by the size of the instance segmentation mask extracted from the first image The size of the instance segmentation mask of the target subject to obtain the scaling factor of the i-th image relative to the first image. As shown in FIG. 20 , it is a schematic diagram of instance segmentation provided by the embodiment of the present application. Among them, graph a in Fig. 20 represents the i-th image, and graph b in Fig. 20 represents the instance segmentation mask of the target subject in the i-th image. In this mask, pixels with pixel values greater than 0 represent the target The pixels of the subject, and the other regions are the pixels representing the background.

The instance segmentation algorithm is a pixel-level segmentation method, and the accuracy of the target subject extracted based on the instance segmentation algorithm is greater, which helps to make the zoom ratio calculated by the terminal more accurate. For example, when the target subject includes multiple people, it can also effectively distinguish the subject portrait from the background.

S301c-3: If the shooting magnification of the i-th image is within the first shooting magnification range, the terminal determines that the camera that captures the i+1-th image is the first camera. If the shooting magnification of the i-th image is within the second shooting magnification range, it is determined that the camera that captures the i+1-th image is the second camera. The first shooting magnification range corresponds to the first camera, and the second shooting magnification range corresponds to the second camera.

That is to say, based on the shooting magnification of the i-th image, the terminal determines the camera that captures the i+1-th image. Since the size/position of the target subject in two adjacent images does not differ too much, the terminal can capture the i+1th image based on the camera determined by the shooting magnification of the i-th image.

Optionally, the magnification of the first camera is a, and the magnification of the second camera is b; a<b; the first shooting magnification range is [a, b), and the second shooting magnification range is a range greater than or equal to b.

For example, taking the terminal including a wide-angle camera and a main camera as an example, since the magnification of the wide-angle camera is 0.6, and the magnification of the main camera is equal to 1, the shooting magnification range corresponding to the wide-angle camera is [0.6,1), and the main camera The corresponding shooting magnification range is a range greater than or equal to 1.

Further optionally, if the terminal also includes a third camera, and the magnification of the third camera is c, a<b<c, then the first shooting magnification range is [a, b), and the second shooting magnification range is [b , c); the shooting magnification range corresponding to the third camera is greater than or equal to c.

For example, taking a terminal including a wide-angle camera, a main camera and a telephoto camera as an example, since the magnification of the wide-angle camera is 0.6, the magnification of the main camera is 1, and the magnification of the telephoto camera is w, where w is an integer greater than 1, Therefore, the shooting magnification range corresponding to the wide-angle camera is [0.6, 1), the shooting magnification range corresponding to the main camera is [1, w), and the shooting magnification range corresponding to the telephoto camera is a range greater than or equal to w.

Based on this, in example 1, if the first image is captured by a wide-angle camera, and the zoom ratio of the i-th image relative to the first image is calculated to be 0.5, then the shooting ratio of the i-th image is 0.6/0.5 = 1.2. Assuming that the magnification of the telephoto camera is 10 (that is, w=10), since the shooting magnification of the i-th image (that is, 1.2) is within the shooting magnification range corresponding to the main camera (that is, [1, 10)), the terminal determines Use the main camera to capture the i+1th image.

As another example, taking a terminal including a wide-angle camera, a main camera, a first telephoto camera, and a second telephoto camera as an example, since the magnification of the wide-angle camera is 0.6, and the magnification of the main camera is 1, the magnification of the first telephoto camera The magnification is w1, the magnification of the second telephoto camera is w2, 1<w1<w2, then the shooting magnification range corresponding to the wide-angle camera is [0.6, 1), and the shooting magnification range corresponding to the main camera is [1, w1), The shooting magnification range corresponding to the first telephoto camera is [w1, w2), and the shooting magnification range corresponding to the second telephoto camera is a range greater than or equal to w2.

Based on this, in Example 2, if the first image is captured by a wide-angle camera, and the calculated zoom ratio of the i-th image relative to the first image is 0.2, then the shooting ratio of the i-th image is 0.6/0.2 =3. Suppose the magnification of the first telephoto camera is 2 (that is, w1=2), and the magnification of the second telephoto camera is 10 (that is, w2=10), since the shooting magnification of the i-th image (that is, 3) is in the first telephoto The shooting magnification range corresponding to the camera (that is, [2, 10)), therefore, the terminal determines to use the first telephoto camera to capture the i+1th image.

It should be noted that, in specific implementation, the terminal may also switch to the camera when the shooting magnification of the i-th image reaches the minimum critical value of the shooting magnification range corresponding to a certain camera, so as to reduce the The problem of shooting delay caused by switching cameras.

As shown in FIG. 21 , it is a schematic diagram of a process of collecting N+1 images by a terminal provided in the embodiment of the present application. Wherein, the terminal includes a camera 1 to a camera x, where x is an integer greater than or equal to 2, and a camera with a larger number has a larger magnification.

Based on Figure 21, the process for the terminal to collect N+1 images may include the following steps:

The terminal uses camera 1 (that is, the initial camera) to capture the first image.

The terminal uses camera 1 to capture the second image, performs anti-shake processing on the second image based on the first image, and then obtains the zoom ratio of the anti-shake processed second image relative to the first image. If it is determined based on the magnification that the shooting magnification of the second image is within the range of shooting magnification corresponding to the camera a, the camera a is used to capture the third image. Among them, 1≤a≤x.

The terminal uses camera a to capture the third image, performs anti-shake processing on the third image, and then obtains the zoom ratio of the anti-shake processed third image relative to the first image. If it is determined based on the magnification that the shooting magnification of the third image is within the range of shooting magnification corresponding to the camera b, the camera b is used to capture the fourth image. Among them, a≤b≤x.

By analogy, the terminal collects the 4th to N+1th images.

Optionally, the size of the target subject in the image acquired later in the N+1 images is smaller than the size of the target subject in the first image.

In an implementation manner, among the N+1 images, the size of the target subject in the image collected later is smaller than the size of the target subject in the image collected before. As shown in FIG. 22a , it is a schematic diagram of an image collected by a terminal in S301 according to an embodiment of the present application. Figure a in Figure 22a shows the first image collected by the terminal, picture b shows the second image collected by the terminal, and picture c shows the third image collected by the terminal.

In another implementation, in the N+1 images, the size of the target subject in the later captured image may be larger than the target subject in the previous captured image, but smaller than the target in the first image The size of the subject. As shown in FIG. 22b , it is a schematic diagram of an image collected by a terminal in S301 according to an embodiment of the present application. In Fig. 22b, N+1=3 is taken as an example. Figure a in Figure 22b shows the first image collected by the terminal, picture b shows the second image collected by the terminal, and picture c shows the third image collected by the terminal.

An example is used below to illustrate that in the N+1 images, the size of the target subject in the image collected later may be larger than the size of the target subject in the image collected before, but smaller than the size of the target subject in the first image size.

Assuming that the size of the target subject in the first image is d, and the camera that captures the first image is a 1X camera, then, since both the first image and the second image are captured using a 1X camera, the second The size of the target subject in the first image is d/2 as an example, the zoom ratio of the second image relative to the first image is 0.5, therefore, the shooting ratio of the second image is 1/0.5=2. Subsequent terminals can use the 2X camera to capture the third image. On the one hand, since the terminal uses the 2X camera to capture the third image and the 1X camera to capture the second image, it is possible that the size of the target subject in the third image is larger than that in the second image. On the other hand, since the terminal is farther and farther away from the target subject in the process of acquiring the second to third images, the size of the target subject in the third image is smaller than that of the target subject in the first image. size.

Optionally, the method may further include: the terminal displays first information in the current preview interface, and the first information is used to instruct to stop shooting the Hitchcock zoom video.

For example, the terminal may display the first information on the current preview interface when the currently used camera is the camera with the highest magnification in the terminal. For the user, it is possible to stop shooting the Hitchcock zoom video within a period of time after obtaining the first information.

When the distance between the terminal and the target subject is getting farther and farther away, if the currently used camera is the camera with the largest magnification in the terminal, the terminal The camera can no longer be switched. At this time, by displaying the first information on the current preview interface, it is helpful to prompt the user to stop shooting the video in time, otherwise the subsequent images collected by the terminal can only be enlarged to make the target subject in the enlarged image The size of is consistent with the size of the target subject in the first image, which results in poor clarity of the target image generated based on subsequent acquired images. That is to say, the embodiment of the present application provides a method for instructing the user to stop shooting the Hitchcock zoom video, which helps to improve the user experience.

In this embodiment of the present application, there is no limitation on what information is specifically included in the first information to indicate to stop shooting the Hitchcock zoom video. For example, you can directly indicate that "the camera currently used is the camera with the highest magnification in this terminal", or you can indirectly indicate that the camera currently in use is the camera with the highest magnification in this terminal by indicating "Please stop recording video". As shown in Figure 23a Shown, is the schematic diagram of a kind of current preview interface that the embodiment of the present application provides.In the current preview interface, the image 501 (being the current preview image) of the currently played Hitchcock zoom video is included in the current preview interface, and the first information "please stop recording Video" 502.

Optionally, the method may further include: the terminal displays second information in the current preview interface, where the second information is used to indicate that the target subject is still.

For example, the terminal may display the second information on the current preview interface when determining that the position of the target subject in the current preview image is consistent with the position of the target subject in the previous preview image. Since one of the requirements of the Hitchcock zoom video is that the position of the target subject in each image is consistent, so, in this way, the user can know whether the current condition of obtaining the Hitchcock zoom video is satisfied during the process of obtaining the Hitchcock zoom video. Video requirements, thereby improving user experience.

In this embodiment of the present application, there is no limitation on what information is specifically included in the first information to indicate that the target subject is stationary. For example, as shown in FIG. 23b, it is a schematic diagram of a current preview interface provided by the embodiment of the present application. The current preview interface includes an image 501 of the currently presented Hitchcock zoom video (that is, the current preview image) and second information 503 "target subject is still".

Optionally, the method may further include: the terminal displays third information in the current preview interface, the third information is used to indicate that the target subject is in the center of the current preview image. In this way, when the third information is not displayed in the terminal, the user can move the terminal so that the target subject is in the center of the current preview image, which helps to improve the quality of the Hitchcock video.

For example, after the terminal detects the position of the target subject in the current preview image (such as the center of the target subject, or the outline of the target subject or the minimum outer rectangle of the target subject, etc.), When the center of the preview image is in a preset area), the third information is displayed in the current preview interface.

In this embodiment of the present application, there is no limitation on what information is specifically included in the third information to indicate that the target subject is in the center of the current preview image. For example, as shown in FIG. 23c, it is a schematic diagram of a current preview interface provided by the embodiment of the present application. The current preview interface includes an image 501 of the currently presented Hitchcock zoom video (that is, the current preview image) and third information 504 "the target subject is in the center of the current preview image".

Alternatively, the terminal may display fourth information on the current preview interface, where the fourth information is used to indicate that the target subject is not in the center of the current preview image. In this way, when the fourth information is displayed in the terminal, the user can move the terminal so that the target subject is in the center of the current preview image, which helps to improve the quality of the Hitchcock video.

Optionally, collecting N+1 images in real time for the first scene includes: collecting the first image when the target subject is in the center of the current preview image. In this way, it helps to improve the quality of Hitchcock's video.

Optionally, during the process of collecting N+1 images, the moving speed of the terminal is less than or equal to a preset speed.

Since the number of cameras in the terminal is limited, and the terminal moves too fast, the camera may be switched too fast, and when the camera with the maximum magnification is switched, the camera cannot be switched. When using the camera with the maximum magnification to collect images, as the terminal is farther away from the target subject, the target subject in the captured image becomes smaller and smaller. When generating a Hitchcock zoom video, these images need to be processed Zoom in, which results in poor image clarity. Based on this, this possible design is proposed. In this way, it helps to improve the quality of the Hitchcock zoom video.

The embodiment of the present application does not limit the specific value of the preset speed, for example, it may be an empirical value.

S302: For N images collected later among the N+1 images, the terminal performs white balance processing based on a preset neural network to obtain N optimized images. The preset neural network is used to ensure the white balance consistency of adjacent images in time domain.

The preset neural network here may be the white balance network provided above by the embodiment of the present application, such as the white balance network shown in FIG. 8 . The terminal can download the preset neural network in the network device, or can obtain the preset neural network through local training. This embodiment of the present application does not limit it.

Optionally, since there are some differences between the images captured by different cameras, the terminal can take each of the N images collected after the N+1 images, in addition to performing white balance processing, it can also perform Brightness, chromaticity and other parameters are corrected to avoid (or try to avoid) the problem of image inconsistency caused by switching cameras.

For example, for image chroma and luminance, respectively obtain the luminance value/chroma value of the temporally adjacent image, and obtain the multiplicative factor or additive factor of the luminance value/chroma value to the luminance value/chroma value of the image The chromaticity is converted, so that the luminance/chromaticity value of the subsequent image is close to the luminance value/chromaticity value of the previous image after conversion, so that the luminance value/chromaticity value of adjacent images in the temporal domain remains consistent.

S303: The terminal enlarges and cuts the N optimized images to obtain N target images; wherein, the size of the target subject in the N target images is consistent with the size of the target subject in the first image collected in the N+1 images , the relative position of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the size of the N target images is consistent with the first image.

Based on the above description, it can be seen that the size of the target subject in the later acquired image is smaller than the size of the target subject in the first image, therefore, the terminal needs to perform optimization on the corresponding N optimized images of the N later acquired images Zoom in and crop.

The terminal enlarges the N optimized images, so that the size of the target subject in the enlarged N optimized images is consistent with the size of the target subject in the first image, and the relative size of the target subject in the enlarged N optimized images is position, consistent with the relative position of the target subject in the first image. The terminal crops the enlarged N images to obtain N target images, so that each of the N target images has the same size as the first image.

In one example, as shown in FIG. 24 . Diagram a in FIG. 24 represents the first image among the N+1 images, and diagram b in FIG. 24 represents one of the optimized images among the N optimized images. Figure c in Figure 24 represents the image obtained after enlarging the optimized image shown in figure b in Figure 24, wherein the size of the target subject in the enlarged image is the same as that shown in figure a in Figure 24 The size of the target subjects in one image is consistent. Diagram d in FIG. 24 represents the target image obtained after cropping the image shown in diagram c in FIG. 24 , and the size of the target image is the same as that of the first image shown in diagram a in FIG. 24 , wherein , in the cropping process, try to ensure that the position of the target subject in the target image is consistent with the position of the target subject in the first image.

It should be noted that since the "distance between the terminal and the target subject" and "the distance between the terminal and the object in the background" are different, in two different images among the collected N+1 images, this The magnification of the target subject in the two images is different from the magnification of the same object in the background in the two images. This will result in a different background in the target image after zooming in and cropping an optimized image than in the first image. For example, the backgrounds in panels a and d in Figure 24 are different. Based on this, the terminal can generate a Hitchcock zoom video in which "the size and relative position of the target subjects in different images are consistent, but the background is inconsistent" based on the N target images and the first image.

S304: The terminal generates a Hitchcock zoom video based on the N target images and the first image.

The playback time interval between adjacent images in a Hitchcock zoom video can be predefined.

In an implementation manner, the terminal presents the Hitchcock zoom video in real time during shooting. That is, the generated Hitchcock zoom video is presented while generating the Hitchcock zoom video.

In this case, when the terminal executes the above S302 and S303: for the i-th image, after executing S301d, input the i-th image into the preset neural network to obtain an optimized image corresponding to the i-th image. And the optimized image corresponding to the i-th image is enlarged and cropped to obtain the target image corresponding to the optimized image. That is to say, after an image is captured, white balance processing, enlargement, cropping, presentation (that is, displaying the target image) and the like can be performed on the image. And, during the process of processing the image, the next image can be collected and processed. Instead of executing S302 after obtaining N+1 images, and executing S303 after executing S302 for all images.

In another implementation manner, the terminal may execute S302 after obtaining N+1 images, and execute S303 after executing S302 for all images. That is to say, the terminal performs post-processing on the collected N+1 images, so as to obtain the Hitchcock zoom video.

It should be noted that if the camera for capturing images is switched during the process of obtaining N+1 images, the position of the target subject in the different images captured before and after switching the camera may be different due to the switching of the camera, that is to say , the target subject may experience instability.

Based on this, optionally, the method may further include: the terminal acquires the position information of the target subject in the first image, and the position information of the target subject in each of the N target images; then, for For each of the N target images, image stabilization processing is performed on the target image based on the subject image stabilization algorithm and the position information of the target subject in the first image to obtain a new target image. Wherein, the position of the target subject in the new target image is consistent with the position of the target subject in the first image.

For example, if the instance segmentation algorithm is used to obtain the mask of the target subject, the position information of the target subject in the corresponding image can be obtained based on the target subject mask. Combined with the location information of the target subject, the feature point detection is performed on the target subject location area, which can effectively eliminate the influence of feature points outside the target subject area. A new target image can be obtained by performing image stabilization on the feature points of the target subject area.

The embodiment of the present application does not limit the subject image stabilization algorithm. For example, it can be an AI image stabilization algorithm.

Based on this, in the above S304, the terminal generates the Hitchcock zoom video based on the N target images and the first image, which specifically includes: the terminal generates the Hitchcock zoom video based on the N new target images and the first image. In this way, the subject of the resulting Hitchcock zoom video can be stabilized.

In the video shooting method provided in the embodiment of the present application, in the process of obtaining the Hitchcock zoom video, white balance processing is performed on the last N images of the N+1 images collected in real time, so that after processing The image of is consistent with the white balance of the first image in the N+1 images collected. In this way, the white balance effect of the obtained Hitchcock zoom video can be better, thereby improving the quality of the Hitchcock zoom video and improving user experience. In addition, in the embodiment of the present application, the terminal can enlarge the size of the target subject in the subsequent captured image by switching the camera, which helps to make the obtained Hitchcock zoom effect video clearer than the traditional technology. Higher degree, thereby improving user experience.

As shown in FIG. 25 , it is a schematic flowchart of another method for shooting a video provided in the embodiment of the present application. This method is applied to the terminal. The terminal includes at least two cameras. The technical solution provided by this embodiment is applied to the scene of shooting the Hitchcock zoom video from far to near, that is, during the shooting of the Hitchcock zoom video, the distance between the terminal and the target subject is getting closer and closer.

In this embodiment, the Hitchcock zoom video is shot under the condition that the terminal is getting closer and closer to the target subject. When the camera is not switched, the size of the target subject in the image collected later is larger than the target in the image collected before The size of the subject. However, the size of the target subject in different images in the Hitchcock zoom video is the same, so it is necessary to reduce the size of the post-acquisition image. After shrinking the image captured later, it is necessary to perform "edge filling" processing so that the size of the image after "edge filling" is consistent with the size of the first image collected by the terminal, which will lead to "edge filling" The resulting image has black borders, resulting in a poor user experience when the image is rendered.

For example, the first image collected by the terminal is shown in Figure a in Figure 26, the second image is shown in Figure b in Figure 26, and the image obtained after reducing the second image is shown in Figure 26 c As shown in the figure, the image obtained after "filling the edge" on the picture c in Figure 26 is shown in the picture d in Figure 26.

Based on this, the basic principle of the technical solution provided by this embodiment is: the terminal can use a camera with a smaller magnification than the camera used to capture the previous image to capture the subsequent image, that is, by switching to a camera with a smaller magnification Realize the shrinking of the target subject, so that there is no need to "fill the edge" on the captured image, thereby improving user experience.

The method shown in Figure 25 may include the following steps:

S400: The terminal determines to shoot the Hitchcock zoom video from far to near, and determines an initial camera.

Optionally, the terminal may determine to shoot the Hitchcock zoom video from far to near under the instruction of the user.

For example, based on the user interface shown in Figure a in FIG. 11 , the user can click on the "Far to Near Mode" 402 control through a touch operation, and in response to the touch operation, the terminal highlights the "Far to Near Mode" 402 control, Simultaneously start booting to shoot Hitchcock zoom video in near and far mode.

It should be noted that the terminal can also be activated in other ways (such as voice command mode) to shoot the Hitchcock zoom video in the far-to-near mode, which is not specifically limited in this embodiment of the present application.

Because in this embodiment, if the terminal switches the camera, it will switch to a camera with a smaller magnification. Therefore, the magnification of the initial camera is usually not the camera with the smallest magnification in the terminal. Usually, it can be predefined that the camera has a larger magnification in the terminal (such as maximum) of a camera.

S401: The terminal collects N+1 images for the first scene, and all of the N+1 images include a target subject. Wherein, during the process of collecting N+1 images, the terminal is getting closer to the target subject. N is an integer greater than or equal to 1. The first image of the N+1 images is collected by the first camera of the terminal, part or all of the images of the last N images of the N+1 images are collected by the second camera of the terminal, and the The magnification is smaller than that of the first camera. The size of the target subject in the N images acquired later in the N+1 images is smaller than or equal to the size of the target subject in the first image acquired in the N+1 images.

That is to say, in this embodiment, the terminal switches from a high-magnification camera to a small-magnification camera during the process of capturing images, which helps to make the terminal get closer to the target subject in the scene where the The size of the target subject in the acquired image is smaller than or equal to the size of the target subject in the previously acquired image.

In an implementation manner, the N+1 images are N+1 images collected continuously, that is, N+1 images collected in real time.

Optionally, the last N images of the N+1 images include N1 images collected before and N2 images collected later, where the N1 images are collected by the second camera, and the N2 images are obtained by the terminal's Collected by the third camera; both N1 and N2 are integers greater than or equal to 1.

Optionally, collecting N+1 images for the first scene includes: obtaining the shooting magnification of the i-th image in the N+1 images; where, 2≤i≤N, i is an integer; if the i-th image If the shooting magnification is within the first shooting magnification range, then based on the second camera, the i+1th image among the N+1 images is collected for the first scene; if the shooting magnification of the i-th image is within the second shooting magnification range , the i+1th image among the N+1 images is collected for the first scene based on the third camera of the terminal. Wherein, the magnification of the second camera is b, and the magnification of the third camera is c; b>c; the first shooting magnification range is greater than or equal to b; the second shooting magnification range is [c, b). Explanations and examples of relevant content in this optional implementation manner are obtained based on reasoning with reference to the examples above, and will not be repeated here.

Optionally, the shooting magnification of the first image is greater than the magnification of the second camera. That is to say, the initial shooting magnification of the terminal is greater than the magnification of the camera used when capturing the second image. For example, the terminal includes a 5X camera and a 1X camera, the camera used to capture the first image may be a 5X camera, and the shooting magnification at this time may be greater than 5, or [1, 5). The camera used to capture the second image is a 1X camera.

Optionally, the method may further include: displaying first information in the current preview interface, where the first information is used to indicate to stop shooting the Hitchcock zoom video.

For example, the terminal may display the first information on the current preview interface when the currently used camera is the camera with the smallest magnification in the terminal. For the user, it is possible to stop shooting the Hitchcock zoom video within a period of time after obtaining the first information.

When the distance between the terminal and the target subject is getting closer and closer to the Hitchcock zoom video, if the currently used camera is the camera with the smallest magnification in the terminal, since the magnification of the current camera cannot be smaller, the terminal The camera can no longer be switched. At this time, by displaying the first information on the current preview interface, it is helpful to prompt the user to stop shooting the video in time, otherwise the subsequent images collected by the terminal need to be reduced and filled, thereby reducing the playback of Hitchcock zoom User experience during video. That is to say, the embodiment of the present application provides a method for instructing the user to stop shooting the Hitchcock zoom video, which helps to improve the user experience.

Optionally, the method may further include: displaying second information in the current preview interface, where the second information is used to indicate that the target subject is still. Optionally, the method may further include: displaying third information in the current preview interface, where the third information is used to indicate that the target subject is in the center of the current preview image. Optionally, collecting N+1 images for the first scene includes: collecting the first image when the target subject is in the center of the current preview image. For the specific implementation manner and examples, reference may be made to the above, and details will not be repeated here.

In a possible design, the moving speed of the terminal is less than or equal to a preset speed. Since the number of cameras in the terminal is limited, the speed of switching the cameras may be too fast if the terminal moves too fast, and when switching to the camera with the minimum magnification, the camera cannot be switched any more. When using the minimum magnification camera to capture images, as the terminal gets closer to the target subject, the target subject in the later captured image will become larger and larger, which may cause the target subject in the later captured image to be larger than the above The size of the target subject in the first image of N+1 images. When generating a Hitchcock zoom video, these images need to be downscaled and edge filled, thereby degrading the user experience. Based on this, this possible design is proposed. In this way, it helps to improve the quality of high Hitchcock zoom videos.

S402: For the N images collected later among the N+1 images, the terminal performs white balance processing based on a preset neural network to obtain N optimized images. Among them, the preset neural network is used to ensure the white balance consistency of adjacent images in time domain.

S403: The terminal enlarges and crops the N optimized images to obtain N target images. Wherein, the size of the target subject in the N target images is consistent with the size of the target subject in the first image collected in the N+1 images. The relative positions of the target subjects in the N target images are consistent with the relative positions of the target subjects in the first image. The N target images are the same size as the first image.

S404: The terminal generates a Hitchcock zoom video based on the N target images and the first image.

For specific implementation manners of S402-S404, reference may be made to relevant descriptions of S302-S304 above, which will not be repeated here.

In the method for shooting video provided in the embodiment of the present application, in the scene where the terminal is getting closer to the target subject, by switching to a camera with a smaller magnification, the size of the target subject in the later captured image is smaller than or equal to the previous The size of the target subject in the captured image, resulting in a Hitchcock zoom video based on the captured image. Moreover, white balance processing is performed on the last N images among the collected N+1 images, so that the white balance of the processed image is consistent with that of the first image among the collected N+1 images. In this way, the white balance effect of the obtained Hitchcock zoom video can be better, thereby improving the quality of the Hitchcock zoom video and improving user experience. In addition, in the embodiment of the present application, the terminal can reduce the size of the target subject in the subsequent captured image by switching the camera. Compared with the traditional technology, it does not need to perform "filling" processing on the captured image. Therefore, it can Improve user experience.

In addition, the present application also provides a method for shooting a Hitchcock zoom video, which can be applied to a scene where the terminal is getting closer to the target subject. The method may include the steps of:

Step 1: Refer to the above S400.

Step 2: The terminal collects N+1 images in real time for the first scene, and the N+1 images all include the target subject. Wherein, during the process of collecting N+1 images, the terminal is getting closer to the target subject. N is an integer greater than or equal to 1. The first image of the N+1 images is collected by the first camera of the terminal, part or all of the images of the last N images of the N+1 images are collected by the second camera of the terminal, and the The magnification is smaller than that of the first camera.

In some examples, for the specific implementation manner of this step, reference may be made to the relevant description of S301 above, which will not be repeated here.

In some other examples, based on the scheme of determining the camera used to capture the image in S301 above (as shown in Figure 19a), it can be known that the camera used when capturing the first image and the second image in the N+1 images same. However, in this embodiment, during the process of shooting the Hitchcock zoom video, the terminal is getting closer and closer to the target subject, and without switching the camera, the size of the target subject in the image captured later is larger than that captured earlier The size of the target subject in the image. Therefore, using the method shown in FIG. 19a to determine the camera used to capture the image will result in the size of the target subject in the second image being larger than the size of the target subject in the first image.

In this regard, in a solution of the embodiment of the present application, the camera used by the terminal to capture the second image is different from the camera used to capture the first image. Also, the magnification of the camera used to capture the second image is smaller than the magnification of the camera used to capture the first image, which helps to realize that the size of the target image in the second image is smaller than the target subject in the first image The size of the image.

Step 3: Refer to the above S402.

Step 4: The terminal enlarges and crops the optimized image satisfying the first condition among the N optimized images to obtain at least one target image. Wherein, the optimized image satisfying the first condition is an optimized image in which the size of the included target subject is smaller than the size of the target image in the first image. The size of the target subject in the at least one target image is consistent with the size of the target subject in the first image collected in the N+1 images. The relative position of the target subject in the at least one target image is consistent with the corresponding position of the target subject in the first image. The at least one target image is the same size as the first image.

Regarding the processing manner of the optimized image satisfying the first condition, reference may be made to the relevant description in S303, which will not be repeated here.

Step 5: The terminal generates a Hitchcock zoom video based on the at least one target image and the first image.

For example, the terminal generates a Hitchcock zoom video based on the at least one target image, the first image, and an image that does not satisfy the first condition among the N optimized images.

Due to the technical solution provided by this embodiment, the size of the target subject in the image collected later may be greater than, equal to or smaller than the size of the target subject in the first image. Therefore, this embodiment distinguishes optimized images satisfying the first condition from optimized images not satisfying the first condition. For an optimized image that does not satisfy the first condition, it can be directly used as an image in the Hitchcock zoom video without performing magnification and cropping.

For the specific implementation manner of step 5, reference may also be made to the relevant description in S304.

Optionally, the size of the target subject in the N+1th image collected in the N+1th image is smaller than or equal to the size of the target subject in the first image.

Optionally, the size of the target subject in the N+1th image is greater than the size of the target subject in the first image, and the difference between the size of the target subject in the N+1th image and the size of the target subject in the first image The value is less than or equal to the preset threshold. That is to say, when the size of the target subject in the N+1th image is larger than the size of the target subject in the first image, the difference between the two cannot be too large, so that the Hitchcock zoom can be generated based on the N+1th image The video can satisfy "the size of the target subject in different images of the Hitchcock zoom video is consistent".

It should be noted that the Hitchcock zoom video is obtained in the scene where the terminal is getting closer to the target subject. The degree to which the target subject becomes larger is greater than the degree to which the target subject in the captured image shrinks because the terminal switches the current camera to a camera with a smaller magnification (or does not switch the camera), then it may cause the The size of the target subject in the image is larger than the size of the target subject in the first image.

To this:

In a solution of the embodiment of the present application, if the difference between the size of the target subject in the image currently collected by the terminal and the size of the target subject in the first image is greater than a preset threshold, and the target subject in the currently collected image If the size of the target subject is larger than the size of the target subject in the first image, stop acquiring images. Correspondingly, the terminal generates a Hitchcock zoom video by using the previously collected images.

Since one of Hitchcock's zoom video requirements is that the size of the target subject in different images is consistent, "The difference between the size of the target subject in the image currently captured by the terminal and the size of the target subject in the first image is greater than the preset threshold, and the size of the target subject in the currently captured image is greater than the size of the target subject in the first image", indicating that based on the currently collected image of the terminal, the target subject in different images in the Hitchcock zoom video cannot be satisfied. The size is consistent, therefore, the acquisition of images is stopped.

Based on this, optionally, the method may further include: collecting the N+2th image for the first scene, where the N+2th image includes the target subject; the distance between the terminal and the target subject when collecting the N+2 images , which is smaller than the distance between the terminal and the target subject when N+1 images are collected. In this case, the above step 4 includes: the difference between the size of the target subject in the N+2th image and the size of the target subject in the first image is greater than a preset threshold, and the size of the target subject in the N+2th image When the size of the target subject is larger than the size of the target subject in the first image, a Hitchcock zoom video is generated based on the N target images and the first image.

Optionally, when the terminal stops capturing images based on the above solution, it may output first information, where the first information is used to indicate that the shooting of the Hitchcock zoom video is stopped. That is to say, the embodiment of the present application provides a method for instructing the user to stop shooting the Hitchcock zoom video, so that the user can stop the mobile terminal based on the image, thereby improving user experience.

The embodiment of the present application does not limit the specific implementation manner of the first information, for example, the first information may be output in the form of image, text, voice, and the like. In this embodiment, when determining to stop capturing images based on the above solution, the terminal may display the current preview interface as shown in FIG. 23a, thereby prompting the user to stop shooting the Hitchcock zoom video.

In yet another solution of the embodiment of the present application, the N+1 images are collected by a camera with a minimum magnification in the terminal. The size of the target subject in the N+1th image is larger than the size of the target subject in the first image, and the difference between the size of the target subject in the N+1th image and the size of the target subject in the first image is equal to the preset When the threshold is set, first information is output, and the first information is used to instruct to stop shooting the Hitchcock zoom video.

On the one hand, since the N+1 images are collected by the camera with the smallest magnification in the terminal, if images continue to be collected subsequently, the camera cannot be switched. On the other hand, the Hitchcock zoom video requires that the size of the target subject be consistent across images. However, the size of the target subject in the N+1th image is larger than the size of the target subject in the first image, and the difference between the size of the target subject in the N+1th image and the size of the target subject in the first image is equal to the preset When the threshold is set, it means that the size of the target subject in the N+1th image has jumped compared to the size of the target subject in the first image, and has reached the critical value for obtaining the Hitchcock zoom video. At this time, if you continue When collecting images, the size of the target subject in the continued image collection will jump more than the size of the target subject in the first image because the camera can no longer be switched, resulting in the inability to obtain the Hitchcock zoom video. Considering this, the embodiment of the present application provides the above-mentioned method for stopping shooting a Hitchcock zoom video.

In another solution of the embodiment of the present application, the optimized image satisfying the first condition in S403 may be replaced with an optimized image whose size of the included target subject is smaller than that of the target image in the reference image. The reference image is one of the N+1 images "before the image corresponding to the optimized image, the distance between it and the image is the shortest, and the size of the contained target subject is greater than or equal to the target contained in the first image subject size" image.

For example, taking the terminal including 0.6X, 1X, 2X, 5X, and 10X cameras as an example, the shooting magnification ranges corresponding to these cameras are: [0.6,1), [1,2), [2,5), [5,10), the range greater than or equal to 10.

A 10X camera is used to capture the first image, and the size of the target subject in this image is d.

Use the 5X camera to collect the second image, the size of the target subject in this image is 0.8d. It can be seen that the shooting magnification of the second image is: 5/(0.8d/d)=6.25, 6.25∈[5,10), so the camera that captures the third image is a 5X camera.

A 5X camera is used to capture the third image, and the size of the target subject in this image is 1.5d. It can be seen that the shooting magnification of the third image is: 5/(1.5d/d)=3.33, 3.33∈[2,5), so the camera that captures the fourth image is a 2X camera.

Use the 2X camera to collect the fourth image, the size of the target subject in this image is 0.8d. It can be seen that the shooting magnification of the fourth image is: 2/(0.8d/d)=2.5, 2.5∈[2,5), so the camera that captures the fifth image is a 2X camera.

...

Based on this example, the optimized image that satisfies the first condition is the optimized image corresponding to the second image, and the optimized image corresponding to the fourth image. For the optimized image of the 4th image, when zooming in and cropping it, the reference image is the 3rd image. For the optimized image of the second image, when zooming in and cropping it, the reference image is the first image.

As shown in FIG. 27 , it is a schematic flowchart of a method for shooting a video provided in the embodiment of the present application. The method shown in FIG. 27 is applied to a terminal, and the terminal includes at least two cameras, and the magnifications of the at least two cameras are different. The method shown in Figure 27 may include the following steps:

S500: The terminal respectively collects at least two images for the first scene by using at least two cameras at a first moment; wherein, one camera corresponds to one image, and at least two images include a target subject.

That is to say, in the embodiment of the present application, the images collected for the video to be shot are images collected by multiple cameras for the same scene at the same moment.

S501: Based on the preset playback duration and preset playback frame rate of the video, the terminal determines the number N of frames of the image to be inserted between the first image and the second image among the at least two images; wherein, the first image is at least two An image captured by the first camera in the images, where the first camera is the camera with the largest magnification among the at least two cameras. The second image is an image captured by the second camera among the at least two head images, and the second camera is a camera with the smallest magnification among the at least two cameras. N is an integer greater than or equal to 1.

This is taking into account: in the images collected by different cameras for the same scene at the same time, the size of the target subject in the image collected by the camera with the largest magnification is the largest, while the size of the target subject in the image collected by the camera with the smallest magnification is the largest. The size is minimal while the technical solution presented.

S502: The terminal determines N images to be inserted based on the number N of frames of the image to be inserted and some or all of the images in the at least two images. Wherein, the part or all of the images at least include the first image and the second image.

During specific implementation, the terminal first extracts the size of the target subject in each of the collected images, and then determines the value of the pixel corresponding to the image to be inserted based on the size of the target subject in the corresponding image. For a specific example, refer to the example shown in FIG. 28 .

The more images the terminal determines to be inserted based on the at least two images, the more it helps to improve the accuracy of frame interpolation, so that the images in the finally generated video can better reflect the real scene, thereby improving user experience.

S503: The terminal generates a video based on the at least two images and the N images to be inserted. Wherein, the size of the target subject in each image of the video gradually becomes larger or smaller.

In an example, a 10X camera, a 3X camera, a 1X camera and a 0.6X camera are set in the terminal.

When executing S500, the terminal collects images based on the four cameras at the same time to obtain images 1-4, as shown in FIG. 28 . Among them, panels a-d in Fig. 28 represent images 1-4, respectively.

以n＝1为例，待拍摄视频中相邻两帧之间的缩放倍率为

When executing S501, assuming that the preset playback duration of the video to be shot is n seconds, n is an integer greater than or equal to 1, and the preset playback frame rate is 24 frames per second, that is, a total of 24 frames of images are played per second, then the video has a total of The number of images required is n*24. From this, it can be obtained that the zoom ratio between two adjacent frames in the video to be shot is

Taking n=1 as an example, the zoom ratio between two adjacent frames in the video to be shot is

When executing S502, the terminal may perform the following steps:

First, the terminal determines the reference image, and determines the shooting magnification of the image to be inserted based on the reference image and the zoom ratio between two adjacent frames in the video to be shot, and the number of images to be inserted is N. Wherein, the reference image may be an image captured by a camera with a maximum magnification (ie, image 1 ), or an image captured by a camera with a minimum magnification (ie, image 4 ). Taking the reference image as image 4 as an example, the shooting magnifications of the N images to be inserted are: 0.6*1.124=0.6744, 0.6744*1.124=0.758, 0.758*1.124=0.852,...

Secondly, the terminal is based on the shooting magnification of any frame image to be inserted, and the values of pixels in the two images collected by the terminal whose shooting magnification is greater than the shooting magnification of the frame image to be inserted and smaller than the shooting magnification of the frame image to be inserted, Perform frame interpolation to obtain the value of the pixel in the image to be inserted. By analogy, the terminal can obtain N images to be inserted.

It can be understood that before performing frame interpolation, the terminal needs to perform target subject detection on the two images, so as to obtain the size of the target subject. The e-h diagrams in Figure 28 show the schematic diagrams of the target subject detection, and the parts in the rectangular boxes in these figures represent the target subjects. Moreover, the steps of frame insertion are shown in FIG. 28 .

Optionally, the two images are images whose shooting magnification is greater than the shooting magnification of the frame image to be inserted and has the smallest difference with the shooting magnification of the frame image to be inserted, and the shooting magnification is smaller than the shooting magnification and the shooting magnification of the frame image to be inserted The image with the smallest difference in shooting magnification of the frame image to be inserted.

For example, for images to be inserted with a shooting magnification between 0.6-1, image 4 and image 3 are used for frame interpolation. For images to be inserted with shooting magnifications between 1-3, use image 3 and image 2 for frame interpolation. For images to be inserted with shooting magnifications between 3-10, use image 2 and image 1 for frame interpolation.

Finally, the terminal generates the video (or dynamic picture) according to the order of the size of the included target subjects from large to small or from small to large according to the images to be inserted and images 1-4. The steps for generating a video are illustrated in FIG. 28 .

In the traditional technology, the same camera is usually used to collect images under different object distances to generate a video, and the size of the target subject in each image of the video gradually becomes larger or smaller. This may be due to the position deviation of the terminal (such as left and right or up and down) when the images are collected at different times, or the movement of dynamic objects in the background, etc., resulting in large differences in the background of different images, thereby reducing the the quality of the video. In the video shooting method provided in this embodiment, the terminal collects multiple frames of images for the same scene at the same time through multiple cameras, and performs frame interpolation based on the multiple frames of images, so as to generate the video. This helps to improve the quality of the generated video compared to conventional techniques. In addition, it helps to improve the fun of the animation effect and enhance the user's stickiness to the terminal.

The foregoing mainly introduces the solutions provided by the embodiments of the present application from the perspective of methods. In order to realize the above functions, it includes corresponding hardware structures and/or software modules for performing various functions. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed by hardware or computer software drives hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The embodiment of the present application may divide the terminal into functional modules according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

As shown in FIG. 29 , it is a schematic structural diagram of a terminal provided in the embodiment of the present application. The terminal 220 shown in FIG. 29 can be used to realize the functions of the terminal in the foregoing method embodiments, and thus also realize the beneficial effects of the foregoing method embodiments. In the embodiment of the present application, the terminal may be the terminal 100 shown in FIG. 1 .

As shown in FIG. 29 , the terminal 220 includes a collection unit 221 and a processing unit 222 . Optionally, as shown in FIG. 30 , the terminal 220 further includes a display unit 223 .

In some embodiments:

The collection unit 221 is configured to collect N+1 images in real time for the first scene, and each of the N+1 images includes the target subject; wherein, during the process of collecting the N+1 images, the terminal is getting farther and farther away from the target subject . N is an integer greater than or equal to 1. The processing unit 222 is configured to perform the following steps: for the N images collected after the N+1 images, perform white balance processing based on a preset neural network to obtain N optimized images; the preset neural network is used to ensure that the time domain phase White balance consistency of adjacent images. Enlarge and crop the N optimized images to obtain N target images; wherein, the size of the target subject in the N target images is consistent with the size of the target subject in the first image collected in the N+1 images, and the N The relative position of the target subject in the target image is consistent with the relative position of the target subject in the first image; N target images are the same size as the first image; Hitchcock is generated based on the N target images and the first image Zoom video. For example, referring to FIG. 10 , the acquisition unit 221 is configured to execute S301, and the processing unit 222 is configured to execute S302-S304.

Optionally, the N+1 images include N1+1 images collected before and N2 images collected later, where the N1+1 images are collected by the first camera of the terminal, and the N2 images are collected by the terminal's Collected by the second camera; both N1 and N2 are integers greater than or equal to 1.

Optionally, the acquisition unit 221 is specifically configured to: acquire the shooting magnification of the i-th image among the N+1 images, where, 2≤i≤N, i is an integer; if the shooting magnification of the i-th image is at the first Within the shooting magnification range, the i+1th image among the N+1 images is collected based on the first camera for the first scene. If the shooting magnification of the i-th image is within the second shooting magnification range, the i+1th image in the N+1 images collected based on the second camera for the first scene; wherein, the magnification of the first camera is a, The magnification of the second camera is b; a<b; the first shooting magnification range is [a, b); the second shooting magnification range is a range greater than or equal to b. For example, referring to FIG. 19b, the collection unit 221 may be used to execute S301c-3.

Optionally, the shooting magnification of the i-th image is determined based on the scaling magnification of the size of the target subject in the i-th image relative to the size of the target subject in the first image, and the magnification of the camera that captures the first image.

Optionally, the size of the target subject in the ith image is characterized by at least one of the following features: the width of the target subject in the ith image, the height of the target subject in the ith image, the The area of the target subject, or the number of pixels occupied by the target subject in the i-th image.

Optionally, the processing unit 222 is further configured to extract the target subject from the i-th image by using an instance segmentation algorithm, so as to determine the size of the target subject in the i-th image.

Optionally, the display unit 223 is configured to display first information in the current preview interface, and the first information is used to indicate to stop shooting the Hitchcock zoom video.

Optionally, the display unit 223 is configured to display second information in the current preview interface, where the second information is used to indicate that the target subject is still.

Optionally, the display unit 223 is configured to display third information in the current preview interface, and the third information is used to indicate that the target subject is in the center of the current preview image.

Optionally, the acquisition unit 221 is specifically configured to acquire the first image when the target subject is in the center of the current preview image.

Optionally, the display unit 223 is configured to display a user interface, the user interface includes a first control, and the first control is used to indicate shooting a Hitchcock zoom video from near to far; and, receive an operation on the first control. The acquisition unit 221 is specifically configured to, in response to the operation, acquire N+1 images for the first scene in real time.

Optionally, the moving speed of the terminal is less than or equal to a preset speed.

Optionally, the preset neural network is used to predict the white balance gain of the image to be processed in combination with the feature map of the historical network layer, so as to ensure the white balance consistency of adjacent images in the temporal domain; The network layer used for the white balance gain of the image before processing the image and temporally continuous with the image to be processed.

Optionally, the preset neural network is trained based on preset constraint conditions; wherein, the preset constraint conditions include: the white balance gain prediction values for simulating multiple images continuous in time domain are consistent.

Optionally, the processing unit 222 performs white balance processing on the N images acquired after the N+1 images, based on a preset neural network, to obtain N optimized images, specifically for: converting the N+1 images The jth image in is input to the preset neural network to obtain the white balance gain prediction value of the jth image; where, 2≤j≤N-1, j is an integer. Applying the white balance gain prediction value of the jth image to the jth image to obtain an optimized image corresponding to the jth image; wherein, the N optimized images include the optimized image corresponding to the jth image.

In other embodiments:

The collection unit 221 is configured to collect N+1 images in real time for the first scene, and each of the N+1 images includes the target subject; wherein, during the process of collecting the N+1 images, the terminal is getting closer and closer to the target subject ; N is an integer greater than or equal to 1. The first image of the N+1 images is collected by the first camera of the terminal, part or all of the images of the last N images of the N+1 images are collected by the second camera of the terminal, and the The magnification is smaller than that of the first camera. The size of the target subject in the N images acquired later in the N+1 images is smaller than or equal to the size of the target subject in the first image acquired in the N+1 images. The processing unit 222 is configured to perform the following steps: for the N images collected later, perform white balance processing based on a preset neural network to obtain N optimized images; the preset neural network is used to ensure that the white balance of adjacent images in the time domain is consistent sex. Enlarge and crop the N optimized images to obtain N target images. The size of the target subject in the N target images is consistent with the size of the target subject in the first image collected in the N+1 images. The N target The relative position of the target subject in the image is consistent with the relative position of the target subject in the first image. The N target images are the same size as the first image. Based on the N target images and the first image, a Hitchcock zoom video is generated. For example, referring to FIG. 25, the collection unit 221 may be used to execute S401, and the processing unit 222 may be used to execute S402-S404.

Optionally, the N images collected later include N1 images collected before and N2 images collected later, where the N1 images are collected by the second camera, and the N2 images are collected by the third camera of the terminal ; Both N1 and N2 are integers greater than or equal to 1.

Optionally, in terms of acquiring N+1 images for the first scene, the acquisition unit 221 is specifically configured to: acquire the shooting magnification of the i-th image in the N+1 images, where 2≤i≤N, i is an integer; if the shooting magnification of the i-th image is within the range of the first shooting magnification, the i+1-th image among the N+1 images of the first scene is collected based on the second camera. If the shooting magnification of the i-th image is within the second shooting magnification range, the i+1 th image of the N+1 images is collected for the first scene based on the third camera. Wherein, the magnification of the second camera is b, and the magnification of the third camera is c; b>c; the first shooting magnification range is greater than or equal to b; the second shooting magnification range is [c, b).

Optionally, the shooting magnification of the i-th image is determined based on the scaling magnification of the size of the target subject in the i-th image relative to the size of the target subject in the first image, and the magnification of the camera that captures the first image.

Optionally, the size of the target subject in the ith image is characterized by at least one of the following features: the width of the target subject in the ith image, the height of the target subject in the ith image, the The area of the target subject, or the number of pixels occupied by the target subject in the i-th image.

Optionally, the processing unit 222 is further configured to extract the target subject from the i-th image by using an instance segmentation algorithm, so as to determine the size of the target subject in the i-th image.

Optionally, the display unit 223 is configured to display first information in the current preview interface, and the first information is used to indicate to stop shooting the Hitchcock zoom video.

Optionally, the display unit 223 is configured to display second information in the current preview interface, where the second information is used to indicate that the target subject is still.

Optionally, the display unit 223 is configured to display third information in the current preview interface, and the third information is used to indicate that the target subject is in the center of the current preview image.

Optionally, the acquisition unit 221 is specifically configured to: acquire the first image when the target subject is in the center of the current preview image.

Optionally, the display unit 223 is configured to display a user interface, the user interface includes a second control, and the second control is used to indicate shooting a Hitchcock zoom video from far to near; and receive an operation on the second control. The acquisition unit 221 is specifically configured to, in response to the operation, acquire N+1 images for the first scene.

Optionally, the moving speed of the terminal is less than or equal to a preset speed.

Optionally, the preset neural network is used to predict the white balance gain of the image to be processed in combination with the feature map of the historical network layer, so as to ensure the white balance consistency of adjacent images in the temporal domain; The network layer used for the white balance gain of the image before processing the image and temporally continuous with the image to be processed.

Optionally, the preset neural network is trained based on preset constraint conditions; wherein, the preset constraint conditions include: the white balance gain prediction values for simulating multiple images continuous in time domain are consistent.

Optionally, for the N images collected later in the N+1 images, white balance processing is performed based on a preset neural network to obtain N optimized images, including: inputting the j-th image in the N+1 images to Preset the neural network to obtain the white balance gain prediction value of the jth image; where, 2≤j≤N-1, j is an integer. Applying the white balance gain prediction value of the jth image to the jth image to obtain an optimized image corresponding to the jth image; wherein, the N optimized images include the optimized image corresponding to the jth image.

In other embodiments:

The acquisition unit 221 includes a first camera and a second camera, and the magnification of the first camera is different from that of the second camera. The acquisition unit 221 is configured to respectively acquire a first image and a second image for the first scene at a first moment through the first camera and the second camera; wherein, both the first image and the second image contain a target subject. The processing unit 222 is configured to perform the following steps: determine the number N of frames of images to be inserted between the first image and the second image based on the preset playback duration and preset playback frame rate of the video; where N is greater than or equal to 1 an integer of . Based on the frame number N, the first image and the second image, N images to be inserted are determined. A video is generated based on the first image, the second image and the image to be inserted; wherein, the size of the target subject in each image of the video gradually becomes larger or smaller. For example, referring to FIG. 27, the collection unit 221 may be used to execute S500, and the processing unit 222 may be used to execute S501-S503.

Optionally, the acquisition unit 221 further includes a third camera, and the magnification of the third camera is between those of the first camera and the second camera. The acquisition unit 221 is further configured to acquire a third image for the first scene at the first moment through the third camera; wherein, the third image includes the target subject. The processing unit 222 determines N images to be inserted based on the number of frames, the size of the target subject in the first image, and the size of the target subject in the second image, specifically for: based on the number of frames N, the size of the first image and The aspect of determining the N images to be inserted for the second image is specifically used for: determining the N images to be inserted based on the frame number N, the first image, the second image and the third image.

For a specific description of the foregoing optional manners, reference may be made to the foregoing method embodiments, and details are not repeated here. In addition, for the explanations and descriptions of beneficial effects of any terminal 220 provided above, reference may be made to the corresponding method embodiments above, and details are not repeated here.

As an example, with reference to FIG. 1 , the above acquisition unit may be implemented by a camera 193 . The above-mentioned functions of the processing unit 222 can all be implemented by the processor 110 calling the program code stored in the internal memory 121 .

Another embodiment of the present application also provides a terminal, including: a processor, a memory, and a camera, the camera is used to collect images, the memory is used to store computer programs and instructions, and the processor is used to call computer programs and instructions, and cooperate with the camera to execute the above-mentioned Corresponding steps performed by the terminal in the method flow shown in the method embodiment.

Another embodiment of the present application further provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed on a terminal, various steps performed by the terminal in the method flow shown in the above method embodiments are executed.

In some embodiments, the disclosed methods can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

It should be understood that the arrangements described herein are for example purposes only. Accordingly, those skilled in the art will appreciate that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and groups of functions, etc.) can be used instead, and some elements may be omitted altogether depending on the desired result. . In addition, many of the described elements are functional entities that may be implemented as discrete or distributed components, or implemented in conjunction with other components in any suitable combination and location.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When computer-executed instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or may contain one or more data storage devices such as servers and data centers that can be integrated with the medium. Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (solid state disk, SSD)) and the like.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (65)

1. A method for shooting video, which is applied to a terminal and comprises the following steps:

acquiring N +1 images in real time aiming at a first scene, wherein the N +1 images comprise target main bodies; wherein, in the process of collecting the N +1 images, the terminal is farther and farther away from the target main body; n is an integer of 1 or more;

carrying out white balance processing on the later acquired N images in the N +1 images based on a preset neural network to obtain N optimized images; the preset neural network is used for predicting the white balance gain of the image to be processed to ensure the white balance consistency of adjacent images in a time domain, and the image to be processed is an image in the adjacent images in the time domain;

amplifying and cutting the N optimized images to obtain N target images; wherein the size of the target subject in the N target images is consistent with the size of the target subject in a first image acquired in the N +1 images, and the relative position of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the N target images are consistent with the first image in size;

generating a Hirschk zoom video based on the N target images and the first image.

2. The method of claim 1,

the N +1 images comprise N1+1 images acquired in the front and N2 images acquired in the back, wherein the N1+1 images are acquired by a first camera of the terminal, and the N2 images are acquired by a second camera of the terminal; and both the N1 and the N2 are integers which are more than or equal to 1.

3. The method of claim 1 or 2, wherein the acquiring N +1 images in real-time for a first scene comprises:

acquiring the shooting magnification of the ith image in the N +1 images; wherein i is not less than 2 and not more than N, and i is an integer;

acquiring an i +1 th image in the N +1 images aiming at the first scene based on a first camera of the terminal if the shooting magnification of the i image is in a first shooting magnification range;

if the shooting magnification of the ith image is within a second shooting magnification range, acquiring an (i + 1) th image in the (N + 1) th images aiming at the first scene based on a second camera of the terminal;

wherein the multiplying power of the first camera is a, and the multiplying power of the second camera is b; a is less than b; the first photographing magnification range is [ a, b); the second shooting magnification range is a range of b or more.

4. The method of claim 3,

the shooting magnification of the ith image is determined based on the scaling magnification of the size of the target subject in the ith image relative to the size of the target subject in the first image and the magnification of a camera which acquires the first image.

5. The method of claim 4, wherein the size of the target subject in the ith image is characterized by at least one of:

the width of the target subject in the ith image,

the height of the target subject in the ith image,

the area of the target subject in the ith image, or,

the number of pixel points occupied by the target subject in the ith image.

6. The method according to claim 4 or 5, characterized in that the method further comprises:

extracting the target subject from the ith image using an example segmentation algorithm to determine a size of the target subject in the ith image.

7. The method of any of claims 1 to 6, further comprising:

and displaying first information in the current preview interface, wherein the first information is used for indicating that the shooting of the Hockey zoom video is stopped.

8. The method of any one of claims 1 to 7, further comprising:

and displaying second information in the current preview interface, wherein the second information is used for indicating that the target main body is static.

9. The method according to any one of claims 1 to 8, further comprising:

and displaying third information in the current preview interface, wherein the third information is used for indicating that the target main body is in the center of the current preview image.

10. The method of any of claims 1 to 9, wherein the acquiring N +1 images in real-time for a first scene comprises:

the first image is acquired when the target subject is in the center of the current preview image.

11. The method of any one of claims 1 to 10, further comprising:

displaying a user interface, wherein the user interface comprises a first control used for indicating that a Cock zoom video in a Hirsch area is shot from near to far;

the acquiring N +1 images in real time for a first scene includes:

receiving an operation for the first control, and acquiring the N +1 images for the first scene in real time in response to the operation.

12. The method according to any one of claims 1 to 11, wherein the moving speed of the terminal is equal to or less than a preset speed.

13. The method of any one of claims 1 to 12,

the preset neural network is used for predicting the white balance gain of the image to be processed by combining the characteristic diagram of the historical network layer so as to ensure the white balance consistency of the adjacent images in the time domain; wherein the historical network layer is a network layer used in predicting a white balance gain of an image preceding and temporally continuous with the image to be processed.

14. The method of claim 13,

the preset neural network is obtained by training based on preset constraint conditions; wherein the preset constraint condition comprises: the predicted values of the white balance gains for a plurality of images which are continuous in an analog time domain are consistent.

15. The method according to any one of claims 1 to 14, wherein the performing white balance processing on N later-acquired images of the N +1 images based on a preset neural network to obtain N optimized images comprises:

inputting a jth image in the N +1 images into the preset neural network to obtain a white balance gain predicted value of the jth image; wherein j is more than or equal to 2 and less than or equal to N-1, and j is an integer;

applying the white balance gain predicted value of the jth image to obtain an optimized image corresponding to the jth image; wherein the N optimized images comprise the optimized image corresponding to the jth image.

16. A method for shooting video, which is applied to a terminal and comprises the following steps:

acquiring N +1 images aiming at a first scene, wherein the N +1 images comprise target main bodies; in the process of acquiring the N +1 images, the terminal is closer to the target main body; n is an integer of 1 or more; a first image in the N +1 images is acquired by a first camera of the terminal, and part or all of the last N images in the N +1 images are acquired by a second camera of the terminal, wherein the multiplying power of the second camera is smaller than that of the first camera; the size of the target subject in later acquired N images of the N +1 images is smaller than or equal to the size of the target subject in a first image acquired in the N +1 images;

performing white balance processing on the N images based on a preset neural network to obtain N optimized images; the preset neural network is used for predicting the white balance gain of an image to be processed to ensure the white balance consistency of adjacent images in a time domain, and the image to be processed is an image in the adjacent images in the time domain;

amplifying and cutting the N optimized images to obtain N target images; wherein the size of the target subject in the N target images is consistent with the size of the target subject in the first image, and the relative position of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the N target images are consistent with the first image in size;

generating a Hirschk zoom video based on the N target images and the first image.

17. The method according to claim 16, wherein the N images comprise N1 images acquired before and N2 images acquired after, wherein the N1 images are acquired by the second camera and the N2 images are acquired by a third camera of the terminal; and both the N1 and the N2 are integers which are more than or equal to 1.

18. The method of claim 16 or 17, wherein acquiring N +1 images for a first scene comprises:

acquiring the shooting magnification of the ith image in the N +1 images; wherein i is not less than 2 and not more than N, and i is an integer;

acquiring an i +1 th image of the N +1 images for the first scene based on the second camera if the shooting magnification of the i image is within a first shooting magnification range;

if the shooting magnification of the ith image is within a second shooting magnification range, acquiring an (i + 1) th image in the (N + 1) th images aiming at the first scene based on a third camera of the terminal;

wherein the magnification of the second camera is b, and the magnification of the third camera is c; b is more than c; the first shooting magnification range is a range greater than or equal to b; the second photographing magnification range is [ c, b).

19. The method of claim 18,

the shooting magnification of the ith image is determined based on the zoom magnification of the size of the target subject in the ith image relative to the size of the target subject in the first image, and the magnification of a camera that captures the first image.

20. The method of claim 19, wherein the size of the target subject in the ith image is characterized by at least one of:

the width of the target subject in the ith image,

the height of the target subject in the ith image,

the area of the target subject in the ith image, or,

the number of pixel points occupied by the target subject in the ith image.

21. The method according to claim 19 or 20, further comprising:

extracting the target subject from the ith image using an example segmentation algorithm to determine a size of the target subject in the ith image.

22. The method of any one of claims 16 to 21, further comprising:

and displaying first information in the current preview interface, wherein the first information is used for indicating that the shooting of the Hockey zoom video is stopped.

23. The method of any one of claims 16 to 22, further comprising:

and displaying second information in the current preview interface, wherein the second information is used for indicating that the target main body is static.

24. The method of any one of claims 16 to 23, further comprising:

and displaying third information in the current preview interface, wherein the third information is used for indicating that the target main body is in the center of the current preview image.

25. The method of any of claims 16 to 24, wherein acquiring N +1 images for a first scene comprises:

the first image is acquired when the target subject is in the center of the current preview image.

26. The method of any one of claims 16 to 25, further comprising:

displaying a user interface, wherein the user interface comprises a second control used for indicating that a Cock zoom video in a Hirsch area is shot from far to near;

the acquiring N +1 images for a first scene includes:

receiving an operation on the second control, and acquiring the N +1 images for the first scene in response to the operation.

27. The method according to any one of claims 16 to 26, wherein the moving speed of the terminal is equal to or less than a preset speed.

28. The method according to any one of claims 16 to 27,

the preset neural network is used for predicting the white balance gain of the image to be processed by combining the characteristic diagram of the historical network layer so as to ensure the white balance consistency of the adjacent images in the time domain; wherein the historical network layer is a network layer used in predicting a white balance gain of an image preceding and temporally continuous with the image to be processed.

29. The method of claim 28,

the preset neural network is obtained by training based on preset constraint conditions; wherein the preset constraint condition comprises: the predicted values of the white balance gains for a plurality of images which are continuous in an analog time domain are consistent.

30. The method according to any one of claims 16 to 29, wherein the performing white balance processing on N later-acquired images of the N +1 images based on a preset neural network to obtain N optimized images comprises:

inputting a jth image in the N +1 images into the preset neural network to obtain a white balance gain predicted value of the jth image; wherein j is more than or equal to 2 and less than or equal to N-1, and j is an integer;

applying the white balance gain predicted value of the jth image to obtain an optimized image corresponding to the jth image; wherein the N optimized images comprise the optimized image corresponding to the jth image.

31. A method for shooting videos is applied to a terminal, the terminal comprises a first camera and a second camera, the magnification of the first camera is different from that of the second camera, and the method comprises the following steps:

respectively acquiring a first image and a second image for a first scene at a first time through the first camera and the second camera; wherein the first image and the second image both contain a target subject;

determining the number N of frames of the images to be inserted between the first image and the second image based on the preset playing duration and the preset playing frame rate of the video; wherein N is an integer of 1 or more;

determining N images to be inserted based on the frame number N, the first image and the second image;

generating the video based on the first image, the second image and the N images to be inserted; wherein the size of the target subject in each image of the video gradually becomes larger or smaller.

32. The method of claim 31, wherein the terminal further comprises a third camera having a magnification between the magnifications of the first camera and the second camera, the method further comprising:

acquiring, by the third camera, a third image for the first scene at the first time; wherein the third image contains the target subject;

the determining N images to be inserted based on the frame number, the first image, and the second image includes:

and determining N images to be inserted based on the frame number, the first image, the second image and the third image.

33. A terminal, characterized in that the terminal comprises:

the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring N +1 images in real time aiming at a first scene, and the N +1 images comprise target main bodies; wherein, in the process of collecting the N +1 images, the terminal is farther and farther away from the target main body; n is an integer of 1 or more;

a processing unit for performing the steps of:

carrying out white balance processing on the later acquired N images in the N +1 images based on a preset neural network to obtain N optimized images; the preset neural network is used for predicting the white balance gain of the image to be processed to ensure the white balance consistency of adjacent images in a time domain, and the image to be processed is an image in the adjacent images in the time domain;

amplifying and cutting the N optimized images to obtain N target images; wherein the size of the target subject in the N target images is consistent with the size of the target subject in a first image acquired in the N +1 images, and the relative position of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the N target images are consistent with the first image in size;

generating a Hirschk zoom video based on the N target images and the first image.

34. The terminal of claim 33,

the N +1 images comprise N1+1 images acquired in the front and N2 images acquired in the back, wherein the N1+1 images are acquired by a first camera of the terminal, and the N2 images are acquired by a second camera of the terminal; and both the N1 and the N2 are integers which are more than or equal to 1.

35. The terminal according to claim 33 or 34, wherein the acquisition unit is specifically configured to:

acquiring the shooting magnification of the ith image in the N +1 images; wherein i is not less than 2 and not more than N, i is an integer;

acquiring an i +1 th image in the N +1 images aiming at the first scene based on a first camera of the terminal if the shooting magnification of the i image is in a first shooting magnification range;

if the shooting magnification of the ith image is within a second shooting magnification range, acquiring an (i + 1) th image in the (N + 1) th images aiming at the first scene based on a second camera of the terminal;

wherein the multiplying power of the first camera is a, and the multiplying power of the second camera is b; a is less than b; the first photographing magnification range is [ a, b); the second shooting magnification range is a range of b or more.

36. The terminal of claim 35,

the shooting magnification of the ith image is determined based on the zoom magnification of the size of the target subject in the ith image relative to the size of the target subject in the first image, and the magnification of a camera that captures the first image.

37. The terminal of claim 36, wherein the size of the target subject in the ith image is characterized by at least one of:

the width of the target subject in the ith image,

the height of the target subject in the ith image,

the area of the target subject in the ith image, or,

the number of pixel points occupied by the target subject in the ith image.

38. The terminal according to claim 36 or 37,

the processing unit is further configured to extract the target subject from the ith image using an instance segmentation algorithm to determine a size of the target subject in the ith image.

39. The terminal according to any of claims 33 to 38, characterized in that the terminal further comprises:

and the display unit is used for displaying first information in the current preview interface, wherein the first information is used for indicating that the shooting of the Hirschhorn zoom video is stopped.

40. The terminal according to any of claims 33 to 39, wherein the terminal further comprises:

and the display unit is used for displaying second information in the current preview interface, wherein the second information is used for indicating that the target main body is static.

41. The terminal according to any of claims 33 to 40, characterized in that the terminal further comprises:

and the display unit is used for displaying third information in the current preview interface, wherein the third information is used for indicating that the target main body is in the center of the current preview image.

42. A terminal according to any of claims 33 to 41,

the acquisition unit is specifically configured to acquire the first image when the target subject is in the center of the current preview image.

43. The terminal according to any of claims 33 to 42, characterized in that the terminal further comprises:

the display unit is used for displaying a user interface, the user interface comprises a first control, and the first control is used for indicating that a Cock zoom video in a Hirsch area is shot from near to far; and receiving an operation directed to the first control;

the acquiring unit is specifically configured to acquire, in response to the operation, the N +1 images for the first scene.

44. A terminal as claimed in any one of claims 33 to 43, wherein the speed of movement of the terminal is less than or equal to a predetermined speed.

45. The terminal according to any of claims 33 to 44,

the preset neural network is used for predicting the white balance gain of the image to be processed by combining the characteristic diagram of the historical network layer so as to ensure the white balance consistency of the adjacent images in the time domain; wherein the historical network layer is a network layer used in predicting a white balance gain of an image preceding and temporally continuous with the image to be processed.

46. The terminal of claim 45,

the preset neural network is obtained by training based on preset constraint conditions; wherein the preset constraint condition comprises: the predicted values of the white balance gains for a plurality of images which are continuous in an analog time domain are consistent.

47. The terminal according to any one of claims 33 to 46, wherein the processing unit is configured to, in the respect of performing white balance processing on the N images acquired later in the N +1 images based on a preset neural network to obtain N optimized images, specifically:

inputting a jth image in the N +1 images into the preset neural network to obtain a white balance gain predicted value of the jth image; wherein j is more than or equal to 2 and less than or equal to N-1, and j is an integer;

applying the white balance gain predicted value of the jth image to obtain an optimized image corresponding to the jth image; wherein the N optimized images comprise the optimized image corresponding to the jth image.

48. A terminal, characterized in that the terminal comprises:

the system comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring N +1 images aiming at a first scene, and the N +1 images comprise target bodies; wherein, in the process of acquiring the N +1 images, the terminal is closer to the target main body; n is an integer of 1 or more; a first image in the N +1 images is acquired by a first camera of the terminal, and part or all of the last N images in the N +1 images are acquired by a second camera of the terminal, wherein the multiplying power of the second camera is smaller than that of the first camera; the size of the target subject in later-acquired N images of the N +1 images is smaller than or equal to the size of the target subject in a first image acquired in the N +1 images;

a processing unit for performing the steps of:

performing white balance processing on the N images based on a preset neural network to obtain N optimized images; the preset neural network is used for predicting the white balance gain of an image to be processed to ensure the white balance consistency of adjacent images in a time domain, and the image to be processed is an image in the adjacent images in the time domain;

amplifying and cutting the N optimized images to obtain N target images; wherein the size of the target subject in the N target images is consistent with the size of the target subject in the first image, and the relative position of the target subject in the N target images is consistent with the relative position of the target subject in the first image; the N target images are consistent with the first image in size;

generating a Hirschk zoom video based on the N target images and the first image.

49. The terminal of claim 48, wherein the N images comprise N1 images captured in front and N2 images captured in back, wherein the N1 images are captured by the second camera and the N2 images are captured by a third camera of the terminal; and both the N1 and the N2 are integers which are more than or equal to 1.

50. The terminal of claim 48 or 49, wherein the acquisition unit is specifically configured to:

acquiring the shooting magnification of the ith image in the N +1 images; wherein i is not less than 2 and not more than N, and i is an integer;

acquiring an i +1 th image of the N +1 images for the first scene based on the second camera if the shooting magnification of the i image is within a first shooting magnification range;

acquiring an i +1 th image in the N +1 images for the first scene based on a third camera of the terminal if the shooting magnification of the i image is within a second shooting magnification range;

wherein the magnification of the second camera is b, and the magnification of the third camera is c; b is more than c; the first shooting magnification range is a range greater than or equal to b; the second photographing magnification range is [ c, b).

51. The terminal of claim 50,

the shooting magnification of the ith image is determined based on the zoom magnification of the size of the target subject in the ith image relative to the size of the target subject in the first image, and the magnification of a camera that captures the first image.

52. The terminal of claim 51, wherein the size of the target subject in the ith image is characterized by at least one of:

the width of the target subject in the ith image,

the height of the target subject in the ith image,

the area of the target subject in the ith image, or,

the number of pixel points occupied by the target subject in the ith image.

53. The terminal according to claim 51 or 52,

the processing unit is further configured to extract the target subject from the ith image using an instance segmentation algorithm to determine a size of the target subject in the ith image.

54. The terminal according to any of claims 48 to 53, wherein the terminal further comprises:

and the display unit is used for displaying first information in the current preview interface, wherein the first information is used for indicating that the shooting of the Hirschhorn zoom video is stopped.

55. The terminal according to any of the claims 48 to 54, characterized in that the terminal further comprises:

and the display unit is used for displaying second information in the current preview interface, and the second information is used for indicating that the target main body is static.

56. The terminal according to any of claims 48 to 55, characterized in that the terminal further comprises:

and the display unit is used for displaying third information in the current preview interface, wherein the third information is used for indicating that the target main body is in the center of the current preview image.

57. A terminal as claimed in any one of claims 48 to 56,

the acquisition unit is specifically configured to: the first image is acquired when the target subject is in the center of the current preview image.

58. A terminal as claimed in any of claims 48 to 57, further comprising:

the display unit is used for displaying a user interface, the user interface comprises a second control, and the second control is used for indicating that the Koch zoom video is shot from far to near; and receiving an operation directed to the second control;

the acquisition unit is specifically configured to, in response to the operation, acquire the N +1 images for the first scene.

59. A terminal as claimed in any one of claims 48 to 58, wherein the speed of movement of the terminal is less than or equal to a predetermined speed.

60. The terminal according to any of the claims 48 to 59,

the preset neural network is used for predicting the white balance gain of the image to be processed by combining the characteristic diagram of the historical network layer so as to ensure the white balance consistency of the adjacent images in the time domain; wherein the historical network layer is a network layer used in predicting a white balance gain of an image preceding and temporally continuous with the image to be processed.

61. The terminal according to claim 60,

the preset neural network is obtained by training based on preset constraint conditions; wherein the preset constraint condition comprises: the predicted values of the white balance gains for a plurality of images which are continuous in an analog time domain are consistent.

62. The terminal according to any one of claims 48 to 61, wherein the performing white balance processing on the N later-collected images in the N +1 images based on a preset neural network to obtain N optimized images comprises:

inputting a jth image in the N +1 images into the preset neural network to obtain a white balance gain predicted value of the jth image; wherein j is not less than 2 and not more than N-1, j is an integer;

applying the white balance gain predicted value of the jth image to obtain an optimized image corresponding to the jth image; wherein the N optimized images comprise an optimized image corresponding to the jth image.

63. A terminal is characterized by comprising an acquisition unit and a processing unit, wherein the acquisition unit comprises a first camera and a second camera, and the multiplying power of the first camera is different from that of the second camera;

the acquisition unit is used for respectively acquiring a first image and a second image aiming at a first scene at a first time through the first camera and the second camera; wherein the first image and the second image both contain a target subject;

the processing unit is used for executing the following steps:

determining the number N of frames of the images to be inserted between the first image and the second image based on the preset playing duration and the preset playing frame rate of the video; wherein N is an integer of 1 or more;

determining N images to be inserted based on the frame number N, the first image and the second image;

generating the video based on the first image, the second image and the N images to be inserted; wherein the size of the target subject in each image of the video gradually becomes larger or smaller.

64. The terminal of claim 63, wherein the capture unit further comprises a third camera having a magnification between the magnifications of the first camera and the second camera;

the acquisition unit is further configured to acquire, by the third camera, a third image for the first scene at the first time; wherein the third image includes the target subject;

the processing unit, in the aspect of determining N images to be inserted based on the frame number, the first image, and the second image, is specifically configured to:

and determining N images to be inserted based on the frame number, the first image, the second image and the third image.

65. A terminal, comprising: a processor, a memory for storing computer programs and instructions, and a camera, the processor for invoking the computer programs and instructions to perform the method of any of claims 1-32 in cooperation with the camera.

Images