WO2022068537A1

WO2022068537A1 - Image processing method and related apparatus

Info

Publication number: WO2022068537A1
Application number: PCT/CN2021/116944
Authority: WO
Inventors: 张超; 刘宏马; 张雅琪; 贾志平
Original assignee: 华为技术有限公司
Priority date: 2020-09-29
Filing date: 2021-09-07
Publication date: 2022-04-07
Also published as: CN115379112A; CN113556461B; CN113556461A

Abstract

The present application discloses an image processing method, being applied to the field of video recording or photographing. Said method comprises: in the process of photographing or video recording by an electronic device, cropping, by means of a cropping frame having a preset size and a preset movement trajectory selected by a user, an image acquired by a camera, and displaying, in a photographing preview interface or a video recording interface, the image in the cropping frame. In this way, the photographed video or picture can have an automatic zooming effect, and detail features of a photographed object can be highlighted.

Description

An image processing method and related device

This application claims the priority of the Chinese patent application with the application number 202011053345.5 and the application title "An image processing method and related device" filed with the China Patent Office on September 29, 2020, the entire contents of which are incorporated herein by reference middle.

technical field

The present application relates to the technical field of computer vision, and in particular, to an image processing method and related apparatus.

Background technique

With the development of smart terminals, taking pictures and videos has become one of its most important features. When a user uses a portable electronic device such as a mobile phone to shoot, there will be a need for zooming or telephoto, so as to clearly capture a distant scene or capture a wider landscape.

At present, when an electronic device such as a mobile phone records a video or displays a preview screen, if the user wants to adjust the framing focal length of the screen, the electronic device such as the mobile phone only supports the user's manual zoom adjustment, and the user operation is complicated. When an electronic device such as a mobile phone records a video, after the user can only manually adjust the focal length once, the electronic device such as the mobile phone can only record with the focal length adjusted by the user, so that the recorded video cannot reflect the details of the subject in the video. Moreover, the user's focusing operation is complicated.

SUMMARY OF THE INVENTION

The present application provides an image processing method, a related device, and a related device, which can make the shot video or picture have the effect of automatic zooming, and highlight the detailed characteristics of the shooting subject.

In a first aspect, the present application provides an image processing method, including: an electronic device displays a first shooting preview interface, the first shooting preview interface includes a first preview frame, and the first preview frame displays real-time capture by a camera of the electronic device obtained screen. After the electronic device detects the first operation on the first shooting preview interface, the electronic device displays a shooting option interface, where the shooting option interface includes a first shooting option and a second shooting option. After the electronic device detects the second operation for the first shooting option, the electronic device displays a second shooting preview interface, the second shooting preview interface includes a second preview frame, and the second preview frame displays the camera of the electronic device real-time captured images. The electronic device starts to capture the first video content. At the first moment after starting to shoot the first video content, the electronic device displays in the second preview frame the first part of the picture captured in real time by the camera of the electronic device. At a second moment after starting to shoot the first video content, the electronic device displays, in the second preview frame, the second part of the picture captured in real time by the camera of the electronic device. The electronic device displays the shooting option interface. After the electronic device detects the third operation for the second shooting option, the electronic device displays a third shooting preview interface, the third shooting preview interface includes a third preview frame, and the third preview frame displays the camera of the electronic device real-time captured images. The electronic device starts capturing the second video content. At the first moment after starting to shoot the second video content, the electronic device displays, in the third preview frame, the third part of the picture captured in real time by the camera of the electronic device. At a second moment after starting to shoot the second video content, the electronic device displays, in the third preview frame, the fourth part of the picture captured in real time by the camera of the electronic device, wherein the first part and the second part are , the third part and the fourth part are different.

The present application provides an image processing method, in which an electronic device can use a cropping frame with a preset size and a preset motion trajectory to crop an image captured by a camera during a photo-taking or video-recording process, and display the image in the cropping frame on the in the preview interface or the recording interface. In this way, the shot video or picture can have the effect of automatic zooming, and the detailed characteristics of the shooting subject can be highlighted.

In a possible implementation manner, before the electronic device starts to capture the first video content, the electronic device detects a fourth operation on the second capture preview interface. In response to the fourth operation, the electronic device starts to capture the first video content.

In a possible implementation manner, the second shooting preview interface further includes an option switching control; after the electronic device displays the second part of the picture captured in real time by the camera of the electronic device in the second preview frame, the The electronic device detects a fifth operation for the option toggle control. In response to the fifth operation, the electronic device displays the photographing option interface. In this way, after a shooting option has been selected, the user can be provided with an entry for changing the shooting option, so that the user can change the shooting option to shoot video content.

In a possible implementation manner, after the electronic device displays the second part of the picture captured in real time by the camera of the electronic device in the second preview frame, the method further includes: 2. Shoot the sixth operation of the preview interface. In response to the sixth operation, the electronic device saves the first part as a video frame at a first moment in the video file, and saves the second part as a video frame at a second moment in the video file. In this way, the electronic device can shoot video recordings with effects such as automatic zoom or lens shift.

In a possible implementation manner, the method further includes: detecting, by the electronic device, a seventh operation at a first moment after starting to capture the first video content. In response to the seventh operation, the electronic device saves the first part as a picture. In this way, the electronic device can save a preview video image as a picture during a preview process with effects such as automatic zooming or lens panning.

In a possible implementation manner, the second shooting preview interface further includes a first window. The electronic device displays, in the first window, a picture captured in real time by a camera of the electronic device. At a first moment after starting to capture the first video content, the electronic device displays the bounding box of the first portion in the first window. At a second moment after starting to capture the first video content, the electronic device displays the bounding box of the second portion in the first window. In this way, it is convenient for the user to see the picture captured by the camera in real time, and it is convenient for the user to find the subject to be photographed.

In a possible implementation manner, the second shooting preview interface further includes a first window. At a first moment after starting to shoot the first video content, the electronic device displays, in the first window, a video frame at the first moment in the video sample corresponding to the first shooting option. At a second moment after starting to shoot the first video content, the electronic device displays a video image at the second moment in the video sample corresponding to the first shooting option in the first window. In this way, it is convenient for users to compare the shooting effects in real-time with the video samples.

In the implementation of the present application, the display position of the first window and the second preview frame is any one of the following: at least a partial area of the first window overlaps with the display area of the second preview frame; or, the first window A window is displayed outside the second preview frame in the second shooting preview interface; alternatively, the first window is displayed in the upper right corner of the second preview frame; alternatively, the first window is displayed in the second preview frame The upper left area of the preview box.

In a possible implementation manner, the first window includes a window closing control; the electronic device detects an eighth operation for the window closing control in the first window; in response to the eighth operation, the electronic device Close the display of this first window. In this way, when the user does not need the first window, the user can be manually triggered to close, so as to ensure that the shooting picture displayed on the electronic device is not blocked.

In a possible implementation manner, after the electronic device closes and displays the first window, the electronic device displays a window opening control in the second shooting preview interface; the electronic device detects the ninth window opening control for the window opening control. operation; in response to the ninth operation, the electronic device displays the first window on the second shooting preview interface. In this way, when the user needs to call the small window, the user can trigger the electronic device to redisplay the first window.

In a possible implementation manner, the second shooting preview interface further includes a mirror movement mode closing control. The electronic device detects a tenth operation acting on the mirror motion mode off control. In response to the tenth operation, the electronic device displays the first shooting preview interface. In this way, the user can trigger to close the shooting mode at any time.

In a possible implementation manner, the second shooting preview interface further includes a second window, and the electronic device displays the waveform diagram of the first music segment corresponding to the first shooting option in the second window. Wherein, the audio information in the waveform chart at the first moment after starting to shoot the first video content is different from the audio information in the waveform chart at the second moment after starting to shoot the first video content, and the audio information includes the following One or more of: tempo information, amplitude information, and style information.

Wherein, after the electronic device starts to shoot the first video content, the electronic device can play the first music segment.

In a possible implementation manner, when the electronic device saves the first part as a video picture at the first moment in the video file, and saves the second part as a video picture at the second moment in the video file, the electronic The device saves the first piece of music in the audio data of the video file.

In a possible implementation manner, the electronic device performs image stabilization processing on the image captured in real time by the camera of the electronic device before displaying the image captured in real time by the camera of the electronic device, and the image stabilization processing includes one of the following: Or more: electronic image stabilization EIS processing, optical image stabilization processing, mechanical image stabilization processing. In this way, the shooting picture can be made smooth and excessive.

In a possible implementation manner, before the electronic device displays in the second preview frame the first part of the picture captured by the camera of the electronic device in real time, the electronic device performs image super-resolution reconstruction on the first part. Before the electronic device displays the second part of the picture captured in real time by the camera of the electronic device in the second preview frame, the electronic device performs image super-resolution reconstruction on the second part. Before the electronic device displays the third part of the picture captured by the camera of the electronic device in real time in the third preview frame, the electronic device performs image super-resolution reconstruction on the third part. Before the electronic device displays the fourth part of the picture captured by the camera of the electronic device in real time in the third preview frame, the electronic device performs image super-resolution reconstruction on the fourth part. In this way, the shooting picture can be made clearer.

In a possible implementation manner, the camera of the electronic device includes a wide-angle camera and a non-wide-angle camera. The first preview frame displays the image captured by the non-wide-angle camera in real time. The second preview frame displays the picture captured by the wide-angle camera in real time. The third preview frame displays the image captured by the wide-angle camera in real time. In this way, the captured picture can have a larger viewing angle.

In a second aspect, the present application provides an electronic device, including: a processor, a camera, and a touch screen. The touch screen may be used to display a first shooting preview interface, where the first shooting preview interface includes a first preview frame, and the first preview frame displays a picture captured by the camera in real time. The processor is further configured to instruct the touch screen to display a shooting option interface after detecting a first operation on the first shooting preview interface, where the shooting option interface includes a first shooting option and a second shooting option. The processor is further configured to instruct the touch screen to display a second shooting preview interface after detecting a second operation for the first shooting option, the second shooting preview interface includes a second preview frame, and the second preview frame displays the The picture captured by the camera of the electronic device in real time. The processor is further configured to instruct the camera to start shooting the first video content. The processor is further configured to instruct the touch screen to display the first part of the picture captured by the camera in real time in the second preview frame at a first moment after starting to shoot the first video content. The processor is further configured to instruct the touch screen to display the second part of the picture captured by the camera in real time in the second preview frame at a second moment after starting to shoot the first video content. The touch screen is also used to display the shooting option interface. The processor is further configured to instruct the touch screen to display a third shooting preview interface after detecting a third operation for the second shooting option, the third shooting preview interface includes a third preview frame, and the third preview frame displays the The picture captured by the camera of the electronic device in real time. The processor is further configured to instruct the camera to start shooting the second video content. The processor is further configured to instruct the touch screen to display the third part of the picture captured by the camera in real time in the third preview frame at the first moment after starting to shoot the second video content. The processor is further configured to instruct the touch screen to display the fourth part of the picture captured by the camera in real time in the third preview frame at a second moment after starting to shoot the second video content, wherein the first part, the The second part, the third part and the fourth part are all different.

In a possible implementation manner, before the processor instructs the camera to start shooting the first video content, the processor is further configured to detect a fourth operation on the second shooting preview interface. The processor is specifically configured to instruct the camera to start shooting the first video content in response to the fourth operation.

In a possible implementation manner, the second shooting preview interface further includes an option switching control. After the processor instructs the touch screen to display the second part of the picture captured by the camera of the electronic device in real time in the second preview frame, the processor is further configured to detect a fifth operation for the option switching control. The processor is specifically configured to display the shooting option interface in response to the fifth operation. In this way, after a shooting option has been selected, the user can be provided with an entry for changing the shooting option, so that the user can change the shooting option to shoot video content.

In a possible implementation manner, after the processor instructs the touch screen to display the second part of the picture captured by the camera in real time in the second preview frame, the processor is further configured to detect that the preview for the second shooting is The sixth operation of the interface. The processor is further configured to, in response to the sixth operation, save the first part as a video picture at a first moment in the video file, and save the second part as a video picture at a second moment in the video file. In this way, the electronic device can shoot video recordings with effects such as automatic zoom or lens shift.

In a possible implementation manner, the processor is further configured to detect the seventh operation at a first moment after starting to shoot the first video content. The processor is further configured to save the first part as a picture in response to the seventh operation. In this way, the electronic device can save a preview video image as a picture during a preview process with effects such as automatic zooming or lens panning.

In a possible implementation manner, the second shooting preview interface further includes a first window. The touch screen is also used for displaying the picture captured in real time by the camera of the electronic device in the first window. The processor is further configured to instruct the touch screen to display the bounding box of the first part in the first window at a first moment after starting to shoot the first video content. The processor is further configured to instruct the touch screen to display the bounding box of the second part in the first window at a second moment after starting to shoot the first video content. In this way, it is convenient for the user to see the picture captured by the camera in real time, and it is convenient for the user to find the subject to be photographed.

In a possible implementation manner, the second shooting preview interface further includes a first window. The processor is further configured to instruct the touch screen to display the video image at the first moment in the video sample corresponding to the first shooting option in the first window at the first moment after the first video content is started to be captured. The processor is further configured to instruct the touch screen to display, in the first window, a video image at the second moment in the video sample corresponding to the first shooting option at a second moment after the first video content is started to be captured. In this way, it is convenient for users to compare the shooting effects in real-time with the video samples.

In a possible implementation manner, the first window includes a window closing control; the processor is further configured to detect an eighth operation for the window closing control in the first window. The processor is further configured to close and display the first window in response to the eighth operation. In this way, when the user does not need the first window, the user can be manually triggered to close, so as to ensure that the shooting picture displayed on the electronic device is not blocked.

In a possible implementation manner, after the electronic device closes and displays the first window. The touch screen is further used for displaying a window opening control in the second shooting preview interface. The processor is further configured to detect the ninth operation of opening the control for the window. The processor is further configured to instruct the touch screen to display the first window on the second shooting preview interface in response to the ninth operation. In this way, when the user needs to call the small window, the user can trigger the electronic device to redisplay the first window.

In a possible implementation manner, the second shooting preview interface further includes a mirror movement mode closing control. The processor is further configured to detect a tenth operation acting on the mirror movement mode closing control. The processor is further configured to instruct the touch screen to display the first shooting preview interface in response to the tenth operation. In this way, the user can trigger to close the shooting mode at any time.

In a possible implementation manner, the second shooting preview interface further includes a second window. The touch screen is further configured to display the waveform diagram of the first music segment corresponding to the first shooting option in the second window. Wherein, the audio information in the waveform chart at the first moment after starting to shoot the first video content is different from the audio information in the waveform chart at the second moment after starting to shoot the first video content, and the audio information includes the following One or more of: tempo information, amplitude information, and style information. In this way, the video picture can be automatically zoomed following the rhythm of the music, which enhances the visual effect of the shooting picture.

Wherein, the electronic device may further include a speaker. The processor is further configured to instruct the speaker to play the first music segment after starting to shoot the first video content.

In a possible implementation manner, when the processor saves the first part as a video picture at the first moment in the video file, and saves the second part as a video picture at the second moment in the video file, the processor , and is also used to save the first music segment in the audio data of the video file.

In a possible implementation manner, the processor is further configured to perform image stabilization processing on the images captured in real time by the camera before displaying the images captured in real time by the camera, and the image stabilization processing includes one or more of the following: Electronic image stabilization EIS processing, optical image stabilization processing, mechanical image stabilization processing. In this way, the shooting picture can be made smooth and excessive.

In a possible implementation manner, the processor is further configured to perform image super-resolution reconstruction on the first part before the touch screen displays the first part of the picture acquired by the camera in real time in the second preview frame. The processor is further configured to perform image super-resolution reconstruction on the second part before the touch screen displays the second part of the picture acquired by the camera in real time in the second preview frame. The processor is further configured to perform image super-resolution reconstruction on the third part before the touch screen displays the third part of the picture acquired by the camera in real time in the third preview frame. The processor is further configured to perform image super-resolution reconstruction on the fourth part before the touch screen displays the fourth part of the picture acquired by the camera in real time in the third preview frame. In this way, the shooting picture can be made clearer.

In a possible implementation manner, the camera of the electronic device includes a wide-angle camera and a non-wide-angle camera.

The first preview frame displays the image captured by the non-wide-angle camera in real time. The second preview frame displays the picture captured by the wide-angle camera in real time. The third preview frame displays the image captured by the wide-angle camera in real time. In this way, the captured picture can have a larger viewing angle.

In a third aspect, the present application provides an electronic device including a touch screen, a camera, one or more processors and one or more memories. The one or more processors are coupled to the touch screen, the camera, and one or more memories for storing computer program code, the computer program code including computer instructions, when the one or more processors execute the computer When the instruction is given, the electronic device is caused to execute the image processing method in any possible implementation manner of any one of the foregoing aspects.

In a fourth aspect, the present application provides an electronic device, including: one or more functional modules. One or more functional modules are used to execute the image processing method in any possible implementation manner of any of the above aspects.

In a fifth aspect, an embodiment of the present application provides a computer storage medium, including computer instructions, which, when the computer instructions are run on an electronic device, cause the communication apparatus to execute the image processing method in any of the possible implementations of any of the above aspects .

In a sixth aspect, an embodiment of the present application provides a computer program product that, when the computer program product runs on a computer, enables the computer to execute the image processing method in any of the possible implementations of any of the foregoing aspects.

Description of drawings

FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

2 is a schematic diagram of a software structure of an electronic device provided by an embodiment of the present application;

3A-3Q are schematic diagrams of a group of interfaces provided by the embodiments of the present application;

4A-4F are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

5A-5E are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

6A-6H are schematic diagrams of a set of interfaces provided by another embodiment of the present application;

7A-7C are schematic diagrams of position changes of the cropping frame on the original image provided by the embodiments of the present application;

7D-FIG. 7F are schematic diagrams of a group of cropping effects of an original image by using a cropping frame according to an embodiment of the present application;

8A-8C are schematic diagrams of position changes of a cropping frame on an original image provided by another embodiment of the present application;

FIG. 8D-FIG. 8G are schematic diagrams of a group of cropping effects of an original image by using a cropping frame according to an embodiment of the present application;

9A-9F are schematic diagrams of position changes of the cropping frame on the original image provided by the embodiments of the present application;

9G-9K are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

10A-10K are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

11A-11E are schematic diagrams of a set of interfaces provided by another embodiment of the present application;

12A-12D are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

13A-13E are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

14A-14E are schematic diagrams of a group of interfaces provided by another embodiment of the present application;

15A-15E are schematic diagrams of a set of interfaces provided by another embodiment of the present application;

16 is a schematic diagram of waveform comparison of a sound signal and a rhythm signal provided by an embodiment of the application;

17A-17J are schematic diagrams of a set of interfaces provided by another embodiment of the present application;

18 is a schematic diagram of a module architecture of a video stream cropping system provided by an embodiment of the present application;

19 is a schematic diagram of the hardware architecture of a video stream cropping system provided by an embodiment of the application;

FIG. 20 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be cleared and described in detail below with reference to the accompanying drawings. Wherein, in the description of the embodiments of the present application, unless otherwise specified, “/” means or, for example, A/B can mean A or B; “and/or” in the text is only a description of an associated object The association relationship indicates that there can be three kinds of relationships, for example, A and/or B can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application , "plurality" means two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as implying or implying relative importance or implying the number of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the "multiple" The meaning is two or more.

FIG. 1 shows a schematic structural diagram of an electronic device 100 .

The embodiment will be described in detail below by taking the electronic device 100 as an example. It should be understood that the electronic device 100 shown in FIG. 1 is only an example, and the electronic device 100 may have more or fewer components than those shown in FIG. 1 , two or more components may be combined, or Different component configurations are possible. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2. Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, And a subscriber identification module (subscriber identification module, SIM) card interface 195 and so on. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may contain multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

The PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the wireless communication module 160 . For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include global positioning system (global positioning system, GPS), global navigation satellite system (global navigation satellite system, GLONASS), Beidou navigation satellite system (beidou navigation satellite system, BDS), quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

The electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Wherein, in some embodiments of the present application, the camera 193 may include multiple cameras. The plurality of cameras may include wide-angle cameras and non-wide-angle cameras. In a possible implementation manner, when the electronic device 100 enables the mirror motion mode in the following embodiment, the electronic device 100 can obtain the video stream by using a wide-angle camera. In this way, the electronic device 100 can obtain a video picture with a wider viewing angle.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

In some embodiments, the electronic device 100 needs to identify the image content in the shooting picture, detect and track the position of the designated image content, and determine the outline of the designated image content. The application processor can send the preprocessed RGB image (or BGR image or single-channel image or grayscale image, etc.) to the NPU. The NPU can detect and track the position of the specified image content in the RGB image (or BGR image or single-channel image or grayscale image, etc.) through the AI model, and determine the outline of the specified image content. The NPU can output the detection frame information or outline information of the specified image content to the application processor. The application processor may determine the cropping frame information according to the detection frame information/or outline information of the specified image. The application processor may crop the video stream based on the cropping frame information, or the application processor may instruct the ISP to crop the video stream based on the cropping frame information.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound near the microphone 170C through the human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The earphone jack 170D is used to connect wired earphones. The earphone interface 170D may be the USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenarios.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance. The electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light to the outside through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present invention takes an Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100 as an example.

FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.

The application layer can include a series of application packages.

As shown in Figure 2, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

AndroidRuntime includes core libraries and virtual machines. Androidruntime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

In the following, the workflow of the software and hardware of the electronic device 100 is exemplarily described in conjunction with the capturing and photographing scene.

When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the camera application icon, for example, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer. The camera 193 captures still images or video.

The present application provides an image processing method, in which the electronic device 100 can use a cropping frame with a preset size and a preset motion trajectory to crop an image captured by a camera during a photographing or video recording process, and display the image in the cropping frame. In the preview interface or recording interface. In this way, the shot video or picture can have the effect of automatic zooming, and the detailed characteristics of the shooting subject can be highlighted.

The following describes an image processing method involved in an embodiment of the present application in combination with an application scenario.

In some application scenarios, the electronic device 100 may receive a camera movement template selected by the user before recording. The lens motion template may include a size change rule of the cropping frame and a motion track of the cropping frame on the original image acquired by the camera. After the electronic device 100 starts recording, the electronic device 100 can use the cropping frame defined in the lens motion template to crop each frame of the original image captured by the camera, and display the cropped image on the video recording interface in frame order. . In this way, the video shot by the electronic device 100 can have the effect of automatic zooming, which increases the viewing quality of the video.

Exemplarily, as shown in FIG. 3A , the electronic device 100 may display an interface 310 with a home screen, the interface 310 displays a page on which application icons are placed, and the page includes a plurality of application icons (eg, weather application icons, stock application icon, calculator application icon, settings application icon, mail application icon, gallery application icon 312, music application icon, video application icon, browser application icon, etc.). A page indicator is also displayed below the multiple application icons to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator are a plurality of tray icons (eg, a dialing application icon, an information application icon, a contact application icon, and a camera application icon 311 ), and the tray icons remain displayed when the page is switched. In some embodiments, the above-mentioned page may also include multiple application icons and page indicators. The page indicator may not be a part of the page, but exists alone. The above-mentioned picture icon is also optional, which is not limited in this embodiment of the present application.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the camera application icon 311, and in response to the input operation, the electronic device 100 may display the photographing interface 320 as shown in FIG. 3B .

As shown in FIG. 3B, the shooting interface 320 may include an echo control 321, a shooting control 322, a camera conversion control 323, a picture captured by the camera 324, a setting control 325, a zoom ratio control 326, and one or more shooting mode controls ( For example, "Night Scene Mode" control 327A, "Portrait Mode" control 327B, "Large Aperture" control 327C, "Normal Photo Mode" control 327D, "Video Mode" control 327E, "Pro Mode" control 327F, More Mode Control 327G ). The echo control 321 can be used to display the captured image. The shooting control 322 is used to trigger saving of the image captured by the camera. The camera switching control 323 can be used to switch cameras for taking pictures. The setting control 325 can be used to set the camera function. The zoom ratio control 326 can be used to set the zoom ratio of the camera. Wherein, the zoom magnification control 326 can trigger the electronic device 100 to display a zoom slide bar, and the zoom slide bar can receive the user's upward (or downward) sliding operation, so that the electronic device 100 increases (or decreases) the zoom magnification of the camera . Possibly, the zoom magnification control 326 can display a zoom increase control and a zoom decrease control on the electronic device 100, and the zoom increase control can be used to receive and respond to the user's input to trigger the electronic device 100 to increase the zoom magnification of the camera; The widget can be used to receive and respond to the user's input to trigger the electronic device 100 to reduce the zoom ratio of the camera. The shooting mode control can be used to trigger the opening of the image processing flow corresponding to the shooting mode. For example, a "night scene mode" control 372A may be used to trigger an increase in brightness and color richness, etc. in a captured image. The "Portrait Mode" control 327B may be used to trigger blurring of the background of the person in the captured image. The “large aperture” control 327C can be used to trigger the electronic device 100 to invoke an image processing algorithm to highlight the subject and blur the background. As shown in FIG. 3B , the shooting mode currently selected by the user is the "normal shooting mode".

The electronic device 100 may receive a user input operation (eg, a click) on the "more" control 327G, and in response to the input operation, the electronic device 100 may display the mode selection page 330 as shown in FIG. 3C.

As shown in FIG. 3C, the mode selection page 330 includes one or more mode controls (eg, "HDR Mode" control, "Time Lapse Mode" control, "Watermark Mode" control, "Slow Motion Mode" control, " "Intelligent Object Recognition Mode" control, "Document Correction Mode" control, "Panorama Mode" control, "Streamer Shutter Mode" control, "3D Dynamic Mode" control, "Online Translation Mode" control, "Intelligent Mirror Mode" control 331, etc. Wait).

In a possible implementation manner, the mode selection page 330 may be transparently displayed on the picture 324 captured by the above-mentioned camera.

The electronic device 100 may receive an input operation (eg, single click) of the user acting on the "smart mirror movement mode" control 331, and in response to the input operation, the electronic device 100 may display the mirror movement template interface 340 as shown in FIG. 3D .

As shown in Figure 3D, the motion template selection interface 340 includes one or more motion template options (eg, "travel" motion option 341A, "comfortable" motion option 341B, "kinetic" motion option 341C, etc. ), selection box 342 , mirror movement template display area 351 , mirror movement track display area 353 , intelligent mirror movement prompt box 343 , echo control 346 , and mirror movement shooting control 345 . A mirror movement template play control 352 is displayed on the mirror movement template display area 351 . There is a close control 344 in the smart mirror movement prompt box 343 , and the close control 344 can be used to trigger the electronic device 100 to exit the mirror movement template selection interface 340 . Wherein, the size change rules of the cropping frame or the motion trajectory of the cropping frame corresponding to different mirror motion template options are different. As shown in FIG. 3D , the “travel” camera movement option 341A has been selected in the selection box 342 , and the video sample corresponding to the “travel” camera movement option 341A is displayed in the camera movement template display area 351 . The camera movement track display area 353 may display the cropping frame size change rule corresponding to the "travel" camera movement option 341A and/or the movement track of the cropping frame in the original image. For example, the center point of the crop box corresponding to the "travel" motion option 341A may be at the same location as the center point of the original image, and the size of the crop box corresponding to the "travel" motion option 341A may first decrease proportionally from the size of the original image over time. to the preset minimum size, and then scale up from the preset minimum size to the size of the original image. The size of the cropping frame corresponding to the "travel" lens movement option 341A changes in proportion to the size of the original image. This preset minimum size is smaller than the size of the original image.

The size change rules of the cropping frame corresponding to different mirror motion templates and/or the motion trajectory of the cropping frame in the original image will be described in detail in subsequent embodiments, and will not be repeated here.

In a possible implementation manner, the lens movement template selection interface 340 may be transparently displayed on the picture 324 captured by the above-mentioned camera.

The electronic device 100 may receive an input operation (eg, click) that the user acts on the camera movement template playback control 352, and in response to the input operation, as shown in FIG. 3E, the electronic device 100 may display the template in the camera movement track display area 353. The picture, the crop box 355, and the center point 354 of the crop box 355. The electronic device 100 may crop the template picture based on the cropping frame 355 to obtain a cropped picture. The electronic device 100 can adjust the size of the cropped picture to the display size of the moving-motion template display area 351 and display it in the display area 351 of the moving-motion template. In a possible implementation manner, the template picture may be each frame picture in a piece of video footage.

As shown in FIG. 3F, the cropping frame 355 is reduced to a preset minimum size. Wherein, the smaller the cropping frame 355 is, the larger the magnification of the picture displayed in the mirror movement template display area 351 is.

As shown in FIG. 3G , after the cropping frame 355 is reduced to a preset minimum size, it can be enlarged again. The magnification in the picture displayed in the lens movement template display area 351 gradually decreases from the highest value.

As shown in FIG. 3H , the cropping frame 355 can be gradually enlarged to the size of the original picture, and then reduced to a preset size after being enlarged to the size of the original picture.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the camera movement control 345, and in response to the input operation, the electronic device 100 may display the recording interface 360 as shown in FIG. The size change rule of the cropping frame corresponding to the "travel" mirror motion template 341A and/or the motion track of the cropping frame in the original image is used to crop the video stream obtained by the camera during video shooting.

As shown in FIG. 3I , the video recording interface 360 may include a preview image 371 obtained by the camera, an echo control 346, a video capture control 361, a mirror movement template switching control 362, an intelligent mirror movement prompt box 343, and a mirror movement template preview window 363 . Wherein, the mirror movement template preview window 363 displays the cropping effect picture corresponding to the selected mirror movement template (for example, the mirror movement template corresponding to the above-mentioned "travel" mirror movement option 341A has been selected). The lens-moving template preview window 363 includes a closing control 364, and the closing control 364 can be used to trigger the electronic device 100 to close the lens-moving template preview window 363. The display position of the mirror movement template preview window 363 may be at the upper right corner of the display screen or at the upper left corner of the display screen, which is not limited herein. The camera movement template switching control 362 may display the name of the camera movement template selected by the electronic device 100 (for example, "travel"). The moving template switching control 362 can be used to trigger the electronic device 100 to switch the moving template. In a possible implementation manner, the electronic device 100 may receive and respond to a user's input operation (eg, hold and drag) on the mirror movement template preview window 363, and move the display position of the mirror movement template preview window 363.

The electronic device 100 may receive an input operation (such as a single click) that the user acts on the video capture control 361, and in response to the input operation, the electronic device 100 may start to change the rule and/or crop the size of the cropping frame corresponding to the selected mirror motion template. The motion trajectory of the frame in the original image is cropped for each frame of the video stream obtained by the camera. The electronic device 100 may adjust the cropped image of each frame to a specified size and display it in the video recording interface 360 in sequence of frames. The electronic device 100 may save the cropped video stream.

As shown in FIG. 3J , after the electronic device 100 starts recording, the electronic device 100 can display the recording time frame 366 and the recording end control 365 on the recording interface 360 , and replace the echo control 346 and the mode switching control 362 to pause the recording. Controls 367 and Camera Controls 368. The recording time frame 366 may be used to display the recording time of the electronic device 100 . The recording end control 365 can be used for the electronic device 100 to end the recording. The pause recording control 367 can be used to trigger the electronic device 100 to pause recording. The photographing control 368 can be used to trigger the electronic device 100 to save a certain frame or multiple frames of video recordings as a picture during the video recording process. The electronic device 100 may crop each frame of the original image acquired by the camera through the cropping frame, and capture the image within the cropping frame. The electronic device 100 can adjust the image in the cropping frame to a specified size and display it on the video recording interface 360 .

For example, the center point of the crop box may be at the same location as the center point of the original image. When the electronic device 100 receives the user's input operation on the video capture control 361 within the first 8s, the size of the cropping frame can be gradually reduced from the size of the original image stream to the preset minimum size in proportion. When the electronic device 100 From the 8th second to the 16th second after receiving the input operation of the user acting on the video shooting control 361, the size of the cropping frame is gradually increased proportionally to the size of the original image. As shown in FIG. 3J , when the electronic device 100 receives the third second after receiving the user's input operation acting on the video capture control 361, the electronic device 100 can obtain the original image P1 through the camera, and determine that the cropping frame is in the original image P1. position in image P1. The electronic device 100 may cut out the image within the cropping frame on the original image P1, and adjust the cutout image to a specified size to obtain the cropped image 372. The electronic device 100 may display the cropped image 372 on the recording interface 360 . The electronic device 100 may display the effect picture of the selected mirror movement template at the third second in the mirror movement template preview window 363 . The magnification ratio of the image content in the cropped image 372 is greater than the magnification ratio of the image content in the preview image 371 shown in FIG. 3I.

As shown in FIG. 3K , when the electronic device 100 receives the input operation of the user acting on the video capture control 361 at the 8th second, the cropping frame is reduced to the preset minimum size, and the electronic device 100 can obtain the original image through the camera P2, and determine the position of the cropping frame in the original image P2. The electronic device 100 may cut out the image in the cropping frame on the original image P2, and adjust the cutout image to a specified size to obtain the cropped image 373. The electronic device 100 may display the cropped image 373 on the recording interface 360 . The electronic device 100 may display the effect picture of the selected mirror motion template at the 8th second in the template preview window 363 . The magnification of the image content in the cropped image 373 is greater than the magnification of the image content in the cropped image 372 shown in FIG. 3J.

As shown in FIG. 3L , when the electronic device 100 receives the user's input operation on the video capture control 361 in the 12th second, the size of the cropping frame is larger than the preset minimum size and smaller than the size of the frame image, the electronic device 100 can The original image P3 is acquired through the camera, and the position of the cropping frame in the original image P3 is determined. The electronic device 100 may cut out the image within the cropping frame on the original image P3, and adjust the cutout image to a specified size to obtain the cropped image 374. The electronic device 100 may display the cropped image 374 on the recording interface 360 . The electronic device 100 may display, in the template preview window 363, the effect picture of the selected mirror motion template at the 12th second. Wherein, the magnification ratio of the image content in the cropped image 374 is smaller than the magnification ratio of the image content in the cropped image 373 shown in FIG. 3K.

The electronic device 100 may receive an input operation (eg, click) that the user acts on the recording end control 365, and in response to the input operation, the electronic device 100 may end the recording, and save the video stream after the video stream is cropped by the cropping frame corresponding to the lens motion template. Record video. As shown in FIG. 3M , after the electronic device 100 finishes recording, the electronic device 100 can display the pause recording control 367 and the photographing control 368 with the echo control 346 and the mode switching control 362 , and display the captured recording on the echo control 346 The image frame in the finished recorded video.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the echo control 346, and in response to the input operation, the electronic device 100 may display a video browsing interface 380 as shown in FIG. 3N.

As shown in FIG. 3N , the video browsing interface 380 may include a recorded video 381 captured in an intelligent mirror movement mode, a video playback control 382 , a total video time 383 , and a menu 384 . The menu 384 may include a share button, a favorite button, an edit button, a delete button, and more buttons. The share button can be used to trigger the sharing of the recorded video 381 . The favorite button can be used to trigger to save the recorded video 381 to the picture favorite folder. The edit button can be used to trigger editing functions such as rotation, trimming, adding filters, and blurring of the recorded video 381. The delete button can be used to trigger deletion of the recorded video 381 . The More button can be used to trigger opening of more functions related to the recorded video 381.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the video playback control 382 , and in response to the input operation, the electronic device 100 may play the recorded video 381 .

As shown in FIG. 30 , after the electronic device 100 starts to play the recorded video 381 , a pause control 386 and a playback progress bar 385 can be displayed on the video preview interface 380 . The pause control 386 can be used to pause the playback of the recorded video 381 . When the recorded video 381 is played to the third second, the electronic device 100 may display the video screen 391 on the video preview interface 380 .

As shown in FIG. 3P , when the recorded video 381 is played to the eighth second, the electronic device 100 may display a video screen 392 on the video preview interface 380 . Wherein, the magnification ratio of the image content in the video frame 392 is smaller than the magnification ratio of the image content in the video frame 391 shown in FIG. 30 .

As shown in FIG. 3Q , when the recorded video 381 is played to the 12th second, the electronic device 100 may display a video screen 393 on the video preview interface 380 . The magnification ratio of the image content in the video frame 393 is smaller than the magnification ratio of the image content in the video frame 392 shown in FIG. 3P.

In a possible implementation manner, after selecting the mirror movement template, the electronic device 100 may receive the user's input at any time, and trigger the display of the above mirror movement template selection interface. In this way, it is convenient for the user to choose and replace other mirror movement templates.

Exemplarily, as shown in FIG. 4A , the electronic device 100 may display a recording interface 360 . Wherein, for the text description of the video recording interface 360, reference may be made to the above-mentioned embodiment shown in FIG. 3I, and details are not repeated here.

The electronic device 100 may receive an input operation (eg, single click) of the user acting on the moving template switching control 362, and in response to the input operation, the electronic device 100 may display the moving template selection interface 340 as shown in FIG. 4B . Wherein, for the text description of the lens movement template selection interface 340, reference may be made to the aforementioned embodiment shown in FIG. 3D , which will not be repeated here.

As shown in FIG. 4B, the selection box 342 has selected the "travel" motion template 341A. The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the "kinetic" motion template 341C, and in response to the input operation, the electronic device 100 may select the "kinetic" motion template 341C.

As shown in FIG. 4C, the selection box 342 has selected the "Motion" motion template 341C. The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the camera movement control 345, and in response to the input operation, the electronic device 100 may display the photographing interface 410 as shown in FIG. 4D, and use the selected "" The corresponding cropping frame information (including the size change rule of the cropping frame and/or the motion track of the cropping frame in the original image) corresponding to the "Dynamic" lens motion template 341C is used to crop the video picture obtained by the camera when the video is captured.

As shown in FIG. 4D , the shooting interface 410 may include a preview image 441 obtained by the camera, an echo control 346 , a video shooting control 361 , a mirror movement template switching control 362 , an intelligent mirror movement prompt box 343 , and a mirror movement template preview window 431 . Wherein, the motion template preview window 431 displays the video with the cropping effect corresponding to the selected motion template (for example, the motion template corresponding to the above-mentioned "Motion" motion option 341C has been selected). The camera-moving template preview window 431 includes a closing control 432, and the closing control 432 can be used to trigger the electronic device 100 to close the camera-moving template preview window 431. The motion template switching control 362 may display the name of the motion template selected by the electronic device 100 (for example, "Motion"). The moving template switching control 362 can be used to trigger the electronic device 100 to switch the moving template.

In a possible implementation manner, the electronic device 100 may receive the user's input, and close/open the lens motion template preview window on the shooting interface. In this way, the electronic device 100 can open/close the preview window of the mirror movement template at any time according to the needs of the user, so that the preview window of the mirror movement template will not always block the shooting picture.

Exemplarily, as shown in FIG. 4D , the electronic device 100 may receive an input operation (eg, click) that the user acts on the closing control 432, and in response to the input operation, the electronic device 100 may close the lens motion template preview window 431, and A picture-in-picture on control 433 as shown in FIG. 4E is displayed.

As shown in FIG. 4E , the PIP enable control 433 can be displayed on the top right side of the display screen, or can be displayed in other positions without limitation. The electronic device 100 may receive an input operation (eg, a single click) of the user for the PIP enable control 433, and in response to the input operation, the electronic device 100 may display a mirror motion template preview window 431 as shown in FIG. 4F .

In a possible implementation manner, after the electronic device 100 determines the lens movement template selected by the user, a viewfinder display window may be displayed on the shooting interface. Wherein, the framing display window includes the original image obtained by the camera and the cropping frame. In this way, the electronic device 100 can show the dynamic change of the cropping frame to the user, which is convenient for the user to perceive the position of the cropping frame in the original image.

Exemplarily, as shown in FIG. 5A , the electronic device 100 may display a lens motion template selection interface 340 . Wherein, for the text description of the lens movement template selection interface 340, reference may be made to the aforementioned embodiment shown in FIG. 3D , which will not be repeated here. The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the camera movement control 345 , and in response to the input operation, the electronic device 100 may display the photographing interface 510 as shown in FIG. 5B .

As shown in FIG. 5B , the shooting interface 510 includes the original image 511 obtained by the camera, an echo control 346 , a video shooting control 361 , a mirror movement template switching control 362 , an intelligent mirror movement prompt box 343 , and a viewfinder display window 521 . Wherein, the viewfinder display window 521 displays the original image 511a (the image content included in the original image 511a is the same, and the display scale is different), and the close control 522 is displayed. The close control 522 can be used to trigger the electronic device 100 to close the viewfinder display window 521 . Wherein, the viewfinder display window 521 may be at the upper right corner of the display screen or at the upper left corner of the display screen, which is not limited herein.

In a possible implementation manner, the electronic device 100 may receive and move the display position of the viewfinder display window 521 in response to a user's input operation (eg, press and drag) on the viewfinder display window 521 .

The electronic device 100 may receive an input operation (such as a single click) that the user acts on the video capture control 361, and in response to the input operation, the electronic device 100 may start to change the rule and/or crop the size of the cropping frame corresponding to the selected mirror motion template. The motion trajectory of the frame in the original image is cropped for each frame of the video stream obtained by the camera. The electronic device 100 can adjust the cropped image of each frame to a specified size and sequentially display it in the video recording interface 360 in frame order, and display the position of the cropping frame on the original image in the viewfinder display window 521 . The electronic device 100 may save the cropped video stream.

As shown in FIG. 5C , after the electronic device 100 starts recording, the electronic device 100 may display a recording time frame 366 and a recording end control 365 on the recording interface 360 . The electronic device 100 may display the original image captured by the camera in the viewfinder display window 521 , and display the position of the cropping frame 523 in the original image in the viewfinder display window 521 . The electronic device 100 can cut out the image in the cropping frame 523 on the original image, adjust it to a specified size, and obtain a cropped image. The electronic device 100 may display the cropped image on the video recording interface 360 as a video recording image. For example, when the electronic device 100 receives the user's input operation on the video capture control 361 in the third second, the electronic device 100 can obtain the original image 512a through the camera, and determine the size of the cropping frame 523 in the original image 512a. Location. The electronic device 100 may cut out the image in the cropping frame 523 on the original image 512 a , and adjust the cut-out image to a specified size to obtain the cropped image 512 . The electronic device 100 may display the original image 512 a in the framing presentation window 521 , and display the position of the cropping frame 523 on the original image 512 a in the framing presentation window 521 . The electronic device 100 may display the image 512 on the recording interface 360 . Wherein, the magnification ratio of the image content in the cropped image 512 is greater than the magnification ratio of the image content in the original image 511 shown in FIG. 5B .

In a possible implementation manner, there may be multiple cameras on the electronic device 100, and the electronic device 100 may display the above-mentioned image captured by one camera (such as a wide-angle camera) among the multiple cameras in the viewfinder display window 521, and display the An image captured by another camera (eg, a depth-of-field camera) among the plurality of cameras by adjusting the zoom ratio is displayed in the shooting interface 510 as a preview image or a video image.

In a possible implementation manner, the electronic device 100 may adjust the original image captured by the camera to a specified magnification and display it in the viewfinder display window 521 . The electronic device 100 may perform cropping through a cropping frame based on the original image adjusted to a specified magnification to obtain a cropped image, and display the cropped image on the shooting interface 510 as a preview image or a video image.

Wherein, when the electronic device 100 shoots a video, what the electronic device 100 acquires through the camera is an image stream. The size of each frame of the original image in the image stream is fixed. The electronic device 100 may determine the position of the cropping frame in each frame of the original image in the image stream according to the size change rule of the cropping frame corresponding to the selected mirror motion template and/or the motion trajectory of the cropping frame in the original image. Then, the electronic device 100 may crop each frame of the original image in the image stream by using the cropping frame to obtain the cropped image stream within the cropping frame. The electronic device 100 can adjust the cropped image stream in the cropping frame to a specified size, and display it on the video recording interface 360 in frame order.

For example, within the first 8 s after the electronic device 100 receives the input operation of the user acting on the video capture control 361, the size of the cropping frame can be gradually reduced from the size of the original image to the preset minimum size in proportion. From the 8th second to the 16th second after receiving the user's input operation acting on the video shooting control 361, the size of the cropping frame is gradually increased proportionally to the size of the original image.

As shown in FIG. 5D , when the electronic device 100 receives the user's input operation on the video capture control 361 at the 8th second, the cropping frame 523 is reduced to a preset minimum size. The electronic device 100 may acquire the original image 513a through the camera, and determine the position of the cropping frame 523 in the original image 513a. The electronic device 100 may cut out the image in the cropping frame 523 on the original image 513a, and adjust the cutout image to a specified size to obtain the cropped image 513. The electronic device 100 may display the original image 513 a in the framing presentation window 521 , and display the position of the cropping frame 523 on the original image 513 a in the framing presentation window 521 . The electronic device 100 may display the cropped image 513 on the recording interface 360 . The magnification ratio of the image content in the cropped image 513 is greater than the magnification ratio of the image content in the cropped image 512 shown in FIG. 5C .

As shown in FIG. 5E , 12 seconds after the electronic device 100 receives the user's input operation on the video capture control 361 , the size of the cropping frame 523 is larger than the preset minimum size and smaller than the size of the frame image. The electronic device 100 may acquire the frame image of the 12th second through the camera, and determine the position of the cropping frame in the original image 514a. The electronic device 100 may cut out the image in the cropping frame 523 on the original image 514a, and adjust the cutout image to a specified size to obtain the cropped image 514. The electronic device 100 may display the original image 514a in the framing presentation window 521, and display the position of the cropping frame 523 on the original image 514a in the framing presentation window 521. The electronic device 100 may display the cropped image 514 on the recording interface 360 . Wherein, the magnification ratio of the image content in the cropped image 514 is smaller than the magnification ratio of the image content in the cropped image 513 shown in FIG. 5D .

The electronic device 100 can receive an input operation (such as a single click) that the user acts on the recording end control 365, and in response to the input operation, the electronic device 100 can end the recording, and save the clipping of the video stream through the clipping frame information corresponding to the above-mentioned mirror motion template. video recording later.

In some application scenarios, the electronic device 100 may select a mirror movement mode to take a photo. After the electronic device 100 is switched to the mirror movement mode, the electronic device 100 may receive the mirror movement template selected by the user. Different mirror motion templates may correspond to different cropping frame information (including the size change rule of the cropping frame and the motion track of the cropping frame on the original image obtained by the camera). The electronic device 100 can use the cropping frame information corresponding to the mirror movement template to crop each frame of image collected by the camera, and use the cropped image as a preview image to display on the shooting interface. When the electronic device 100 receives the user's input (eg, single click) acting on the photographing control, the electronic device 100 may save the preview screen displayed at this time as a picture. In this way, the electronic device 100 can realize automatic zooming through the lens movement template selected by the user, which is convenient for the user to select the subject or the background of the shooting scene.

Exemplarily, as shown in FIG. 6A , the electronic device 100 may receive an input operation (eg, a single click) for the user to select the mirror movement mode control 327H, and in response to the input operation, the electronic device 100 may switch from the “normal photographing mode” to “ Smart Mirror Mode". In the smart motion mode, the electronic device 100 can crop the frame image acquired by the camera according to the size change rule of the cropping frame corresponding to the motion template selected by the user and/or the motion trajectory of the cropping frame in the original image, and then crop the frame image obtained by the camera. The resulting image is displayed on the shooting interface as a preview screen.

As shown in FIG. 6B , the electronic device 100 may display a motion template selection interface 610 in response to an input operation (eg, a single click) of the user selecting the motion motion mode control 327H. The motion selection interface 610 may include one or more motion template options (eg, "travel" motion option 611A, "comfortable" motion option 611B, "kinetic" motion option 611C, etc.), selection box 612, Determine the control 613, the mirror movement template display area 614, and the mirror movement track display area 616. The mirror movement template display area 614 may display a mirror movement template play control 615 . The camera movement track display area 616 may display the center point 617 of the crop frame corresponding to the "travel" movement option 611A in the original image, and the size change. For example, the center point of the cropping frame corresponding to the "travel" motion option 611A may be at the same location as the central point of the original image, and the size of the cropping frame corresponding to the "travel" motion option 611A may first decrease in proportion to the size of the original image over time. to the preset minimum size, and then scale up from the preset minimum size to the size of the original image. Optionally, the template selection interface 610 may also display one or more shooting mode controls (for example, a "night scene mode" control 327A, a "portrait mode" control 327B, a "large aperture" control 327C, and a "mirror movement" mode control. 327H, "Normal Photo Mode" control 327D, "Video Mode" control 327E, "Pro Mode" control 327F, etc.). Wherein, for the text description of one or more shooting mode controls, reference may be made to the embodiment shown in FIG. 3B , which will not be repeated here.

The electronic device 100 may receive an input operation (eg, click) that the user acts on the camera movement template playback control 615, and in response to the input operation, as shown in FIG. 6C, the electronic device 100 may display the template in the camera movement track display area 616. Picture, crop box 618, and center point 617 of crop box 618. The electronic device 100 may crop the template picture based on the cropping frame 618 to obtain a cropped picture. The electronic device 100 can adjust the size of the cropped image to the display size of the moving-motion template display area 614 and display it in the display area 614 of the moving-motion template.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the determination control 613, and in response to the input operation, the electronic device 100 may display the photographing interface 620 as shown in FIG. 6D, and use the selected "travel" The size change rule of the cropping frame corresponding to the mirror movement template 611A and/or the motion trajectory of the cropping frame in the original image is used to crop the original preview image obtained by the camera.

As shown in FIG. 6D , the shooting interface 620 may include a mirror movement template switching control 621, a viewfinder display window 622, a cropped image 631, one or more shooting mode controls (for example, the “night scene mode” control 327A, the “portrait mode” Control 327B, "Large Aperture" control 327C, "Motion Mirror" mode control 327H, "Normal Photo Mode" control 327D, "Video Mode" control 327E, "Pro Mode" control 327F, etc.), Echo Control 321, Shooting Control 322. Camera conversion control 323. For the text description of one or more shooting mode controls, echo control 321 , shooting control 322 and camera conversion control 323 , reference may be made to the embodiment shown in FIG. 3B , which will not be repeated here. The mirror movement template switching control 621 can be used to trigger the electronic device 100 to display the above-mentioned mirror movement template selection interface 610 and switch the mirror movement template selected by the user based on the user's input. The framing presentation window 622 may display the original image 631a. The close control 623 can be used to trigger the electronic device 100 to close the viewfinder display window 623 . As shown in FIG. 6D , the size of the current cropping frame is the same as the size of the original image 631 . Therefore, the image content in this cropped image 631 is the same as the image content in the original image 631a.

For example, within the first 8 s after the electronic device 100 receives the input operation performed by the user on the determination control 613 shown in FIG. 6C , the size of the cropping frame can be gradually reduced proportionally from the size of the original image stream to the preset minimum size. , when the electronic device 100 receives the input operation of the user acting on the determination control 613 from the 8th second to the 16th second, the size of the cropping frame gradually increases proportionally to the size of the original image.

As shown in FIG. 6E , when the electronic device 100 receives the user's input operation on the determination control 613 in the third second, the electronic device 100 can obtain the original image 632a through the camera, and determine that the cropping frame 624 is in the original image location in 632a. The electronic device 100 may cut out the image in the cropping frame 624 on the original image 632a, and adjust the cutout image to a specified size to obtain the cropped image 632. The electronic device 100 may display the original image 632a in the framing presentation window 622, and display in the framing presentation window 622 the position of the cropping frame 624 on the original image 632a. The electronic device 100 may display the cropped image 632 on the photographing interface 620 . The magnification of the image content in the cropped image 632 is greater than the magnification of the image content in the cropped image 631 shown in FIG. 6D .

As shown in FIG. 6F , when the electronic device 100 receives the user's input operation on the determination control 613 at the 8th second, the electronic device 100 can obtain the original image 633a through the camera, and determine that the cropping frame is in the original image 633a in the location. The electronic device 100 can cut out the image in the cropping frame 624 on the original image 633a, and adjust the cutout image to a specified size to obtain the cropped image 633. The electronic device 100 may display the original image 633 a in the framing presentation window 622 and display the position of the cropping frame 624 on the original image 633 a in the framing presentation window 622 . The electronic device 100 may display the cropped image 633 on the photographing interface 620 . The magnification ratio of the image content in the cropped image 633 is greater than the magnification ratio of the image content in the cropped image 632 shown in FIG. 6E.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the photographing control 322, and in response to the input operation, the electronic device 100 may save the cropped image 633 displayed on the photographing interface 620 as a picture, and store the cropped image 633 is displayed on the echo control 321.

In a possible implementation manner, the electronic device 100 may display snapshot prompt information on the photography interface 620 during the process of displaying the above-mentioned photographing interface 620 , and the snapshot prompt information may be used to prompt the user to trigger the preview displayed on the photographing interface 620 The screen is saved as a picture.

In a possible implementation manner, when the electronic device 100 receives an input operation (such as a single click) that the user acts on the shooting control 322 on the shooting interface 620, in response to the input operation, the electronic device 100 may receive the input operation. The multi-frame cropped images acting on the shooting control 322 for a period of time before and after are saved as video images of the video file.

In a possible implementation manner, the electronic device 100 may, based on the lens motion template selected by the user, determine a cropped image that satisfies a certain condition from the cropped images cropped by the electronic device 100, and crop the frame. Images are saved as pictures.

In a possible implementation manner, after the electronic device 100 receives the user's input (eg, click) acting on the determination control 613 described in FIG. 6C , the electronic device 100 can automatically crop the multiple frames within a preset time. The image is saved as the video footage of the video file.

As shown in FIG. 6G , when the electronic device 100 receives the user's input operation on the determination control 613 in the 12th second, the electronic device 100 can obtain the original image 634a through the camera, and determine that the cropping frame is in the original image 634a in the location. The electronic device 100 may cut out the image in the cropping frame 624 on the original image 634a, and adjust the cutout image to a specified size to obtain the cropped image 634. The electronic device 100 may display the original image 634a in the framing presentation window 622 and display the position of the cropping frame 624 on the original image 634a in the framing presentation window 622 . The electronic device 100 may display the cropped image 634 on the photographing interface 620 . Wherein, the magnification ratio of the image content in the cropped image 634 is smaller than the magnification ratio of the image content in the cropped image 633 shown in FIG. 6F . Thumbnails of the above cropped images 633 may be displayed on the echo control 321 .

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the echo control 346, and in response to the input operation, the electronic device 100 may display a picture browsing interface 640 as shown in FIG. 6H.

As shown in FIG. 6H , the picture browsing interface 640 may include a picture 641 and a menu 642 . The picture 641 is the same as the cropped image 633 shown in FIG. 6F described above. For the text description of the menu 642, reference may be made to the aforementioned embodiment shown in FIG. 3N , which will not be repeated here.

In a possible implementation manner, the electronic device 100 may receive an input operation (eg, long press) that the user acts on the shooting control 322 , and in response to the input operation, the electronic device 100 may convert the cropped image 633 and the cropped image 633 displayed on the shooting interface 620 and The above-mentioned viewfinder display window 622 is saved as a picture together.

The mirror motion templates involved in the embodiments of the present application are described below.

In some embodiments, the cropping frame may be rectangular, the center of the cropping frame may be at the same position as the center of the original image, and the size of the cropping frame is proportional to the size of the original image. The size of the cropping frame can be gradually proportionally reduced over time from the size of the original image to a preset minimum size, and then gradually increased proportionally from the preset minimum size to the size of the original image. The cropping frame can repeat the above changing process cyclically.

In a possible implementation manner, the size of the cropping frame may first gradually increase proportionally from the preset minimum size to the original image size over time, and then gradually decrease proportionally from the original image size to the preset size. Set minimum size. The cropping frame can repeat the above changing process cyclically.

Exemplarily, as shown in FIG. 7A , the original image 701 is a rectangle. The center of the original image 701 may be at the same position as the center of the crop frame 702 , ie, both at position 704 . The center of the preset minimum size area 703 is also the position 704 . The size of the crop box 702 may vary proportionally between the size of the original image 701 and the preset minimum size. The width of the crop frame 702 may be x, and the height may be y. The width of the original image 701 may be A, and the height may be B. The width of the preset minimum size area 703 may be a, and the height may be b. Wherein, the width x and height y of the cropping frame 702 may satisfy the following relationship: x/y=A/B=a/b and a≤x≤A (or b≤y≤B), the width x of the cropping frame 702 may be from The width value A is reduced to the width value a, and the height y can be reduced from the height value B to the height value b.

As shown in FIG. 7B, the size of the crop box 702 has been reduced to a preset minimum size (width a, height b).

As shown in FIG. 7C , when the size of the cropping frame 702 is reduced to a preset minimum size (width is a, height is b), the size of the cropping frame 702 can be gradually increased to the size of the original image 701 (width is A, height is for B).

The electronic device 100 may acquire multiple frames of original images by acquiring a camera, and the electronic device 100 may determine the cropping frame in each frame of the original image according to the cropping frame information (including the cropping frame size change rule and/or the motion trajectory of the cropping frame in the original image). in the location. The electronic device 100 can intercept the image within the cropping frame in each frame of the original image according to the position of the cropping frame in each frame of the original image, and adjust it to a specified size to obtain multiple frames of cropped images. The electronic device 100 may display the multiple frames of cropped images in sequence on the shooting interface or video recording interface in the above-mentioned embodiment. The electronic device 100 may display the cropped image on the shooting interface or the video recording interface every time a cropped image is cropped out, or after cropping out multiple frames of cropped images, the cropped images of the multiple frames are sequentially displayed in the frame order. Displayed on the shooting interface or video recording interface.

Exemplarily, as shown in FIG. 7D , the electronic device 100 obtains the original image 721 through the camera, and the electronic device 100 can determine, according to the cropping frame information, that the center point of the cropping frame 712 on the original image 721 is the same as the central point of the original image 721 . position, ie all at position 714. At this time, the size of the cropping frame 712 is proportionally reduced from the size of the original image 721 to be between the size of the original image 721 and the preset minimum size. The electronic device 100 can cut out the image in the cropping frame 712 on the original image 721 and adjust it to a specified size to obtain the cropped image 722 . The electronic device 100 may display the cropped image 722 on the above-mentioned shooting interface or video recording interface.

As shown in FIG. 7E , the electronic device 100 acquires the original image 731 after acquiring the above-mentioned original image 721 through the camera. The electronic device 100 may determine, according to the cropping frame information, that the center point of the cropping frame 712 on the original image 731 is at the same position as the central point of the original image 731 , that is, both are at the position 714 . The size of the cropping frame 712 is reduced proportionally from the size of the original image 731 to the preset minimum size, that is, the size of the cropping frame 712 coincides with the preset minimum size area 713 . The electronic device 100 can cut out the image in the cropping frame 712 of the original image 731 and adjust it to a specified size to obtain the cropped image 732 . The electronic device 100 may display the cropped image 732 on the above-mentioned shooting interface or video recording interface. The magnification of the scene in the cropped image 732 is greater than the magnification of the scene in the cropped image 722 shown in FIG. 7D.

As shown in FIG. 7F , the electronic device 100 acquires the original image 741 after acquiring the above-mentioned original image 731 through the camera. The electronic device 100 may determine, according to the cropping frame information, that the center point of the cropping frame 712 on the original image 741 is at the same position as the central point of the original image 741 , that is, both are at the position 714 . At this time, the size of the cropping frame 712 is proportionally enlarged from the preset minimum size to between the size of the original image 741 and the preset minimum size. The electronic device 100 can cut out the image in the cropping frame 712 on the original image 741 and adjust it to a specified size to obtain the cropped image 742 . The electronic device 100 may display the cropped image 742 on the above-mentioned shooting interface or video recording interface. The magnification of the scene in the cropped image 742 is smaller than the magnification of the scene in the cropped image 742 shown in FIG. 7D .

In some embodiments, the center of the cropping frame may not be in the same position as the center of the original image. The size of the crop box changes in proportion to the size of the original image. The size of the crop box can be gradually reduced proportionally from the maximum size to the preset minimum size over time, and then gradually increased proportionally from the preset minimum size to the maximum size. The cropping frame can repeat the above changing process cyclically. The electronic device 100 may determine the maximum size of the cropping frame according to the center point of the cropping frame and the size of the original image. The maximum size is proportional to the original image. When the cropping frame is at the maximum size, the coverage of the cropping frame does not exceed the original image, and any border of the cropping frame is in contact with the border of the original image.

In a possible implementation manner, the size of the cropping frame may gradually increase proportionally from the preset minimum size to the maximum size over time, and then gradually decrease proportionally from the maximum size to the preset minimum size. The cropping frame can repeat the above changing process cyclically.

Exemplarily, as shown in FIG. 8A , the original image 801 is a rectangle. The center of the original image 801 may be at a different position from the center of the cropping frame 802, the center of the original image 801 may be at position 806, the center of the cropping frame 802 may be at position 805, and the center of the preset minimum size area 803 is also at position 805, The center of maximum size area 804 is also at location 805 . Wherein, any frame (eg, the lower frame) of the maximum size area 804 is in contact with the frame of the original image. The width of the cropping frame 8002 is x, and the height may be y. The width of the original image 801 may be A, and the height may be B. The width of the preset minimum size area 803 may be a, and the height may be b. The maximum size area 804 may have a width of c and a height of d. Wherein, the width x and height y of the cropping frame 802 may satisfy the following relationship: x/y=A/B=a/b=c/d and a≤x≤c (or b≤y≤d), the width of the cropping frame 802 The width x can be reduced from the width value c to the width value a, and the height y can be reduced from the height value d to the height value b.

As shown in FIG. 8B, the size of the crop frame 802 has been reduced to a preset minimum size (width a, height b).

As shown in FIG. 8C , when the size of the cropping frame 802 is reduced to a preset minimum size (width is a, height is b), the size of the cropping frame 802 can be gradually increased to the size of the original image 801 (width is c, height is for d).

The electronic device 100 may acquire multiple frames of original images by acquiring a camera, and the electronic device 100 may determine the cropping frame in each frame of the original image according to the cropping frame information (including the cropping frame size change rule and/or the motion trajectory of the cropping frame in the original image). in the location. The electronic device 100 can intercept the image within the cropping frame in each frame of the original image according to the position of the cropping frame in each frame of the original image, and adjust it to a specified size to obtain multiple frames of cropped images. The electronic device 100 may display the multiple frames of cropped images in sequence on the shooting interface or video recording interface in the above-mentioned embodiment. The electronic device 100 may display a cropped image on a shooting interface or a video recording interface every time a frame of cropped image is cropped, or after cropping out multiple frames of cropped images, the multiple frames of cropped images are sequentially displayed in frame order. Displayed on the shooting interface or video recording interface.

Exemplarily, as shown in FIG. 8D , the electronic device 100 obtains the original image 821 through the camera, and the electronic device 100 may determine that the center point of the cropping frame 812 on the original image 821 is at the position 815 according to the cropping frame information. At this time, the size of the cropping frame 812 is proportionally reduced from the maximum size to between the maximum size and the preset minimum size. The electronic device 100 can cut out the image in the cropping frame 812 on the original image 821, and adjust it to a specified size to obtain the cropped image 822. The electronic device 100 may display the cropped image 822 on the above-mentioned shooting interface or video recording interface.

As shown in FIG. 8E , the electronic device 100 acquires the original image 831 after acquiring the above-mentioned original image 821 through the camera. The electronic device 100 may determine that the center point of the cropping frame 812 on the original image 831 is at the position 815 according to the cropping frame information. At this time, the size of the cropping frame 812 is proportionally reduced from the maximum size to the preset minimum size, that is, the size of the cropping frame 812 coincides with the preset minimum size area 813 . The electronic device 100 can cut out the image in the cropping frame 812 on the original image 831 and adjust it to a specified size to obtain the cropped image 832 . The electronic device 100 may display the cropped image 832 on the above-mentioned shooting interface or video recording interface. The magnification of the scene in the cropped image 832 is greater than the magnification of the scene in the cropped image 822 shown in FIG. 8D .

As shown in FIG. 8F , the electronic device 100 acquires the original image 841 after acquiring the above-mentioned original image 831 through the camera. The electronic device 100 may determine that the center point of the cropping frame 712 on the original image 841 is at the position 815 according to the cropping frame information. At this time, the size of the cropping frame 812 is proportionally enlarged from the preset minimum size to between the maximum size and the preset minimum size. The electronic device 100 can cut out the image in the cropping frame 812 on the original image 841 and adjust it to a specified size to obtain the cropped image 842 . The electronic device 100 may display the cropped image 842 on the above-mentioned shooting interface or video recording interface. The magnification of the scene in the cropped image 842 is smaller than the magnification of the scene in the cropped image 842 shown in FIG. 8D .

In a possible implementation manner, the center of the cropping frame may not be at the same position as the center of the original image. The size of the crop box changes in proportion to the size of the original image. First, the center point of the cropping frame can be fixed, and the size of the cropping frame can gradually increase from a preset minimum size over time until the first side of the cropping frame touches the boundary of the original image. Then, the cropping box keeps the first side in contact with the boundary of the original image, the center point of the cropping box can be moved, and the size of the cropping box continues to increase until both the first and second sides of the cropping box touch the boundary of the original image. Then, keeping the first and second sides of the cropping frame in contact with the boundary of the original image, the center point of the cropping frame can be moved, and the size of the cropping frame continues to increase to the size of the original image. The cropping box can repeat the above changing process.

Exemplarily, as shown in FIG. 8G, at time t1, the center point of the cropping frame 803 is at the position point 805 on the original image 801, and the center point of the original image 801 is at the position point 806, which are not at the same position. The size of the cropping frame 803 is a preset minimum size. At time t2, the center point of the cropping frame 803 is at the position point 805, and the size of the cropping frame 803 increases proportionally relative to the time t1 unchanged. At time t3, the center point of the cropping frame 803 is at the position point 805, and the size of the cropping frame 803 is proportionally increased relative to the time t1 until the lower border of the cropping frame 803 contacts the lower border of the original image 801. During the time period from t3 to t4, the lower border of the cropping frame 803 is always in contact with the lower border of the original image 801, and the size of the cropping frame 803 increases proportionally. At time t4, the center point of the cropping frame 803 is displaced relative to the time t1, and the size of the cropping frame 803 is proportionally increased until the lower border of the cropping frame 803 is in contact with the lower border of the original image 801, and the left border of the cropping frame 803 is in contact with the original image. The left border of 801 touches. During the time period from t4 to t5, the lower border of the cropping frame 803 is always in contact with the lower border of the original image 801 and the left border of the cropping frame 803 is always in contact with the left border of the original image 801, and the size of the cropping frame 803 increases proportionally. At time t5, the size of the crop frame 803 is increased to the size of the original image.

In some embodiments, the size of the cropping frame may be a fixed size, and the center point of the cropping frame may move according to a preset trajectory on the original image. The original image may be a rectangle, the cropping frame may also be a rectangle, and the ratio of the width to the height of the cropping frame is equal to the ratio of the width to the height of the original image.

Exemplarily, as shown in FIG. 9A , the center point 913 of the cropping frame 912 may move left and right on the original image 911 with a straight trajectory 921 between the position 922 and the position 923 . When the center point 913 of the cropping frame 912 is at the position 922 , the left border of the cropping frame 912 may coincide with the left border of the original image 911 . When the center point 913 of the cropping frame 912 is at the position 923 , the right border of the cropping frame 912 may coincide with the right border of the original image 911 .

As shown in FIG. 9B , the center point 913 of the crop frame 912 may move up and down on the original image 911 in a straight line trajectory 931 between the position 932 and the position 933 . When the center point 913 of the cropping frame 912 may be at the position 932 , the upper border of the cropping frame 912 may coincide with the upper border of the original image 911 . When the center point 913 of the cropping frame 912 is at the position 933 , the lower border of the cropping frame 912 may coincide with the lower border of the original image 911 .

As shown in FIG. 9C , the center point 913 of the crop box 912 may move from the upper left corner to the lower right corner on the original image 911 in a straight line trajectory 941 between the position 942 and the position 943 . When the center point 913 of the cropping frame 912 may be at position 942 , the upper border of the cropping frame 912 may coincide with the upper border of the original image 911 , and the left border of the cropping frame 912 may coincide with the left border of the original image 911 . When the center point 913 of the cropping frame 912 is at position 943 , the lower border of the cropping frame 912 may coincide with the lower border of the original image 911 , and the right border of the cropping frame 912 may coincide with the right border of the original image 911 .

As shown in FIG. 9D , the center point 913 of the crop box 912 may move from the lower left corner to the upper right corner on the original image 911 in a straight line trajectory 951 between

positions

952 and 953 . When the center point 913 of the cropping frame 912 may be at position 952 , the lower border of the cropping frame 912 may coincide with the lower border of the original image 911 , and the left border of the cropping frame 912 may coincide with the left border of the original image 911 . When the center point 913 of the cropping frame 912 is at the position 953 , the upper border of the cropping frame 912 may coincide with the upper border of the original image 911 , and the right border of the cropping frame 912 may coincide with the right border of the original image 911 .

As shown in FIG. 9E , the motion trajectory 961 of the center point 913 of the cropping frame 912 may be a heart shape, wherein the cropping frame 912 may move clockwise or counterclockwise on the motion trajectory 961 . When the cropping frame 912 moves clockwise or counterclockwise on the motion track 961 , it always does not exceed the size range of the original image 911 .

As shown in FIG. 9F , the motion trajectory 971 of the center point 913 of the cropping frame 912 may be a circular figure, wherein the cropping frame 912 may move clockwise or counterclockwise on the motion trajectory 971 . When the cropping frame 912 moves clockwise or counterclockwise on the motion track 971 , it always does not exceed the size range of the original image 911 .

It should be noted that the motion trajectory of the center point 912 of the cropping frame 912 in the original image 911 is not limited to the above-mentioned graphs, and may also have other graphs, for example, a horizontal “8” curve graph, a Bezier curve graph, a circular arc curve graphics, etc.

In a possible implementation manner, when the electronic device 100 displays a viewfinder display window on the shooting interface or video recording interface in the camera or video mode, the electronic device 100 may display the cropping frame in the viewfinder display window above. Motion trajectory on the original image. When the cropping frame moves along the motion track once, the electronic device 100 may display special effects (such as fireworks special effects, kaleidoscope special effects, heart-shaped beating special effects, etc.) in the viewfinder display window.

Exemplarily, as shown in FIG. 9G , the electronic device 100 may display a photographing interface 980 . The shooting interface 980 may include a lens motion template switching control 981, a viewfinder display window 982, a cropped image 991, and one or more shooting mode controls. For the text description of one or more shooting mode controls, reference may be made to the aforementioned embodiment shown in FIG. 3B , and details are not repeated here. The moving template switching control 981 displays the name of the moving template currently used by the electronic device 100 (for example, "circular moving"). The electronic device 100 may display the original image 991 a obtained at this time, the closing control 983 , the cropping frame 984 , and the motion track of the cropping frame 984 in the viewfinder display window 982 . The electronic device 100 obtains the cropped image 991 by cropping based on the original image 991a and the position of the cropping frame 984 in the original image 991a.

As shown in FIG. 9H , the cropping frame 984 moves once according to a preset motion trajectory (eg, a circular trajectory). The electronic device 100 may acquire the original image 992 a through the camera, and display the original image 992 a in the viewfinder display window 982 . The electronic device 100 may cut out the image in the cropping frame 984 on the original image 992a, and adjust the cutout image to a specified size to obtain the cropped image 992. The electronic device 100 may display the cropped image 992 on the shooting interface 980, and display a special effect picture 986 (eg, a fireworks special effect picture, etc.).

In a possible implementation manner, when the cropping frame moves according to a preset motion trajectory (for example, a circular trajectory) once. The electronic device 100 can display a special effect picture on the shooting interface, and while displaying the characteristic picture, adjust the size of the cropping frame to gradually increase to the size of the original image over time, and display the cropped image in the cropping frame on the shooting interface. or on the recording interface.

Wherein, when the electronic device 100 displays the above-mentioned viewfinder display window on the video recording interface in the mirror-moving video recording mode, the electronic device 100 can receive the user's input of ending the video recording, and in response to the input, the electronic device 100 can record the video recording interface within the video recording time. The cropped image shown above is saved as a video image in the video file together with the above-mentioned special effect image. Alternatively, the electronic device 100 may also save the cropped image displayed on the recording interface during the recording time together with the above-mentioned viewfinder display window as a video picture in the video file. Alternatively, the electronic device 100 may also save the cropped image displayed on the recording interface during the recording time, the above-mentioned special effect picture, and the above-mentioned viewfinder display window together as a video picture in the video file.

Exemplarily, as shown in FIG. 9I , after the electronic device 100 saves the cropped image, the above-mentioned special effect picture and the above-mentioned framing display window displayed on the video recording interface during the video recording time together as the video picture in the video 995, the video preview interface can be displayed. 993. The video preview interface 993 may include a video 995 shot in a circular motion mode, and a video playback control 994a, wherein the video screen of the video 995 includes a cropped image and a viewfinder display window after cropping based on the original image obtained by the camera 994b. The framing display window 994b includes a cropping frame 994c and an original image 995a obtained by the camera.

The electronic device 100 receives an input operation (eg, a single click) of the user acting on the video playback control 994a, and in response to the input operation, the electronic device 100 can play the video 995.

As shown in FIG. 9J , when the video 995 is played to the 3rd second, the electronic device 100 may display a video screen 996 on the video preview interface 993 . The video frame 996 includes a viewfinder display window 994b, the position of the cropping frame 994c on the original image 996a obtained by the camera, and a cropped image cropped from the original image 996a based on the cropping frame 994c. The framing display window 994b can also display the motion track of the cropping frame 994c.

As shown in FIG. 9K , when the video 995 is played to the 12th second, the electronic device 100 may display a video screen 997 on the video preview interface 993 . The video screen 997 includes a viewfinder display window 994b, the position of the cropping frame 994c on the original image 997a obtained by the camera, and the cropped image and special effect screen (such as a fireworks special effect screen) cropped from the original image 997a based on the cropping frame 994c and many more).

In some embodiments, the size of the cropping frame changes in proportion to the size of the original image, and at the same time, the center point of the cropping frame can move according to a preset track on the original image. Wherein, the size of the cropping frame may gradually decrease proportionally from the size of the original image to the preset minimum size over time, and then gradually increase proportionally from the preset minimum size to the size of the original image. The original image may be a rectangle, the cropping frame may also be a rectangle, and the ratio of the width to the height of the cropping frame is equal to the ratio of the width to the height of the original image.

In some application scenarios, the electronic device 100 may receive a user-defined setting of the center point of the cropping frame in the original image when displaying the original image obtained by the camera as a preview image in the shooting interface. The electronic device 100 may determine the maximum size and the minimum size of the cropping frame based on the size of the original image and the center point of the cropping frame. The electronic device 100 may determine the position and size of the cropping frame in each frame of the original image based on the maximum size and the minimum size of the cropping frame, and the speed at which the size of the cropping frame changes proportionally between the maximum size and the minimum size. The electronic device 100 may, based on the position and size of the cropping frame in each frame of the original image, cut out the image within the cropping frame on each frame of the original image, and display the cutout image on the shooting interface as a preview image. In this way, it is convenient for the user to customize the selection of the center point of the cropping frame, and the electronic device 100 can align the center point of the shooting image with the shooting subject, so that the shooting subject is clearer.

Exemplarily, as shown in FIG. 10A , the electronic device 100 may display a mirror motion template selection interface 610 . Wherein, for the text description of the lens movement selection interface 610, reference may be made to the aforementioned embodiment shown in FIG. 6B , which will not be repeated here. Wherein, in the selection box 612 of the camera movement selection interface 610, the "travel" camera movement template 611A has been selected.

As shown in FIG. 10B , the electronic device 100 may receive user input (eg, a click) for the "custom" motion template option 611D, and in response to the input, the electronic device 100 may move the selection box 612 to select the "custom" motion template Template option 611D. Wherein, when the “custom” motion template option 611D is selected in the selection box 612 , the electronic device 100 can close and display the motion template display area 614 and the motion track display area 616 , and display a prompt box 1011 . The prompt box 1011 can be used to display the function description of the "customized" mirror movement template, and the function description can be text (for example, "you can customize the center point of mirror movement", etc.), pictures, videos, and so on.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the determination control 613 , and in response to the input operation, the electronic device 100 may display a photographing interface 1020 as shown in FIG. 10C .

As shown in FIG. 10C , the shooting interface 1020 may include a viewfinder display window 1021, an original image 1031, prompt information 1025, a determination control 1026, a lens motion template switching control 1027, a position marker 1028, and one or more shooting mode controls (eg, "Night Scene Mode" Control 327A, "Portrait Mode" Control 327B, "Large Aperture" Control 327C, "Mirror" Mode Control 327H, "Normal Photo Mode" Control 327D, "Video Mode" Control 327E, "Pro Mode" Control 327F etc.), echo control 321, capture control 322, camera conversion control 323. For the text description of one or more shooting mode controls, echo control 321 , shooting control 322 and camera conversion control 323 , reference may be made to the embodiment shown in FIG. 3B , which will not be repeated here. The mirror movement template switching control 1027 can be used to trigger the electronic device 100 to switch the mirror movement template selected by the user, and the mirror movement template switching control 1027 displays the name of the mirror movement template currently selected by the electronic device 100 (for example, "custom mirror movement" ). The viewfinder display window 1021 may display a close control 1022, an original image 1031a (same image content as the original image 1031, but with a different display scale), a cropping frame 1023, and the cropping frame 1023 at the center point 1024 of the original image 1031a. The close control 1022 can be used to trigger the electronic device 100 to close the viewfinder display window 1021 . The prompt information 1025 can be used to prompt the user to adjust the position of the center point 1023 of the cropping frame 1024 on the image on the original image 1031 . The position marker 1028 may be used to indicate the current position of the center point 1023 of the cropping frame 1024 on the original image 1031a.

As shown in FIG. 10D , the electronic device 100 may receive an input (eg, dragging) of the position marker 1028 from the user, and in response to the input operation, the electronic device 100 may adjust the display position of the position marker 1028 on the original image 1031a. Wherein, after the position of the position marker 1028 on the original image 1031 is moved, the electronic device 100 can adjust the position of the center point 1023 of the cropping frame 1024 in the viewfinder display window 1021 on the original image 1031a.

As shown in FIG. 10E , the electronic device 100 may receive an input (eg, a single click) of the determination control 1026 from the user, and in response to the input, the electronic device 100 may capture the image within the cropping frame on the original image, and will capture the The image is resized to the specified size to get a cropped image. The electronic device 100 may display the cropped image as a preview image on the shooting interface. The electronic device 100 may also not display the above-mentioned prompt information 1025 and the above-mentioned determination control 1026 in response to the user's input to the determination control 1026 .

As shown in FIG. 10F , the position of the center point 1024 of the cropping frame 1023 may be fixed on the original image, and the size of the cropping frame 1023 changes in proportion to the size of the original image. The size of the cropping frame 1023 may gradually increase proportionally from the preset minimum size to the maximum size over time, and then gradually increase proportionally from the maximum size to the preset minimum size. For the process of changing the size of the cropping frame 1023 between the preset minimum size and the maximum size, reference may be made to the embodiments shown in FIG. 8A to FIG. 8F , which will not be repeated here. At this time, the size of the cropping frame 1023 is the preset minimum size. The electronic device 100 may display the original image 1032 a obtained by the camera in the viewfinder display window 1021 . The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1032a, and adjust the cutout image to a specified size to obtain the cropped image 1032. The electronic device 100 may display the cropped image 1032 as a preview image on the shooting interface 1020 . The electronic device 100 may also display controls 1029 on the photographing interface 1020 . The control 1029 can be used to trigger the electronic device 100 to reselect the center point of the cropping frame on the original image based on the user's input.

As shown in FIG. 10G , at this time, the size of the cropping frame 1023 is proportionally increased from the preset minimum size to between the maximum size and the preset minimum size. The electronic device 100 may display the original image 1033a obtained by the camera in the viewfinder display window 1021 at this time. The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1033a, and adjust the cutout image to a specified size to obtain the cropped image 1033. The electronic device 100 can display the cropped image 1033 as a preview image on the shooting interface 1020. . The magnification of the image content in the cropped image 1033 is smaller than the magnification of the image content in the cropped image 1032 shown in FIG. 10F .

As shown in FIG. 10H , at this time, the size of the crop frame 1023 is increased from the maximum size to the maximum size proportionally between the maximum size and the preset minimum size. The electronic device 100 may display the original image 1034a obtained by the camera in the viewfinder display window 1021 at this time. The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1034a, and adjust the cutout image to a specified size to obtain the cropped image 1034. The electronic device 100 can display the cropped image 1034 as a preview image on the shooting interface 1020. . The magnification of the image content in the cropped image 1034 is smaller than the magnification of the image content in the cropped image 1033 shown in FIG. 10G.

The electronic device 100 may receive an input (eg, click) that the user acts on the control 1029, and in response to the input, as shown in FIG. Position marker 1028, the above-mentioned determination control 1026.

As shown in FIG. 10I , the electronic device 100 may receive user input (eg, dragging) for the location marker 1028 , and in response to the input, the electronic device 100 may adjust the display position of the location marker 1028 in the original image 1035 . The electronic device 100 can determine the center point 1024 of the cropping frame 1023 in the original image 1035a in the viewfinder display window 1021 based on the display position of the position marker 1028 in the original image 1035 (same content as the image in the original image 1035, but with a different display scale). ) on the position.

As shown in FIG. 10J , the electronic device 100 may receive an input (eg, a click) from the user for the determination control 1026 , and in response to the input, the electronic device 100 may cut out the image within the cropping frame on the original image, and will cut out the The image is resized to the specified size to get a cropped image. The electronic device 100 may display the cropped image on the shooting interface 1020 as a preview image.

As shown in FIG. 10K , the position of the center point 1024 of the cropping frame 1023 in the original image is fixed, and the size of the cropping frame 1023 changes in proportion to the size of the original image. The electronic device 100 can display the original image 1036a obtained by the camera in the viewfinder display window 1021 . The electronic device 100 may display the cropped image 1036 cut out by the cropping frame 1023 as a preview screen on the photographing interface 1020 .

In some application scenarios, the electronic device 100 may receive a user-defined setting of the center point of the cropping frame in the original image when displaying the original image obtained by the camera as a preview image in the shooting interface. The electronic device 100 may determine the maximum size and the preset minimum size of the cropping frame based on the size of the original image and the center point of the cropping frame. The electronic device 100 may receive the user's input, and adjust the size of the cropping frame at any time during the preview or video recording process. The electronic device 100 may, based on the position and size of the cropping frame in each frame of the original image, cut out the image within the cropping frame on each frame of the original image, and display the cutout image on the shooting interface as a preview image. In this way, it is convenient for the user to customize the selection of the center point of the cropping frame and the size of the cropping frame, and the electronic device 100 can align the central point of the photographed image with the subject, and zoom in or out at any time to make the subject clearer.

Exemplarily, as shown in FIG. 11A , the electronic device 100 may display a photographing interface 1020 . Wherein, for the text description of the photographing interface 1020, reference may be made to the aforementioned embodiment shown in FIG. 10C , which will not be repeated here.

After receiving the user's input and adjusting the display position of the position marker 1028, the electronic device 100 may receive the user's input (for example, clicking) on the determination control 1026, and in response to the input, as shown in FIG. 11B , the electronic device 100 may The original image 1112a acquired by the camera at this time is displayed in the viewfinder display window 1021 . The electronic device 100 may cut out the image in the cropping frame 1023 on the original image 1112a, and adjust the cutout image to a specified size to obtain the cropped image 1112. The electronic device 100 may display the cropped image 1112 as a preview image on the shooting interface.

As shown in FIG. 11B , the electronic device 100 can also display a magnification magnification control 1121 and a magnification reduction control 1122 on the shooting interface 1020 . The magnification control 1121 can be used to reduce the size of the cropping frame 1023 , and the magnification reduction control 1122 can be used to increase the size of the cropping frame 1023 . Wherein, possibly, the above-mentioned magnification control 1121 and magnification reduction control 1122 may also be sliders. When the electronic device 100 receives a user sliding upwards on the sliders, the electronic device 100 can reduce the size of the crop frame 1023 . When the electronic device 100 receives the user sliding down the slider bar, the electronic device 100 may increase the size of the cropping frame 1023 .

The electronic device 100 may receive a user input (eg, single click or long press) acting on the zoom-out control 1122, and in response to the input, the electronic device 100 may increase the size of the crop box 1023 as shown in FIG. 11C. The electronic device 100 can display the original image 1113a obtained by the camera in the viewfinder display window 1021 at this time. The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1113a, adjust it to a specified size, and obtain the cropped image 1113.

As shown in FIG. 11C , the electronic device 100 may display the cropped image 1113 as a preview screen on the shooting interface 1020 . Wherein, the magnification of the image content in the cropped image 1113 is smaller than the magnification of the image content in the cropped image 1112 shown in FIG. 11B .

The electronic device 100 may continue to receive the user's operation (eg, single click or long press) on the zoom-out control 1122, and in response to the operation, the electronic device 100 may increase the size of the crop frame 1023 to the maximum size. For the description of the maximum size, reference may be made to the foregoing embodiments shown in FIGS. 8A to 8F , and details are not repeated here.

As shown in FIG. 11D , the electronic device 100 adjusts the crop frame 1023 to the maximum size based on the user's input operation on the zoom-out control 1122 . The electronic device 100 can display the original image 1114a obtained by the camera in the viewfinder display window 1021 at this time. The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1114a, adjust it to a specified size, and obtain the cropped image 1114. The electronic device 100 may display the cropped image 1114 as a preview image on the shooting interface 1020 . The magnification of the image content in the cropped image 1114 is smaller than the magnification of the image content in the cropped image 1113 shown in FIG. 11C .

The electronic device 100 may receive an operation (eg, single click or long press) of the user acting on the magnification increase control 1121 , and in response to the operation, as shown in FIG. 11E , the electronic device 100 may reduce the size of the cropping frame 1023 . The electronic device 100 may display the original image 1115a obtained by the camera in the viewfinder display window 1021 at this time. The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1115a, adjust it to a specified size, and obtain the cropped image 1115.

As shown in FIG. 11E , the electronic device 100 may display the cropped image 1115 as a preview image on the shooting interface 1020 . Wherein, the magnification of the image content in the cropped image 1115 is smaller than the magnification of the image content in the cropped image 1114 shown in FIG. 11D .

In some application scenarios, the electronic device 100 may receive a user-defined setting of the center point of the cropping frame in the original image when displaying the original image obtained by the camera as a preview image in the shooting interface. The size of the cropping frame can be fixed. During the preview or recording process, the cropping frame can move in a fixed direction on the original image, and the electronic device 100 can receive user input to adjust the moving direction and speed of the cropping frame on the original image at any time. The electronic device 100 may determine the position and size of the cropping frame in each frame of the original image according to the moving direction and the moving speed of the cropping frame. The electronic device 100 can cut out the image within the cropping frame on each frame of the original image based on the position and size of the cropping frame in each frame of the original image, and display the cutout image as a preview image on the shooting interface or the video recording interface. In this way, it is convenient for the user to adjust the acquisition position of the shooting image at any time during the preview or video recording process, so that the shooting image moves in the direction selected by the user, which improves the user experience.

Exemplarily, as shown in FIG. 12A , the electronic device 100 may display a photographing interface 1020 . Wherein, for the text description of the photographing interface 1020, reference may be made to the aforementioned embodiment shown in FIG. 10C , which will not be repeated here.

After receiving the user's input and adjusting the display position of the position marker 1028, the electronic device 100 may receive the user's input (eg, clicking) on the determination control 1026, and in response to the input, as shown in FIG. 12B, the electronic device 100 may The original image 1211a obtained by the camera at this time is displayed in the viewfinder display window 1021 . The electronic device 100 can cut out the image in the cropping frame 1023 on the original image 1211a, and adjust the cutout image to a specified size to obtain the cropped image 1211. The electronic device 100 may display the cropped image 1211 as a preview image on the shooting interface.

As shown in FIG. 12B , the electronic device 100 can display multiple directional movement controls for the crop frame 1023 on the photographing interface 1020 (including up-move controls 1221 , down-move controls 1222 , left-move controls 1223 , and right-move controls 1224 ). The move-up control 1221 can be used to trigger the electronic device 100 to move the position of the crop frame 1023 upward on the original image. The move down control 1222 can be used to trigger the electronic device 100 to move down the position of the cropping frame 1023 on the original image. The left move control 1223 can be used to trigger the electronic device 100 to move the position of the crop frame 1023 on the original image to the left. The right move control 1224 can be used to trigger the electronic device 100 to move the position of the crop frame 1023 on the original image to the right. In a possible implementation manner, the multiple direction movement controls may be cross-shaped sliding bars.

As shown in FIG. 12B and FIG. 12C , the cropping frame 1023 may be moved to the right from the position selected by the user on the original image at the beginning. As shown in FIG. 12B , the electronic device 100 can intercept the image in the cropping frame 1023 on the original image 1211 a and adjust it to a specified size to obtain the cropped image 1211 . As shown in FIG. 12C , the cropping frame 1023 moves to the right on the original image, and the electronic device 100 can use the cropping frame 1023 to cut out the cropped image 1212 from the original image 1212a, and display the cropped image 1212 as a preview image on the shooting interface 1020 superior. The image content in the cropped image 1212 is to the right of the image content in the cropped image 1211 shown in FIG. 12B .

The electronic device 100 may receive a user input (eg, a click) on the move up control 1221, and in response to the input, the electronic device 100 may move up the position of the crop box 1023 on the original image. The electronic device 100 can capture the image in the cropping frame 1023 on the original image, adjust it to a specified size, and obtain the cropped image 1213 .

As shown in FIG. 12D , the electronic device 100 may display the original image 1213 a obtained by the camera at this time in the viewfinder display window 1021 . The electronic device 100 may cut out the image in the cropping frame 1023 on the original image 1213a, and adjust the cutout image to a specified size to obtain the cropped image 1213. The electronic device 100 may display the cropped image 1213 on the shooting interface 1020 as a preview image. The image content in the cropped image 1213 is above the image content in the cropped image 1212 shown in FIG. 12C .

In a possible implementation manner, when the electronic device 100 does not receive the user's input for the above-mentioned directional movement control for a period of time (eg, 3 seconds), the electronic device 100 may exit from displaying the above-mentioned directional movement control. When the electronic device 100 receives the user's input on the photographing interface (eg, a double-click operation), the electronic device 100 may display the above-mentioned directional movement controls again.

In some embodiments, the electronic device 100 may receive a user's long-press operation on the above-mentioned directional movement control, and in response to the long-press operation, the electronic device 100 may increase the moving speed of the cropping frame on the original image.

Exemplarily, for example, the electronic device 100 may obtain the original image P1 at the first second, the original image P2 at the second second, and the original image P3 at the third second sequentially through the camera. The position of the cropping frame on the original image P2 is shifted to the right by 100 pixels compared to the position on the original image P1. The electronic device 100 receives the user's long-press operation on the right shift control from the second to the third second, and the electronic device 100 can increase the speed at which the cropping frame moves to the right on the original image. The position of the cropping frame on the original image P3 is shifted to the right by 200 pixels compared to the position on the original image P2. The above examples are only used to explain the present application and should not be construed as limitations.

In some embodiments, the electronic device 100 may receive the user-defined setting of the center point of the cropping frame in the original image when displaying the original image obtained by the camera as a preview image in the shooting interface. The size of the cropping frame can be fixed. The electronic device 100 may receive the user's input (for example, the above-mentioned operation of moving the control in the direction) during the process of taking a photo preview or recording, and adjust the position of the cropping frame on the original image at any time. The electronic device 100 can cut out the image within the cropping frame on each frame of the original image based on the position and size of the cropping frame in each frame of the original image, and display the cutout image as a preview image on the shooting interface or the video recording interface. In this way, it is convenient for the user to adjust the acquisition position of the shooting image at any time during the preview or video recording process, so that the shooting image moves in the direction selected by the user, which improves the user experience.

In some application scenarios, the electronic device 100 may have moving image content (eg, a flying bird) in the preview screen or the video screen. The electronic device 100 receives the user's input, identifies the moving image content (eg, a flying bird), and determines the position of the moving image content on each frame of the original image. The electronic device 100 can adjust the position of the cropping frame on each frame of the original image, so that the moving image content is always within the cropping frame. The electronic device 100 may, based on the position and size of the cropping frame in each frame of the original image, cut out the image within the cropping frame on each frame of the original image, and display the cutout image on the shooting interface as a preview image. In this way, when the electronic device 100 captures the moving image content, the shooting screen can always display the moving image content in an enlarged manner.

Exemplarily, as shown in FIG. 13A , the electronic device 100 may display a photographing interface 1310 . The shooting interface 1310 may include a viewfinder display window 1311, an original image 1321, one or more shooting mode controls (for example, the "Night Scene Mode" control 327A, the "Portrait Mode" control 327B, the "Large Aperture" control 327C, the "Motion "Mirror" mode control 327H, "Normal camera mode" control 327D, "Video mode" control 327E, "Pro mode" control 327F, etc.), echo control 321, shooting control 322, camera conversion control 323. For the text description of one or more shooting mode controls, echo control 321 , shooting control 322 and camera conversion control 323 , reference may be made to the embodiment shown in FIG. 3B , which will not be repeated here. The viewfinder display window 1311 may be displayed with a close control 1312 and an original image 1321a (the image content is the same as that of the original image 1321, but the display scale is different).

The electronic device 100 may receive a click operation (eg, single click, long press, double click, etc.) performed by the user on the original image 1321, and in response to the input, the electronic device 100 may perform a click operation in the original image 1321 according to the click operation. Action position, it is recognized that the image content 1316 selected by the user is a bird, and the image content 1316 is moving.

As shown in FIG. 13B , at this time, the electronic device 100 acquires the original image 1322 , wherein the position of the image content 1316 in the original image 1322 is different from the position in the original image 1321 . The electronic device 100 displays the original image 1322 as a preview image on the shooting interface 1310, and displays the original image 1322a in the viewfinder display window 1311 (same image content as the original image 1322, but with a different display scale). When recognizing that the image content 1316 is moving, the electronic device 100 may display a prompt message 1318 and a determination control 1319, and display a marker frame 1317 in the area where the image content 1316 is located. The position of the display position of the determination control 1319 on the shooting interface 1310 is not limited. For example, the determination control 1319 may be above the mirror movement mode control 327H, and the determination control 1319 may also be near the image 1316. The electronic device 100 can also display a cropping frame 1313 in the viewfinder display window 1311 , where the center point of the cropping frame 1313 is on the image content 1316 , and the prompt information 1318 can be used to prompt the user whether to track the image content 1316 . The center point of the mark frame 1317 is on the image content 1316 , and the mark frame 1317 is used to indicate the size of the cropping frame and the position of the cropping frame relative to the image content 1316 . In a possible implementation manner, the electronic device 100 may receive an input (eg, dragging) that the user acts on the marking frame 1317 to adjust the size of the cropping frame.

The electronic device 100 may receive an input (such as a single click) that the user acts on the determination control 1319, and in response to the input, as shown in FIG. 13C, the electronic device 100 may display the original image 1323a obtained by the camera at this time in the viewfinder display window 1311 middle. The electronic device 100 can cut out the image in the cropping frame 1313 on the original image 1323a, and adjust the cutout image to a specified size to obtain the cropped image 1323. The electronic device 100 may display the cropped image 1323 as a preview image on the shooting interface 1310 . The image content 1316 is in the center of the cropped image 1323 . The display size of the image content 1316 in the cropped image 1323 is larger than the display size of the image content 1316 in the original image 1322 shown in FIG. 13B .

In a possible implementation manner, the electronic device 100 may directly recognize the image content selected by the user according to the action position of the click operation in the original image 1321 after receiving the user's double-click operation on the original image 1321. 1316. After recognizing the image content 1316 selected by the user, the electronic device 100 may display the original image acquired by the camera in the viewfinder display window 1311 . The electronic device 100 may cut out an image including the image content 1316 on the original image, and adjust the cut out image to a specified size to obtain a cropped image. The electronic device 100 may display the cropped image on the photographing interface 1310 .

As shown in Figure 13D, the position of the image content 1316 in the original image has changed. The cropping frame 1313 moves following the position of the image content 1316 . The electronic device 100 may display the cropped image 1324 cut out from the original image 1324a as a preview image on the shooting interface 1310 at this time. Here, the image content 1316 is still in the center of the cropped image 1324.

In a possible implementation manner, as shown in FIG. 13E , the electronic device 100 may display the original image 1325 a obtained by the camera at this time in the viewfinder display window 1311 . The electronic device 100 may cut out the image in the cropping frame 1313 on the original image 1325a, and adjust the cutout image to a specified size to obtain the cropped image 1325. The electronic device 100 may display the cropped image 1325 as a preview image on the shooting interface 1310 . The electronic device 100 may also display the motion track 1331 of the image content 1316 in the above-mentioned viewfinder display window 1311 .

In some embodiments, the electronic device 100 may recognize the outline of a specified image content (eg, a scene or a task) in a preview or video image. The electronic device 100 may use the outline of the specified image content as the movement track of the center point of the cropping frame, and adjust the cropping frame to move on the outline of the specified image content at a preset speed. Among them, the size of the cropping frame can be fixed. The electronic device 100 may determine the position of the cropping frame on each frame of the original image based on the outline of the specified image content and the preset speed. The electronic device 100 may, based on the position of the cropping frame in each frame of the original image, cut out the image within the cropping frame on each frame of the original image, and display the cutout image as a preview image on the shooting interface or as a video recording image. displayed in the recording interface. In this way, when recording a video or displaying a preview image, the electronic device 100 can automatically adjust the focus according to the outline of the photographed object, so as to highlight the detailed features of the photographed object.

In some application scenarios, the electronic device 100 may receive a music clip selected by the user before taking a photo or recording a video. After the electronic device 100 starts recording, the electronic device 100 can control the size change of the cropping frame and/or the movement track of the cropping frame on the original image by using the audio information of the music piece. The electronic device 100 can determine the position and size of the cropping frame on each frame of the original image captured by the camera. One frame of image is cropped, and the cropped image is displayed on the camera preview interface or video recording interface in frame order. In this way, the shooting picture of the electronic device 100 can be automatically zoomed following the rhythm of the music, which enhances the visual effect of the shooting picture.

Exemplarily, as shown in FIG. 14A , when the electronic device 100 receives an input operation (such as a single click) that the user acts on the “smart mirror movement mode” control 331 shown in FIG. 3C , in response to the input operation, the electronic device 100 can A lens motion template selection interface 1410 is displayed. The motion template selection interface 1410 may include one or more motion template options (eg, "travel" motion options, "comfortable" motion options 1411A, "dynamic" motion options 1411B, etc.), music mode Option 1411C, selection box 1412, smart mirror movement prompt box 1413, mirror movement control 1415, and echo control 1416. There is a close control 1414 in the smart motion prompt box 1413 , and the close control 1414 can be used to trigger the electronic device 100 to exit the motion template selection interface 1410 . For the text description of the intelligent motion motion prompt box 1413 , the motion motion shooting control 1415 , and the echo control 1416 , reference may be made to the aforementioned embodiment shown in FIG. 3D , which will not be repeated here.

Wherein, the electronic device 100 may receive the user's operation (eg click) on the music mode option 1411C, and in response to the input, the electronic device 100 may move the selection box 1412 to select the music mode option 1411C, and in the motion template selection interface 1410 A music template selection box 1420 is displayed above. The music template selection box 1420 may include one or more music piece options (eg, music piece 1 option 1421, music piece 2 option, music piece 3 option, music piece 4 option, etc.). Wherein, the audio information of each music segment is different, and different music segments correspond to different cropping frame information (including the size change rule of the cropping frame and/or the motion track of the cropping frame in the original image). A play control and a selection control are displayed on each music piece option (for example, a play control 1423 and a selection control 1422 are displayed on the music piece 1 option 1421), and the play control can be used to trigger the electronic device 100 to play the music piece corresponding to the play control . The selection control can be used to trigger the electronic device 100 to select the music piece corresponding to the selection control.

The electronic device 100 may receive an input (eg, a single click) of the user acting on the selection control 1422, and in response to the input, as shown in FIG. 14B, the electronic device 100 may select the music piece 1 corresponding to the music piece 1 option 1421, and in the A checkmark is displayed on selection control 1422. The check mark displayed on the selection control 1422 can be used to indicate that the electronic device 100 determines the cropping frame information corresponding to the music piece 1 to crop the original image obtained by the camera during video recording.

As shown in FIG. 14B , after the music piece 1 has been selected, the electronic device 100 may receive an input (eg, a single click) from the user acting on the camera control 1415 , and in response to the input, the electronic device 100 may display as shown in FIG. 14C . The video recording interface 1430 shown.

As shown in FIG. 14C , the video recording interface 1430 may include the original image 1461 obtained by the camera, the echo control 1416, the video capture control 1431, the mirror movement template switching control 1432, the intelligent mirror movement prompt box 1413, the viewfinder display window 1441 and the sound Waveform display window 1445. The motion template switching control 1432 can be used to trigger the electronic device 100 to switch the selected music segment, and the motion template switching control 1432 displays the name of the currently selected music segment (for example, "music segment 1"). The viewfinder display window 1441 displays the original image 1461a obtained by the camera (same image content as the original image 1461 above, but with a different display scale), a close control 1442 and a cropping frame 1443 . The sound waveform display window 1445 can be used to display a sound waveform diagram of the selected music piece (eg, music piece 1 ), wherein the signal waveform diagram can be a change relationship diagram of the sound amplitude and time of the selected music piece.

The electronic device 100 may receive user input (eg, a single click) for the video capture control 1431, and in response to the input, the electronic device 100 may start playing the music piece 1 through the speaker, and parse the audio information of the selected music piece. The electronic device 100 can determine the cropping frame information (including the size change of the cropping frame and/or the motion track of the cropping frame on the original image) according to the audio information (including sound amplitude, sound frequency, etc.) of the selected music piece. ). The electronic device 100 can crop the original image obtained by the camera according to the cropping frame information to obtain a cropped image, and display the cropped image on the video recording interface 1430 as a shooting picture. For the relationship between the audio information of the music segment and the cropping frame information, reference may be made to the subsequent embodiments, which will not be repeated here.

As shown in FIG. 14D , when the electronic device 100 starts recording in response to an input (eg, a single click) to the video capture control 1431, the electronic device 100 may display a recording time frame 1434, a recording end control 1435, and echo the controls 1416 and The mode switching control 1432 is replaced with a pause recording control 1435 and a photographing control 1436 , and the electronic device 100 can display the sound waveform diagram of the music piece 1 on the viewfinder display window 1445 . The pause recording control 1435 can be used to trigger the electronic device 100 to pause recording. The photographing control 1436 can be used to trigger the electronic device 100 to save a certain frame or multiple frames of the video recording as a picture. The electronic device 100 determines the position of the cropping frame 1443 on each frame of the original image acquired by the camera based on the audio information of the music piece 1 . The electronic device 100 may sequentially display each frame of the original image obtained by the camera in the viewfinder display window 1441 in frame order, and display the position of the cropping frame 1443 on each frame of the original image in the viewfinder display window 1441 . The electronic device 100 may cut out the image in the cropping frame 1443 on the original image, and adjust the cutout image to a specified size to obtain a cropped image. The electronic device 100 may display the cropped image on the video recording interface 1430 as a captured image.

As shown in FIG. 14D and FIG. 14E , the size and/or the center point of the cropping frame 1443 at the position of the original image changes following the rhythm of the music piece 1 . For example, the size of the cropping frame 1443 can be increased with the increase of the sound amplitude of the music piece 1. The sound amplitude of the music piece 1 at the 3rd second is greater than that at the 8th second. Therefore, the size of the cropping frame 1443 is in the video recording. The size at the 3rd second after the start is larger than the size at the 8th second after the recording starts. The electronic device 100 acquires the original image 1462a through the camera at the third second after the recording starts. The electronic device 100 may cut out the cropped image 1462 from the original image 1462a through the cropping frame 1443 . The electronic device 100 acquires the original image 1463a through the camera at the 8th second after the recording starts. The cropped image 1463 is cut out from the original image 1463a by the cropping frame 1443. The magnification ratio of the image content in the cropped image 1462 is smaller than the magnification ratio of the image content in the cropped image 1463 .

The electronic device 100 may receive a user input (eg, a single click) on the recording end control 1435, and in response to the input, the electronic device 100 may display a multi-frame cropped image and the selected music clip from the start of the recording to the end of the recording The sound in is saved as a video file.

In some application scenarios, for example, when a user picks up the electronic device 100 to photograph a singer on the stage at a concert, the concert scene is filled with the singer's singing and music. The electronic device 100 may collect sound signals through a microphone during a photo preview or a video recording process. The electronic device 100 can control the size change of the cropping frame and/or the movement track of the cropping frame on the original image by collecting audio information (including sound amplitude, sound frequency, etc.) of the sound signal. The electronic device 100 can determine the position and size of the cropping frame on each frame of the original image captured by the camera. One frame of image is cropped, and the cropped image is displayed on the camera preview interface or video recording interface in frame order. In this way, the shooting picture of the electronic device 100 can be automatically zoomed following the rhythm of the sound signal in the environment, which enhances the visual effect of the shooting picture.

Exemplarily, as shown in FIG. 15A , when the electronic device 100 receives an input operation (such as a single click) that the user acts on the “smart mirror movement mode” control 331 shown in FIG. 3C , in response to the input operation, the electronic device 100 can A lens motion template selection interface 1410 is displayed. In addition to the multiple controls shown in FIG. 14A , an ambient sound option 1424 may also be displayed on the lens motion template selection interface 1410 . A selection control 1425 is displayed on the ambient sound option 1424 .

The electronic device 100 may receive user input (eg, a click) on the selected control 1425, and in response to the input, as shown in FIG. 15B, the electronic device 100 may display a checkmark on the selected control 1425, The displayed check mark can be used to indicate that the electronic device 100 determines the ambient sound collected by the microphone during video recording, analyzes the audio information of the ambient sound (such as sound amplitude, sound rhythm, etc.) to crop the original image obtained by the camera.

As shown in FIG. 15B , after the ambient sound option 1423 has been selected, the electronic device 100 may receive an input (eg, a single click) that the user acts on the camera movement control 1415 , and in response to the input, the electronic device 100 may display the display shown in FIG. 14C The recording interface 1530 is shown.

As shown in FIG. 15C , the video recording interface 1530 may include the original image 1561 obtained by the camera, the echo control 1516, the video capture control 1531, the mirror movement template switching control 1532, the intelligent mirror movement prompt box 1513, the viewfinder display window 1541 and the sound Waveform display window 1545. Wherein, the mirror movement template switching control 1532 can be used to trigger the electronic device 100 to switch the music acquisition source, and the word "ambient sound" is displayed on the mirror movement template switching control 1532. The viewfinder display window 1541 includes an original image 1561a (the image content is the same as that in the original image 1561, and the display ratio is different), a close control 1542 and a cropping frame 1543. The sound waveform display window 1545 may be used to display the collected sound waveform graph of the ambient sound, where the sound waveform graph may be a graph of the change between the sound amplitude and time of the ambient sound.

The electronic device 100 may receive an input (eg, a single click) of the video capture control 1531 from the user, and in response to the input, the electronic device 100 may start to collect ambient sound through a microphone in real time, and parse the audio information in the ambient sound. The electronic device 100 may determine the cropping frame information (including the size change of the cropping frame and/or the motion track of the cropping frame on the original image) according to the audio information (including sound amplitude, sound frequency, etc.) of the environmental sound. The electronic device 100 can crop the original image obtained by the camera according to the cropping frame information to obtain a cropped image, and display the cropped image on the video recording interface 1530 as a shooting picture. For the relationship between the audio information of the ambient sound and the cropping frame information, reference may be made to the subsequent embodiments, which will not be repeated here.

As shown in FIG. 15D , when the electronic device 100 starts recording in response to an input (eg, a single click) to the video capture control 1531, the electronic device 100 may display a recording time frame 1534, a recording end control 1535, and echo the controls 1516 and The mode switch control 1532 is replaced by a pause video control 1535 and a photo control 1536 . The electronic device 100 may display the collected sound waveform diagram of the ambient sound on the viewfinder display window 1545 . The pause recording control 1535 can be used to trigger the electronic device 100 to pause recording. The photographing control 1536 can be used to trigger the electronic device 100 to save a certain frame or multiple frames of the video recording as a picture. The electronic device 100 determines the position of the cropping frame 1543 on each frame of the original image acquired by the camera based on the audio information of the ambient sound. The electronic device 100 may sequentially display each frame of the original image obtained by the camera in the viewfinder display window 1541 in frame order, and display the position of the cropping frame 1543 on each frame of the original image in the viewfinder display window 1541 . The electronic device 100 can cut out the image in the cropping frame 1543 on the original image, and adjust the cutout image to a specified size to obtain a cropped image. The electronic device 100 may display the cropped image on the video recording interface 1530 as a captured image.

As shown in FIG. 15D and FIG. 15E , the size and/or the center point of the cropping frame 1543 at the position of the original image changes following the rhythm of the ambient sound. For example, the size of the cropping frame 1543 can be increased with the increase of the sound amplitude of the ambient sound. The sound amplitude of the ambient sound at the 3rd second is greater than that at the 8th second. Therefore, the size of the cropping frame 1543 is after the recording starts. The size at the 3rd second is larger than the size at the 8th second after the recording starts. The electronic device 100 acquires the original image 1562a through the camera at the third second after the start of recording. The electronic device 100 can cut out the cropped image 1562a from the original image 1562a through the cropping frame 1543, and the electronic device 100 obtains the original image 1563a through the camera at the 8th second after the recording starts. The electronic device 100 may cut out the cropped image 1563 from the original image 1563a through the cropping frame 1543 , wherein the magnification of the image content in the cropped image 1562 is smaller than the magnification of the image content in the cropped image 1563 .

The electronic device 100 may receive a user input (eg, a single click) on the recording end control 1535, and in response to the input, the electronic device 100 may display a multi-frame cropped image from the start of the recording to the end of the recording and from the start of the recording to the end of the recording. The ambient sound collected by the microphone within the time of the end of the recording is saved as a video file.

In some embodiments, the electronic device 100 has a touch screen that can be folded, which may be referred to as a folding screen. When the touch screen of the electronic device 100 is in a half-folded state (for example, the folding angle is about 90 degrees), the touch screen of the electronic device 100 can be divided into at least an A screen and a B screen. For some applications on the electronic device 100, customized interface layouts and functional control layouts may be preset for the half-folded form of the folding screen. Taking the camera application as an example, when the folding screen of the electronic device 100 is in the unfolded state, the electronic device 100 can display the camera application interface (or video recording interface). The device 100 can display the preview image in the shooting interface (or the recording image in the recording interface) on the A screen, and display the function controls in the shooting interface (or the recording interface) (or the function controls in the recording interface) on the B screen.

For example, when the folding screen is in the unfolded state, the electronic device 100 may display the video recording interface 1530 shown in FIG. 15C on the touch screen. When the folded screen is switched to the half-folded state, the electronic device 100 can display the video recording image in the above-mentioned recording interface 1530 on the A screen (for example, the original image 1561 in FIG. 15C or the cropped image 1562 in FIG. 15D ), and on the B screen The function controls in the video recording interface 1530 (such as the echo control 1516, the video capture control 1531, the mirror movement template switching control 1532, the intelligent mirror movement prompt box 1513, the viewfinder display window 1541 and the sound waveform display window 1545 in FIG. ,and many more). In this way, various controls in the application interface can be rearranged according to the physical form of the folding screen, so that the user can obtain a better visual experience when viewing and using the folding screen.

In some embodiments, the electronic device 100 may display different contents in the same application interface in a split screen. When the application split screen is activated, the touch screen of the electronic device 100 may be divided into multiple display areas, and the multiple display areas include display area 1 and display area 2 . Taking a camera application as an example, when the electronic device 100 does not enable the application split screen, the electronic device 100 may display a shooting interface (or a video recording interface) of the camera application. When the electronic device 100 enables the application split screen, the electronic device 100 can display the preview image in the shooting interface (or the video recording image in the video recording interface) on the display area 1 of the touch screen, and display the preview image on the display area 2 of the touch screen The function controls in the shooting interface (or the recording interface) (or the function controls in the recording interface).

For example, when the electronic device 100 does not enable the application split screen, the electronic device 100 can display the video recording interface 1530 shown in FIG. 15C on the touch screen. After the electronic device 100 enables the application split screen, the electronic device 100 can display the video recording image in the above-mentioned video recording interface 1530 on the display area 1 of the touch screen (for example, the original image 1561 in FIG. 15C or the cropped image 1562 in FIG. 15D ). ), the function controls in the recording interface 1530 are displayed on the display area 2 of the touch screen (for example, the echo control 1516 in FIG. viewfinder display window 1541 and sound waveform display window 1545, etc.). In this way, various controls in the application interface can be rearranged according to whether the application split screen is enabled or not, so that the user can obtain a better visual experience after viewing and using the application split screen.

In the embodiment of the present application, the operation of the user triggering the electronic device 100 to start recording, end recording, or take a photo is not limited to controls displayed on the touch screen (eg, start recording controls, end recording controls, shooting controls, etc.). Touch input such as clicks. Optionally, the user triggers the electronic device 100 to start recording, end recording, or take a photo, etc., which may also be voice input. For example, when the electronic device 100 receives the user's voice input, the electronic device 100 can recognize the voice input. Semantics (eg "eggplant", "photograph", "start video recording", "end video recording", etc.), control the electronic device 100 to start video recording, end video recording or take photo, and so on. Optionally, the electronic device 100 may also trigger to take a photo, start or end video recording, etc. by recognizing a designated gesture or facial expression of a person in the shooting preview screen. The electronic device 100 can also control the electronic device 100 to start video recording, end video recording or take pictures, etc. .

The following describes the determination of the above-mentioned cropping frame information (including the size change rule of the cropping frame and/or the motion track of the cropping frame in the original image) according to the audio information of the sound signal in the embodiments of the present application.

The electronic device 100 acquires a sound signal during video recording or shooting preview. The sound signal may be audio data corresponding to the music segment selected by the user in the foregoing embodiment, or may be ambient sound collected by the electronic device 100 through the microphone in the foregoing embodiment. The electronic device 100 may parse audio information of the sound signal, where the audio information of the sound signal includes rhythm information, amplitude information, style information, and the like. The electronic device 100 can adjust the size of the cropping frame, the changing speed of the cropping frame (including the changing speed of the size of the cropping frame and/or the moving speed of the center point of the cropping frame, etc.), filter effects (including different color lighting effects) and so on.

The electronic device 100 may, after acquiring the sound signal, sample and quantize the sound signal to obtain window data. The electronic device 100 may perform difference processing on the window data to obtain difference data. Next, the electronic device 100 may perform Fourier transform on the difference data and differentiate again to obtain a rhythm signal. As shown in FIG. 16 , the electronic device 100 can obtain the rhythm start point and the rhythm end point from the rhythm signal. For example, the electronic device 100 identifies the rhythm start point of the i-th rhythm, the rhythm end point of the The starting point of the i+1 rhythm, the ending point of the i+1 th rhythm, etc., i is a positive integer. Among them, the length between the rhythm start point and the rhythm end point of a rhythm indicates the speed of the rhythm.

In some embodiments, the electronic device 100 can control the size change of the cropping frame on the original image through the rhythm of the sound signal. Wherein, when the rhythm of the sound signal becomes faster, the electronic device 100 can control the size of the cropping frame on the original image to increase, and when the rhythm of the sound signal becomes slower, the electronic device 100 can control the size of the cropping frame on the original image to increase Small. When the size of the cropping frame is proportionally enlarged to the maximum size, the electronic device 100 may control the size of the cropping frame to be proportionally smaller in reverse. When the size of the cropping frame becomes larger to the preset minimum size, the electronic device 100 may control the size of the cropping frame to increase proportionally and inversely.

In a possible implementation, the electronic device 100 may identify the rhythm termination time point of the sound signal, and at the rhythm termination time point of the sound signal, control the size of the cropping frame to be larger or smaller by a specified multiple.

In some embodiments, the electronic device 100 may control the cropping frame to move on the original image. The electronic device 100 can control the moving speed of the cropping frame on the original image through the rhythm of the sound signal. Wherein, when the rhythm of the sound signal becomes faster, the electronic device 100 can control the moving speed of the cropping frame on the original image to increase. When the rhythm of the sound signal becomes slower, the electronic device 100 may control the moving speed of the cropping frame on the original image to become smaller. When the cropping frame moves to the boundary of the original image in one direction, the electronic device 100 can control the cropping frame to move in the reverse direction, or control the positive and negative directions of the cropping frame to move 90 degrees.

In a possible implementation, the electronic device 100 may identify the rhythm ending time point (or rhythm starting time point) of the sound signal, and at the rhythm ending time point (or rhythm starting time point) of the sound signal, control the The switching of the crop box moves the direction once.

In some embodiments, the electronic device 100 can control the size change of the cropping frame on the original image through the amplitude of the sound signal. Wherein, when the amplitude of the sound signal increases, the electronic device 100 can control the size of the cropping frame on the original image to increase, and when the amplitude of the sound signal decreases, the electronic device 100 can control the size of the cropping frame on the original image to decrease . When the size of the cropping frame is proportionally enlarged to the maximum size, the electronic device 100 may control the size of the cropping frame to be proportionally smaller in reverse. When the size of the cropping frame becomes larger to the preset minimum size, the electronic device 100 may control the size of the cropping frame to increase proportionally and inversely.

In some embodiments, the electronic device 100 may control the cropping frame to move on the original image. The electronic device 100 can control the moving speed of the cropping frame on the original image through the amplitude of the sound signal. Wherein, when the amplitude of the sound signal increases, the electronic device 100 can control the moving speed of the cropping frame on the original image to increase. When the rhythm of the sound signal becomes slower, the electronic device 100 may control the moving speed of the cropping frame on the original image to become smaller. When the cropping frame moves to the boundary of the original image in one direction, the electronic device 100 can control the cropping frame to move in the reverse direction, or control the positive and negative directions of the cropping frame to move 90 degrees.

In some embodiments, the electronic device 100 can control the size change of the cropping frame on the original image through the rhythm of the sound signal, the electronic device 100 can control the cropping frame to move on the original image, and control the size of the cropping frame on the original image by the amplitude of the sound signal. Movement speed on the original image.

In some embodiments, the electronic device 100 can control the size change of the cropping frame on the original image through the amplitude of the sound signal, the electronic device 100 can control the cropping frame to move on the original image, and control the size of the cropping frame on the original image through the rhythm of the sound signal. Movement speed on the original image.

In some embodiments, the electronic device 100 may determine the filter color (eg, red, orange, yellow, green, cyan, blue, purple, etc. color lighting effects) through the musical style of the sound signal. After the electronic device 100 intercepts the image in the cropping frame on the original image, it can apply a filter effect to the cropped image based on the filter color, and display the cropped image after the filter effect on the above-mentioned shooting interface or the above-mentioned video recording interface. middle. For example, the electronic device 100 may change a filter color when recognizing that the main chorus of the sound signal is switched.

In a possible implementation manner, the electronic device 100 may identify the rhythm start time point (or rhythm end time point) of the sound signal, and at the rhythm start time point (or rhythm end time point) of the sound signal, change the Filter color once.

In some application scenarios, the electronic device 100 may store video files locally. The electronic device 100 may be based on the cropping frame information (including the size change rule of the cropping frame and/or the motion track of the cropping frame on the video screen) corresponding to the motion template selected by the user. The electronic device 100 can trim each frame of video picture in the video file according to the cropping frame information corresponding to the mirror motion template, and adjust the cropped video picture to a specified size to obtain a cropped video. In this way, the electronic device 100 can trim the saved video file according to the trimming information corresponding to the lens motion template selected by the user, re-zoom the video image, and can highlight the subject or scene background in the video image.

Exemplarily, as shown in FIG. 17A , the electronic device 100 displays an interface 310 with a home screen. The text description for the interface 310 may refer to the embodiment shown in FIG. 3A , which will not be repeated here.

The electronic device 100 may receive user input (eg, a click) on the gallery application icon 312, and in response to the input, the electronic device 100 may display the gallery application interface 1710 as shown in FIG. 17B.

As shown in FIG. 17B, the gallery application interface 1710 can be displayed to include one or more albums (eg, all photo albums, video album 1716, camera album, WeChat album, Weibo album, etc.). The electronic device 100 may display a gallery menu 1711 below the gallery album interface 1710 . The gallery menu 1711 includes a photo control 1712, an album control 1713, a time control 1714, and a discovery control 1715. The photo control 1712 is used to trigger the electronic device 100 to display all local pictures in the form of picture thumbnails. The album control 1713 is used to trigger the electronic device 100 to display the album to which the local picture belongs. As shown in FIG. 17B , the photo album control 1713 is currently selected, and the electronic device 100 displays the gallery application interface 1710 . The moment control 1714 can be used to trigger the electronic device 100 to display a locally stored featured picture. The discovery control 1715 can be used to trigger the electronic device 100 to display a classified album of pictures.

The electronic device 100 may receive user input (eg, a click) on the video album 1716, and in response to the input, the electronic device 100 may display the video album interface 1720 as shown in FIG. 17C.

As shown in Figure 17C, the video album interface 1720 may include one or more video file options (eg, video file option 1721 and video file option 1722).

The electronic device 100 may receive user input (eg, a click) on the video file option 1721, and in response to the input, the electronic device 100 may display a video browsing interface 1730 as shown in FIG. 17D.

As shown in FIG. 17D , the video browsing interface 1730 includes a video 1731 , a video playback control 1732 , a total video time 1733 , a menu 1734 , and a mirror movement mode control 1735 . For the textual description of the menu 1734, reference may be made to the foregoing embodiments, which will not be repeated here.

The electronic device 100 may receive user input (eg, a single click) on the motion mode control 1735, and in response to the input, the electronic device 100 may display a motion template selection interface 1740 as shown in FIG. 17E.

As shown in FIG. 17E, the motion template selection interface 1740 includes one or more motion template options (eg, "travel" motion option 1741A, "comfortable" motion option 1741B, "dynamic" motion option 1741C, etc. etc.), the selection box 1742, the confirmation control 1743, the cancel control 1744, the mirror movement template display area 1751, and the mirror movement track display area 1753. The cancel control 1743 can be used to trigger the electronic device 100 to exit the lens motion template selection interface 1740 . Wherein, the size change rules of the cropping frame or the motion trajectory of the cropping frame corresponding to different mirror motion template options are different. As shown in FIG. 17E , the “travel” camera movement option 1741A has been selected in the selection box 1742 , and the video sample corresponding to the “travel” camera movement option 1741A is displayed in the camera movement template display area 1751 . The camera movement track display area 1753 may display the cropping frame size change rule corresponding to the "travel" camera movement option 1741A and/or the motion trajectory of the cropping frame in the original image of the video dailies. For example, the center point of the crop box corresponding to the "travel" motion option 1741A may be at the same location as the center point of the original image, and the size of the crop box corresponding to the "travel" motion option 1741A may first decrease proportionally from the size of the original image over time. to the preset minimum size, and then scale up from the preset minimum size to the size of the original image. The size of the cropping frame corresponding to the "travel" lens movement option 1741A changes proportionally to the size of the original image. The preset minimum size is smaller than the size of the original image.

The electronic device 100 may receive an input operation (eg, a single click) of the user acting on the determination control 1743, and in response to the input operation, the electronic device 100 may use the cropping box size change rule and /Or the motion track of the cropping frame in the original image, crop the video picture stream in the video 1731 to obtain the cropped video 1761, and display the cropped video 1761 on the video browsing interface 1760 as shown in FIG. 17F .

As shown in FIG. 17F , the video browsing interface 1760 may include a trimmed video 1761 , a video playback control 1762 , a total duration of the trimmed video 1763 , a mirror motion template switching control 1764 and a save control 1765 . The video playback control 1762 can be used to trigger the electronic device 100 to save the cropped video 1761 to a specified storage path. The moving template switching control 1764 can be used to trigger the electronic device 100 to switch the moving template.

In some application scenarios, the electronic device 100 may store video files locally. The electronic device 100 may receive a piece of music selected by the user or collect ambient sounds through a microphone. The electronic device 100 can control the size change of the cropping frame and/or the motion track of the cropping frame on the original video picture in the video file according to the audio information of the music piece or the ambient sound. The electronic device 100 can determine the position and size of the cropping frame on each frame of the original image captured by the camera, and based on the position and size of the cropping frame on each frame of the original image, each frame of video image Perform cropping, and adjust the cropped video image to a specified size to obtain a cropped image stream. The electronic device 100 may save the cropped image stream and the above-mentioned music clip (or ambient sound) as a video file. In this way, the video picture can be automatically zoomed following the rhythm of the music, which enhances the visual effect of the shooting picture.

Exemplarily, as shown in FIG. 17G , the electronic device 100 may display a lens motion template selection interface 1740 . When the electronic device 100 receives and in response to the user's input moves the selection box 1742 to the music mode option 1742C, the electronic device 100 may display the music template selection box 1770 on the lens motion template selection interface 1740 to include one or more Music segment options (eg, music segment 1 option 1771 , music segment 2 option, music segment 3 option, music segment 4 option, etc.), and ambient sound option 1774 . A play control and a selection control are displayed on each music piece option (for example, a play control 1773 and a selection control 1772 are displayed on the music piece 1 option 1771), and the play control can be used to trigger the electronic device 100 to play the music piece corresponding to the play control . The selection control can be used to trigger the electronic device 100 to select the music piece corresponding to the selection control. A selection control 1775 is displayed on the ambient sound option 1774 .

The electronic device 100 may receive user input (eg, a click) on the selected control 1775, and in response to the input, as shown in FIG. 17G , the electronic device 100 may display a checkmark on the selected control 1775 The displayed check mark can be used to represent the ambient sound collected by the electronic device 100 through the microphone, and to crop each frame of the video picture in the above-mentioned video 1731 .

As shown in FIG. 17G , after the ambient sound option 1774 has been selected, the electronic device 100 may receive an input (eg, a single click) from the user acting on the determination control 1743, and in response to the input, the electronic device 100 may start collecting ambient sounds through the microphone , and the ambient sound collection interface 1780 shown in FIG. 17H is displayed.

As shown in FIG. 17H , the ambient sound collection interface 1780 includes a waveform diagram 1781 of the ambient sound, a collection pause control 1782 , a collection time bar 1783 , and a recollection control 1784 . The waveform diagram 1781 of the ambient sound is used to display the signal waveform of the ambient sound. The collection pause control 1782 can be used to trigger the electronic device 100 to pause the collection of ambient sounds through the microphone. The collection time bar 1783 is used to indicate the total time length (for example, 12 seconds) that the electronic device 100 needs to collect ambient sounds and the time length that has been collected (for example, 8 seconds of ambient sounds are currently collected), wherein the total time for collecting ambient sounds The length is the broadcast time length of the video to be trimmed.

As shown in FIG. 17I , when the time for collecting ambient sound by the electronic device 100 reaches the broadcast time length of the video to be trimmed, the electronic device 100 may display an ambient sound playback control 1785 and a determination control 1786 on the ambient sound trimming interface 1780 . The ambient sound playback control 1785 can be used to trigger the electronic device 100 to play the collected ambient sound. In this way, the user can audition whether the collected environment is suitable.

The electronic device 100 may receive the user's input (for example, a single click) for the determination control 1786, and in response to the input, the electronic device 100 may use the collected ambient sound to trim each frame of the video to be trimmed. And enlarge the cropped picture to the specified size to get the cropped picture flow. The electronic device 100 can combine the cropped image stream and the collected ambient sound to obtain a cropped video 1791, and display the cropped video 1761 on the video browsing interface 1790 as shown in FIG. 17J .

As shown in FIG. 17J , the video browsing interface 1790 may include a trimmed video 1791 , a video playback control 1792 , a total duration of trimmed video 1793 , a mirror motion template switching control 1794 and a save control 1795 . The video playback control 1792 can be used to trigger the electronic device 100 to save the cropped video 1791 to a specified storage path. The moving template switching control 1794 can be used to trigger the electronic device 100 to switch the moving template.

The following introduces a video stream cropping system provided in the embodiments of the present application.

FIG. 18 shows a schematic diagram of a video stream cropping system 1800 provided in an embodiment of the present application. Wherein, the video stream cropping system 1800 can be applied to the above-mentioned electronic device 100 to execute the image processing method in the above-mentioned embodiment.

As shown in FIG. 18 , the video stream cropping system 1800 may include: an image stabilization module 1801 , a mirror movement control module 1802 , and a super-division processing module 1803 .

Wherein, the electronic device 100 obtains the video stream through the camera when the above-mentioned mirror movement mode is enabled for recording. Alternatively, the electronic device 100 may acquire the video stream through the camera when the above-mentioned mirror movement mode is turned on and the shooting interface is displayed. Alternatively, the electronic device 100 stores a video file locally, and the electronic device 100 can acquire the video stream in the video file.

In a possible implementation manner, the electronic device 100 may have multiple cameras, and the multiple cameras include a wide-angle camera. When the electronic device 100 enables the above-mentioned mirror movement mode, the electronic device 100 obtains the video stream by using the wide-angle camera. In this way, the electronic device 100 can obtain a video picture with a wider viewing angle.

When the electronic device 100 obtains the video stream through the camera, the electronic device 100 may perform image stabilization processing on the video stream through the image stabilization module 1802 .

In a possible implementation manner, the image stabilization module 1801 may perform electronic image stabilization (electronic image stabilization, EIS) processing on the video stream. Specifically, the image stabilization module 1801 can obtain the electronic device 100 through sensors such as a gyroscope sensor to detect the jitter information of the electronic device 100 when it obtains a video stream through a camera. The image stabilization module 1801 may calculate motion information of adjacent frames in the video stream based on the jitter information. The image stabilization module 1801 can perform motion compensation for each pixel in the frame image based on the motion information of the adjacent frame images, so as to achieve the purpose of stabilizing the image.

In other possible implementation manners, the image stabilization module 1801 may also perform optical image stabilization on the video stream. Specifically, the electronic device 100 can adaptively adjust the optical path through optical visibility to compensate for image motion caused by the camera platform shake.

The image stabilization module 1801 can also perform mechanical image stabilization when acquiring a video stream through a camera. Specifically, the image stabilization module 1801 can acquire the shaking information of the electronic device 100 detected by the electronic device 100 through a device such as a gyroscope sensor, and then adjust the servo system of the camera to achieve the purpose of stabilizing the image.

The above-mentioned image stabilization module 1801 is optional, and it is optional to perform image stabilization processing on the acquired video stream, and the electronic device 100 may perform cropping through the mirror movement control module without performing image stabilization.

The motion control module 1802 can trim the picture of the video stream based on a preset motion template or a sound signal. Specifically, the motion control module 1802 can determine the cropping frame information according to the motion template selected by the user or the rhythm information of the sound signal. The cropping information includes the size and position of the cropping frame on each frame of the video stream. The mirror motion template defines the size change rule of the cropping frame and the motion track of the cropping frame on the original image obtained by the camera. The sound signal may be a music clip stored locally by the electronic device 100, or may be collected by the microphone when the electronic device 100 starts recording or displays the shooting interface in the above-mentioned mirror motion mode.

After determining the size and position of the cropping frame on each frame in the video stream, the mirror movement control module 1802 can cut out the image in the cropping frame in each frame, and adjust the cutout image to the specified size. size, get the cropped video stream.

After obtaining the cropped video stream, the mirror motion control module 1802 inputs the cropped video stream to the super-division processing module 1803 . The super-resolution processing module 1803 can perform image super-resolution reconstruction on the pictures of the cropped video stream. In this way, the clarity of the picture can be improved, the details of the picture can be enriched, and the noise in the picture can be removed. The electronic device 100 may display the super-divided cropped video stream on the video recording interface or the shooting interface in the foregoing embodiment. The electronic device 100 may also save the super-divided cropped video stream in response to the user's input.

In a possible implementation manner, the above-mentioned super-division processing module 1803 is optional, and the electronic device 100 can directly display the cropped video stream processed by the camera movement control module 1802 on the video recording interface or shooting interface in the above-mentioned embodiment. middle. The electronic device 100 may also save the cropped video stream in response to the user's input.

FIG. 19 shows a schematic diagram of a video stream cropping system 1900 provided in an embodiment of the present application. Wherein, the video stream cropping system 1900 can be applied to the above-mentioned electronic device 100 to execute the image processing method in the above-mentioned embodiment.

The video stream cropping system 1900 may include a camera 1901 , an image signal processor (ISP) 1902 , a memory 1903 , and an application processor (AP) 1904 .

The camera 1901 can convert the optical signal into an electrical signal after receiving the optical signal, and input the electrical signal to the image signal processor 1902 . The image signal processor 1902 can output YUV data through a signal processing algorithm and store it in the memory 1903 .

The application processor 1904 can read the YUV data in the memory 1903 (for example, the size is 4608*2592 or 3840*2160), and determine the position and size of the cropping frame in the current frame based on the preset motion template and the sound signal. The application processor 1904 can cut out the image in the cropping frame on the current frame and adjust it to a specified size (for example, 1920*1080 or 3840*2160). The application processor 1904 can cut out the image in the cropping frame in each frame of the video stream, and adjust it to a specified size to obtain the cropped video stream.

In a possible implementation manner, after determining the position and size of the cropping frame on the current frame, the application processor 1904 may send the position and size of the cropping frame on the current frame to the image signal processor 1902 . The image signal processor 1902 may, based on the position and size of the cropping frame on the current frame, cut out the image within the cropping frame on the current frame, and adjust it to a specified size. The image signal processor 1902 obtains the cropped video stream by extracting the image within the cropping frame in each frame of the video stream, and adjusting it to a specified size.

In a possible implementation manner, the electronic device 100 needs to identify the image content in the shooting picture, detect and track the position of the designated image content, and determine the outline of the designated image content. The video stream cropping system 1900 described above also includes a neural network processor (not shown in FIG. 19). After reading the YUV data, the application processor 1904 can preprocess the YUV data. Among them, the preprocessing process includes the application processor 1904 (or the application processor 1904 instructs digital signal processing) to convert the YUV data into an RGB image (or a BGR image or a single-channel image or a grayscale image, etc.), and downsample to the AI model The input image size (for example, 224*224, 288*288, etc.). The application processor 1904 may feed the preprocessed RGB image (or BGR image or single channel image or grayscale image, etc.) to the neural network processor. The neural network processor can detect and track the position of the specified image content in the RGB image (or BGR image or single-channel image or grayscale image, etc.) through the AI model, and determine the outline of the specified image content. The neural network processor may output detection frame information or outline information specifying image content to the application processor 1904 . The application processor 1904 may determine the above-mentioned cropping frame information according to the detection frame information/or outline information of the specified image.

In some embodiments, the application processor 1904 may detect the jitter information of the electronic device 100 through the gyro sensor while the image signal processor 1902 outputs the YUV data. After acquiring the YUV data, the application processor 1904 can calculate the motion information of adjacent frames in the video stream based on the jitter information. The application processor 1904 may perform motion compensation on each pixel in the frame based on the motion information of the adjacent frames, so as to achieve the purpose of stabilizing the image.

In one possible implementation, the application processor 1904 may perform electronic image stabilization (EIS) processing on the video stream. Specifically, the application processor 1904 may obtain the electronic device 100 through a sensor such as a gyroscope sensor to detect the jitter information of the electronic device 100 when it obtains a video stream through a camera. The application processor 1904 may calculate motion information of adjacent frames in the video stream based on the jitter information. The application processor 1904 may perform motion compensation on each pixel in the frame based on the motion information of the adjacent frames, so as to achieve the purpose of stabilizing the image.

Possibly, after calculating the motion information of adjacent frames in the video stream based on the jitter information, the application processor 1904 may send the motion information of the adjacent frames to the graphics processor (not shown in FIG. 19 ). The graphics processor can perform motion compensation for each pixel in the frame based on the motion information of the adjacent frames, so as to achieve the purpose of stabilizing the image.

In other possible implementations, the application processor 1904 may also perform optical image stabilization on the video stream. Specifically, the application processor 1904 can adaptively adjust the optical path through optical visibility to compensate for image motion caused by camera platform shake.

The application processor 1904 can also perform mechanical image stabilization when acquiring the video stream through the camera. Specifically, the application processor 1904 can acquire the jitter information of the electronic device 100 detected by the electronic device 100 through a device such as a gyroscope sensor, and then adjust the servo system (sevomechanism) of the camera to achieve the purpose of stabilizing the image.

In some embodiments, the application processor 1904 may perform image super-resolution reconstruction on pictures of the cropped video stream. In this way, the clarity of the picture can be improved, the details of the picture can be enriched, and the noise in the picture can be removed.

In a possible implementation manner, the application processor 1904 directly outputs to the super-resolution neural network model after cropping each frame in the video stream based on the cropping frame, and uses the super-resolution neural network model to process the cropped picture. Perform image super-resolution reconstruction and adjust to the specified size to obtain the cropped video stream after super-resolution.

In a possible implementation manner, after the application processor 1904 trims each frame of the video stream based on the cropping frame, it is adjusted to a specified size, and then output to the super-resolution neural network model, and the super-resolution neural network is passed through the super-resolution neural network model. The network model performs image super-resolution reconstruction on the cropped picture, and obtains the cropped video stream after super-resolution.

In this embodiment of the present application, when the electronic device 100 receives and responds to the user's input acting on the camera application icon, and displays the shooting interface, the video stream of the first frame rate can be obtained through the camera. After the electronic device 100 receives and responds to the user's input of enabling the mirror motion mode, the electronic device 100 can obtain a video stream with a second frame rate through the camera, where the second frame rate is higher than the first frame rate. In this way, the electronic device 100 can use multiple frames of pictures for fusion (for example, every 3 to 7 frames of pictures) during the super-division process, which improves the picture definition of the cropped video stream.

After the electronic device 100 acquires the multi-frame pictures in the cropped video stream during the super-division process, it can select one of the frame pictures as the reference frame picture, and register the rest of the multi-frame pictures to the reference frame picture. . The electronic device 100 can input the registered multiple frames into the super-resolution neural network model, and fuse the multiple frames through the super-resolution neural network model to obtain high-definition frame images.

Wherein, the electronic device 100 can increase the image capture frame rate of the camera, thereby increasing the picture frame rate of the video stream. For example, before the electronic device 100 starts the mirror movement mode, the camera may capture images at a frame rate of 30 frames per second. After the electronic device 100 enables the mirror movement mode, the camera can capture images at a frame rate of 90 frames per second.

In a possible implementation manner, the electronic device 100 may insert frames by software after the camera captures the image picture, so as to improve the picture frame rate of the video stream. For example, before the electronic device 100 enables the mirror movement mode, the camera may capture images at a frame rate of 30 frames per second without performing frame interpolation to obtain a video stream of 30 frames per second. After the electronic device 100 enables the mirror movement mode, the camera can capture images at a frame rate of 90 frames per second, and insert 90 frames of images per second to obtain a video stream of 180 frames per second.

The following describes the flow of an image processing method provided by an embodiment of the present application.

FIG. 20 shows a schematic flowchart of an image processing method provided by an embodiment of the present application.

As shown in Figure 20, the image processing method includes:

S2001 , the electronic device 100 displays a first shooting preview interface, the first shooting preview interface includes a first preview frame, and the first preview frame displays a picture captured by a camera of the electronic device 100 in real time.

The first shooting preview interface may be the shooting interface 320 in the embodiment shown in FIG. 3B. In some embodiments, the first shooting preview interface may also be the shooting interface 320 in the embodiment shown in FIG. 6A above. For specific content, reference may be made to the foregoing embodiments, which will not be repeated here.

S2002. After the electronic device 100 detects the first operation on the first shooting preview interface, the electronic device 100 displays a shooting option interface. Wherein, the shooting option interface includes a first shooting option and a second shooting option.

The first operation may be the operation for the more mode control 327G in the embodiment shown in FIG. 3B , and the shooting option interface may be the lens movement template selection interface 340 in the embodiment shown in FIG. 3D . After the electronic device 100 detects an operation for the more mode control 327G, the electronic device 100 may display the mode selection page 330 shown in FIG. 3C described above. The electronic device 100 may detect an operation (eg, a single click) on the "Smart Mirror Mode" control 331 in the mode selection page 330, and in response to the operation on the "Smart Mirror Mode" control 331, the electronic device 100 may display the Lens motion template selection interface 340 . For example, the first shooting option may be the "travel" motion option 341A in the lens motion template selection interface 340 shown in FIG. 3D, and the second shooting option may be the "Motion" motion in the lens motion template selection interface 340 shown in FIG. 3D. ” Mirror template option 341C.

In some embodiments, the first operation may be the operation for the more mode control 327G in the embodiment shown in FIG. 3B , and the shooting option interface may be the lens movement template selection interface 1410 in the embodiment shown in FIG. 14A . . After the electronic device 100 detects an operation for the more mode control 327G, the electronic device 100 may display the mode selection page 330 shown in FIG. 3C described above. The electronic device 100 may detect an operation (eg, a single click) on the "Smart Mirror Mode" control 331 in the mode selection page 330, and in response to the operation on the "Smart Mirror Mode" control 331, the electronic device 100 may display a graph. Motion template selection interface 1410 shown in 14A. For example, the first shooting option may be the above-mentioned music piece 1 option 1421 shown in FIG. 14A , and the second shooting option may be the above-mentioned music piece 2 option shown in FIG. 14B .

S2003. After the electronic device 100 detects the second operation for the first shooting option, the electronic device 100 displays a second shooting preview interface. Wherein, the second preview interface includes a second preview frame, and the second preview frame displays an image captured by the camera of the electronic device 100 in real time.

The second operation may be an operation for the “travel” camera movement option 341A in the embodiment shown in FIG. 3D (for example, a single click), and the second shooting preview interface may be the video recording interface in the embodiment shown in FIG. 3I . 360 or the shooting interface 510 in the embodiment shown in FIG. 5B. After the electronic device 100 detects the operation on the “travel” camera movement option 341A, the electronic device 100 may also detect the operation on the camera movement control 345 in the embodiment shown in FIG. 3D or FIG. 3E or FIG. 5A (for example, a single click), and in response to the operation on the camera movement control 345, the electronic device 100 may display the above-mentioned recording interface 360 or the above-mentioned photographing interface 510.

S2004, the electronic device 100 starts to shoot the first video content.

In a possible implementation manner, before the electronic device 100 starts to capture the first video content, the electronic device 100 detects a fourth operation on the second capture preview interface. In response to the fourth operation, the electronic device 100 starts to capture the first video content.

Exemplarily, the second shooting preview interface may be the video recording interface 360 in the embodiment shown in FIG. 3I. The fourth operation may be an operation (eg, a single click) on the video capture control 361 in the video recording interface 360 , and in response to the operation on the video capture control 361 in the video recording interface 360 , the electronic device 100 may start to capture the first video content.

S2005 , at the first moment after starting to shoot the first video content, the electronic device 100 displays, in the second preview frame, the first part of the picture captured in real time by the camera of the electronic device 100 .

S2006 , at a second moment after starting to shoot the first video content, the electronic device 100 displays, in a second preview frame, the second part of the picture captured in real time by the camera of the electronic device 100 .

For example, 3 seconds after the electronic device 100 starts shooting the first video content, the first part displayed in the second preview frame may be the cropped image 372 in the embodiment shown in FIG. 3J. Eight seconds after the electronic device 100 starts shooting the first video content, the second preview frame displays that the second part may be the cropped image 373 in the embodiment shown in FIG. 3K.

Wherein, the second shooting preview interface further includes a first window. The electronic device 100 displays the picture captured in real time by the camera of the electronic device 100 in the first window. At the first moment after starting to capture the first video content, the electronic device 100 displays the bounding box of the first part in the first window. At a second moment after starting to capture the first video content, the electronic device 100 displays the bounding box of the second portion in the first window. In this way, it is convenient for the user to see the picture captured by the camera in real time, and it is convenient for the user to find the subject to be photographed.

Exemplarily, the first window may be the viewfinder display window 521 in the embodiments shown in FIGS. 5B-5E.

In a possible implementation manner, at the first moment after starting to shoot the first video content, the electronic device 100 displays the video at the first moment in the video sample corresponding to the first shooting option in the first window screen. At a second moment after starting to shoot the first video content, the electronic device 100 displays, in the first window, a video frame at the second moment in the video sample corresponding to the first shooting option. In this way, it is convenient for users to compare the shooting effects in real-time with the video samples.

Exemplarily, the first window may be the mirror moving template preview window 363 in the embodiments shown in FIG. 3I-FIG. 3M.

Wherein, the display position of the first window and the second preview frame is any one of the following: at least part of the first window overlaps with the display area of the second preview frame; or, the first window is displayed on the second shooting preview interface on the first Two positions outside the preview frame; or, the first window is displayed in the upper right area of the second preview frame; or, the first window is displayed in the upper left area of the second preview frame.

The first window includes a window closing control; the electronic device 100 detects an eighth operation for the window closing control in the first window; in response to the eighth operation, the electronic device 100 closes and displays the first window. In this way, when the user does not need the first window, the user can be manually triggered to close, so as to ensure that the shooting picture displayed on the electronic device 100 is not blocked.

After the electronic device 100 closes and displays the first window, the electronic device 100 displays the window opening control in the second shooting preview interface; the electronic device 100 detects a ninth operation for the window opening control; in response to the ninth operation, The electronic device 100 displays the first window on the second shooting preview interface. In this way, the user can trigger the electronic device 100 to redisplay the first window when the user needs to call the widget.

For specific content, reference may be made to the foregoing embodiments shown in FIG. 4D to FIG. 4F , which will not be repeated here.

In some embodiments, the second shooting preview interface further includes a second window, and the electronic device 100 displays the waveform diagram of the first music segment corresponding to the first shooting option in the second window. Wherein, the audio information in the waveform chart at the first moment after starting to shoot the first video content is different from the audio information in the waveform chart at the second moment after starting to shoot the first video content, and the audio information includes the following One or more of: tempo information, amplitude information, and style information.

Wherein, after the electronic device 100 starts to shoot the first video content, the electronic device 100 may play the first music segment.

When the electronic device 100 saves the first part as a video frame at the first moment in the video file, and saves the second part as a video frame at the second moment in the video file, the electronic device 100 saves the first music segment in the in the audio data of the video file.

Exemplarily, the second window may be the sound waveform display window 1445 in the embodiment shown in FIG. 14C-FIG. 14E. For specific content, reference may be made to the foregoing embodiments shown in FIGS. 14C to 14E , which will not be repeated here.

The electronic device 100 detects a sixth operation on the second shooting preview interface after the electronic device 100 displays the second part of the picture captured by the camera of the electronic device 100 in real time in the second preview frame. In response to the sixth operation, the electronic device 100 saves the first part as a video frame at the first moment in the video file, and saves the second part as a video frame at the second moment in the video file. In this way, the electronic device 100 can shoot video recordings with effects such as automatic zooming or lens panning.

Exemplarily, for the playback picture of the video file, reference may be made to the foregoing embodiments shown in FIGS. 3N-3Q , and details are not described herein again.

In a possible implementation manner, the electronic device 100 detects the seventh operation at a first moment after starting to capture the first video content. In response to the seventh operation, the electronic device 100 saves the first part as a picture. For example, the seventh operation may be an operation (eg, a single click) for the photographing control 368 in the embodiment shown in FIG. 3J . For specific content, reference may be made to the aforementioned embodiment shown in FIG. 3J , which will not be repeated here.

S2007, the electronic device 100 displays a shooting option interface.

In a possible implementation manner, the second shooting preview interface further includes an option switching control; after the electronic device 100 displays the second part of the picture captured in real time by the camera of the electronic device 100 in the second preview frame, the electronic device 100 100 A fifth operation for the option toggle control is detected. In response to the fifth operation, the electronic device 100 displays the photographing option interface. In this way, after a shooting option has been selected, the user can be provided with an entry for changing the shooting option, so that the user can change the shooting option to shoot video content.

Exemplarily, the fifth operation may be an operation (eg, a single click) on the mirror movement template switching control 362 in the embodiment shown in FIG. 4A. For specific content, reference may be made to the foregoing embodiments shown in FIGS. 4A to 4D , which will not be repeated here.

S2008, after the electronic device 100 detects the third operation for the second shooting option, the electronic device 100 displays a third shooting preview interface, the third shooting preview interface includes a third preview frame, and the third preview frame displays the camera of the electronic device 100 real-time captured images.

Exemplarily, the third shooting preview interface may be the shooting interface 410 shown in FIG. 4D above. For the shooting screen effects corresponding to different shooting options, reference may be made to the embodiments shown in FIGS. 7A to 7F , the embodiments shown in FIGS. 8A to 8G and the embodiments shown in FIGS.

S2009, the electronic device 100 starts to shoot the second video content.

S2010 , at the first moment after starting to shoot the second video content, the electronic device 100 displays, in a third preview frame, the third part of the picture captured in real time by the camera of the electronic device 100 .

S2011 , at a second moment after starting to shoot the second video content, the electronic device 100 displays, in a third preview frame, the fourth part of the picture captured by the camera of the electronic device 100 in real time. Among them, the first part, the second part, the third part and the fourth part are all different.

Wherein, for the display effects of the first part, the second part, the third part and the fourth part, you can refer to the above-mentioned embodiments shown in FIGS. 7A-7F , the embodiments shown in FIGS. 8A-8G and FIGS. 9A-9F The embodiments are not repeated here.

In this embodiment of the present application, before displaying the image captured by the camera of the electronic device 100 in real time, the electronic device 100 performs image stabilization processing on the image captured in real time by the camera of the electronic device 100 , and the image stabilization processing includes one of the following or Various: electronic image stabilization EIS processing, optical image stabilization processing, mechanical image stabilization processing. In this way, the shooting picture can be made smooth and excessive.

In this embodiment of the present application, before the electronic device 100 displays the first part of the picture captured in real time by the camera of the electronic device 100 in the second preview frame, the electronic device performs image super-resolution reconstruction on the first part. Before the electronic device displays the second part of the picture captured in real time by the camera of the electronic device in the second preview frame, the electronic device performs image super-resolution reconstruction on the second part. Before the electronic device displays the third part of the picture captured in real time by the camera of the electronic device in the third preview frame, the electronic device performs image super-resolution reconstruction on the third part. Before the electronic device displays the fourth part of the picture captured by the camera of the electronic device in real time in the third preview frame, the electronic device performs image super-resolution reconstruction on the fourth part. In this way, the shooting picture can be made clearer.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions described in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

An image processing method, comprising:

The electronic device displays a first shooting preview interface, the first shooting preview interface includes a first preview frame, and the first preview frame displays a picture captured in real time by a camera of the electronic device;

After the electronic device detects the first operation on the first shooting preview interface, the electronic device displays a shooting option interface, and the shooting option interface includes a first shooting option and a second shooting option;

After the electronic device detects the second operation for the first shooting option, the electronic device displays a second shooting preview interface, the second shooting preview interface includes a second preview frame, and the second preview frame displays The picture acquired by the camera of the electronic device in real time;

The electronic device starts to shoot the first video content;

At a first moment after starting to shoot the first video content, the electronic device displays, in the second preview frame, the first part of the picture captured in real time by the camera of the electronic device;

At a second moment after starting to shoot the first video content, the electronic device displays, in the second preview frame, the second part of the picture captured in real time by the camera of the electronic device;

The electronic device displays the shooting option interface;

After the electronic device detects the third operation for the second shooting option, the electronic device displays a third shooting preview interface, the third shooting preview interface includes a third preview frame, and the third preview frame displays The picture acquired by the camera of the electronic device in real time;

the electronic device starts to shoot the second video content;

At the first moment after starting to shoot the second video content, the electronic device displays, in the third preview frame, the third part of the picture captured in real time by the camera of the electronic device;

At a second moment after starting to shoot the second video content, the electronic device displays, in the third preview frame, the fourth part of the picture captured by the camera of the electronic device in real time, wherein the first part , the second part, the third part and the fourth part are all different.
The method according to claim 1, wherein before the electronic device starts to shoot the first video content, the method further comprises:

The electronic device detects a fourth operation for the second shooting preview interface;

The electronic device starts to shoot the first video content, which specifically includes:

In response to the fourth operation, the electronic device starts to capture the first video content.
The method according to claim 1, wherein the second shooting preview interface further comprises an option switching control; the electronic device displays in the second preview frame the data captured by the camera of the electronic device in real time. After the second portion of the picture, the method further includes:

the electronic device detects a fifth operation for the option switching control;

The electronic device displays the shooting option interface, which specifically includes:

In response to the fifth operation, the electronic device displays the photographing option interface.
The method according to claim 1 or 2, wherein after the electronic device displays the second part of the picture captured in real time by the camera of the electronic device in the second preview frame, the method further comprises: include:

The electronic device detects a sixth operation for the second shooting preview interface;

In response to the sixth operation, the electronic device saves the first part as a video picture at a first moment in the video file, and saves the second part as a video picture at a second moment in the video file.
The method according to claim 1 or 2, wherein the method further comprises:

The electronic device detects a seventh operation at a first moment after starting to shoot the first video content;

In response to the seventh operation, the electronic device saves the first part as a picture.
The method according to any one of claims 1-5, wherein the second shooting preview interface further comprises a first window; the method further comprises:

The electronic device displays, in the first window, a picture captured in real time by a camera of the electronic device;

At a first moment after starting to capture the first video content, the electronic device displays the bounding box of the first part in the first window;

At a second moment after starting to capture the first video content, the electronic device displays the bounding box of the second portion in the first window.
The method according to claims 1-5, wherein the second shooting preview interface further comprises a first window; the method further comprises:

At the first moment after starting to shoot the first video content, the electronic device displays the video frame at the first moment in the video sample corresponding to the first shooting option in the first window;

At a second moment after starting to shoot the first video content, the electronic device displays, in the first window, a video image at the second moment in the video sample corresponding to the first shooting option.
The method according to claim 6 or 7, wherein the display position of the first window and the second preview frame is any one of the following:

At least a partial area of the first window overlaps with the display area of the second preview frame; or,

The first window is displayed in a position outside the second preview frame in the second shooting preview interface; or,

The first window is displayed in the upper right area of the second preview frame; or,

The first window is displayed in the upper left area of the second preview frame.
The method according to any one of claims 6-8, wherein the first window includes a window closing control; the method further comprises:

The electronic device detects an eighth operation for the window closing control in the first window;

In response to the eighth operation, the electronic device closes and displays the first window.
The method according to claim 9, wherein after the electronic device closes and displays the first window, the method further comprises:

The electronic device displays a window opening control in the second shooting preview interface;

The electronic device detects a ninth operation for the window opening control;

In response to the ninth operation, the electronic device displays the first window on the second shooting preview interface.
The method according to any one of claims 1-10, wherein the second shooting preview interface further includes a mirror movement mode close control; the method further includes:

the electronic device detects a tenth operation acting on the mirror movement mode closing control;

In response to the tenth operation, the electronic device displays the first shooting preview interface.
The method according to any one of claims 1-6, wherein the second shooting preview interface further includes a second window, and the method further includes:

The electronic device displays the waveform diagram of the first music segment corresponding to the first shooting option in the second window;

Wherein, the audio information in the waveform chart at the first moment after starting to shoot the first video content is different from the audio information in the waveform chart at the second moment after starting to shoot the first video content, so the audio information in the waveform chart is different. The audio information includes one or more of the following: rhythm information, amplitude information, and style information.
The method according to claim 12, wherein after the electronic device starts to shoot the first video content, the method further comprises:

The electronic device plays the first piece of music.
The method according to claim 12 or 13, wherein the method further comprises:

When the electronic device saves the first part as a video frame at the first moment in the video file and saves the second part as a video frame at the second moment in the video file, the electronic device saves the The first piece of music is stored in the audio data of the video file.
The method according to claim 1, wherein the method further comprises:

Before displaying the picture captured by the camera of the electronic device in real time, the electronic device performs image stabilization processing on the image captured in real time by the camera of the electronic device, and the image stabilization process includes one or more of the following: electronic Image stabilization EIS processing, optical image stabilization processing, mechanical image stabilization processing.
The method according to claim 1, wherein the method further comprises:

Before the electronic device displays, in the second preview frame, the first part of the picture captured by the camera of the electronic device in real time, the electronic device performs image super-resolution reconstruction on the first part;

Before the electronic device displays, in the second preview frame, the second part of the picture captured by the camera of the electronic device in real time, the electronic device performs image super-resolution reconstruction on the second part;

Before the electronic device displays, in the third preview frame, the third part of the picture captured by the camera of the electronic device in real time, the electronic device performs image super-resolution reconstruction on the third part;

Before the electronic device displays the fourth part of the picture captured in real time by the camera of the electronic device in the third preview frame, the electronic device performs image super-resolution reconstruction on the fourth part.
The method according to claim 1, wherein the camera of the electronic device comprises a wide-angle camera and a non-wide-angle camera;

The first preview frame displays the picture captured in real time by the camera of the electronic device, which specifically includes:

The first preview frame displays a picture acquired by the non-wide-angle camera in real time;

The second preview frame displays the picture captured in real time by the camera of the electronic device, which specifically includes:

The second preview frame displays the picture acquired by the wide-angle camera in real time;

The third preview frame displays the picture captured by the camera of the electronic device in real time, which specifically includes:

The third preview frame displays a picture acquired by the wide-angle camera in real time.
An electronic device, characterized by comprising: a touch screen, a camera, one or more processors, and one or more memories; the one or more processors and the touch screen, the camera, the the one or more memories are coupled for storing computer program code, the computer program code comprising computer instructions that, when executed by the one or more processors, cause the electronic device The method of any of claims 1-17 is performed.
A computer-readable storage medium comprising instructions, wherein when the instructions are executed on an electronic device, the electronic device is caused to perform the method of any one of claims 1-17.