WO2022068511A1

WO2022068511A1 - Video generation method and electronic device

Info

Publication number: WO2022068511A1
Application number: PCT/CN2021/116047
Authority: WO
Inventors: 张韵叠; 苏达; 陈绍君; 胡靓; 徐迎庆; 徐千尧; 郭子淳; 高家思; 周雪怡
Original assignee: 华为技术有限公司; 清华大学
Priority date: 2020-09-29
Filing date: 2021-09-01
Publication date: 2022-04-07
Also published as: CN114363527A; CN114363527B

Abstract

Embodiments of the present application provide a video generation method and an electronic device. The method comprises: The electronic device displays a first interface of a first application. After receiving a first operation performed on a first widget, the electronic device determines an arrangement sequence of a first material, a second material and a third material to be a first sequence, which is different from a third sequence; and generates a first video using the first material, the second material and the third material according to the first sequence. After receiving a second operation performed on a second widget, the electronic device determines an arrangement sequence of the first material, the second material and the third material to be a second sequence, which is different from the third sequence; and generates a second video using the first material, the second material and the third material according to the second sequence. The third sequence is a time sequence which the first material, the second material and the third material are stored in the electronic device. The generated video has coherent pictures and high quality.

Description

Video generation method and electronic device

This application claims the priority of the Chinese patent application with the application number 202011057180.9 and the application title "Video Generation Method and Electronic Device" filed with the China Patent Office on September 29, 2020, the entire contents of which are incorporated into this application by reference.

technical field

The embodiments of the present application relate to the field of electronic technologies, and in particular, to a video generation method and an electronic device.

Background technique

With the popular trend of short videos, users' demands to generate videos quickly on electronic devices such as mobile phones are increasing day by day. Currently, videos generated by electronic devices have poor line-of-sight consistency and low quality, which cannot meet the high demands of users for video imagery and film and television. Therefore, there is an urgent need for a method capable of generating a video with a coherent line of sight and high quality.

SUMMARY OF THE INVENTION

The present application provides a video generation method and electronic device, so as to conveniently and quickly generate a video, and the video has a coherent line of sight and high quality, which enhances the video's sense of footage and cinema, and improves the user's experience.

In a first aspect, the present application provides a video generation method, including: an electronic device displays a first interface of a first application, and the first interface includes a first control and a second control; and the electronic device receives a first control that acts on the first control. After the first operation of the The third material generates the first video; after receiving the second operation acting on the second control, the electronic device determines that the arrangement order of the first material, the second material and the third material is the second order, and the second order is the same as the second order. The three sequences are different; and according to the second sequence, a second video is generated from the first material, the second material and the third material. The first material, the second material and the third material are different image materials stored in the electronic device, and the third order is the time sequence in which the first material, the second material and the third material are stored in the electronic device.

With the method provided in the first aspect, by identifying the scene type of the material, matching the appropriate video template, adjusting the arrangement order of the material based on the scene type that has been set for each segment in the video template, and combining every scene type in the video template The camera movement, speed and transition that have been set for each clip can automatically generate a video with a coherent line of sight and high quality, without relying on the user's manual editing, which enhances the camera and cinematic sense of the video, and improves the user experience.

In a possible design, the first video is divided into a plurality of segments with the beat point of the music as a dividing line; the first material, the second material and the third material appear at least once in the first video, and in the first video The materials appearing in any two adjacent segments of the second video are different; the first material, the second material and the third material appear at least once in the second video, and the materials appearing in any two adjacent segments of the second video are different . This ensures that the resulting video has a professional camera and cinematic feel.

In a possible design, the method further includes: displaying the second interface of the first application on the electronic device; after the electronic device receives the third operation acting on the second interface, displaying the first material, the second material and the first material Three materials generate the first video. Therefore, the electronic device can generate a video with a coherent line of sight and high quality based on the material selected by the user.

In a possible design, the method further includes: the electronic device determines from the first material, the second material, the third material and the fourth material to generate the first video from the first material, the second material and the third material; The fourth material is an image material stored in the electronic device that is different from the first material, the second material and the third material. Thus, the electronic device can automatically generate a video based on the stored material to meet the user's timely needs.

In a possible design, the first interface further includes a third control; the method further includes: after the electronic device receives the fourth operation acting on the third control, displaying a third interface, where the third interface includes: configuring options of information, the configuration information includes: at least one parameter of duration, filter, frame, material or title; after the electronic device receives the fifth operation acting on the options of the configuration information, in the first order, based on the configuration information , and generate a third video from the first material, the second material, and the third material. Thus, the types of videos are enriched, and the user's requirements for adjusting various parameters of the videos are satisfied.

In a possible design, the first interface further includes a fourth control; the method further includes: after the electronic device generates the first video, in response to a fourth operation acting on the fourth control, the electronic device saves the first video. Therefore, it is convenient for the user to watch and edit the generated video subsequently.

In a possible design, the method specifically includes: the electronic device determines the scene category type corresponding to the first material, the scene category type corresponding to the second material, and the scene category type corresponding to the third material; the electronic device is based on the scene category type corresponding to the first material. The scene type, the scene type corresponding to the second material, the scene type corresponding to the third material, and the scene type set for each segment in the first video template, determine the scene type that matches the scene type corresponding to the first segment. material, the first segment is any segment in the first video template; the arrangement order of the materials corresponding to all the segments in the first video template is the first order; the electronic device is based on the scene type corresponding to the first material, the first The scene type corresponding to the second material, the scene type corresponding to the third material, and the scene type set for each clip in the second video template, determine the material that matches the scene type corresponding to the second clip, the second clip is any fragment in the second video template; and the arrangement order of the materials corresponding to all the fragments in the second video template is the second order; wherein, the first video template is different from the second video template, and the Each segment corresponds to each segment in the second video template, and each segment in the second video corresponds to each segment in the second video template.

In a possible design, the method further includes: the electronic device converts the first material, the second material into and the third material to generate a first video; the electronic device converts the first material, the second material and the third material into The material generates a second video.

In a possible design, when the first material is a picture material, the method specifically includes: the scene type corresponding to the first material in the electronic device is the same as the scene type corresponding to the first segment, or, the scene corresponding to the first material is of the same type. When the scene type is adjacent to the sequence of the scene type corresponding to the first segment according to the preset rule, the first material is determined as the material matching the scene type corresponding to the first segment; the electronic device is in the scene corresponding to the first material. The category type is the same as the scene category type corresponding to the second clip, or, when the scene category type corresponding to the first material is adjacent to the scene category type corresponding to the second clip according to the preset rules, the first material is determined to be the same as the scene category type corresponding to the second clip. The scene type corresponding to the two clips matches the material.

In a possible design, when the first material is a video material, the method specifically includes: the scene type of the electronic device corresponding to the fourth material is the same as the scene type corresponding to the first clip or the scene type corresponding to the fourth material. The type is adjacent to the sequence of the scene types corresponding to the first segment according to the preset rules, and when the duration of the fourth material is equal to the duration of the first segment, the fourth material is intercepted from the first material, and the fourth material is determined as The material that matches the scene type corresponding to the first clip; the scene type corresponding to the fourth material in the electronic device is the same as the scene type corresponding to the second clip, or the scene type corresponding to the fourth material is the same as the second type according to the preset rules. When the sequence of the scene types corresponding to the clips is adjacent, and the duration of the fourth material is equal to the duration of the second clip, the fourth material is intercepted from the second clip, and the fourth material is determined as the scene corresponding to the second clip A material whose type matches; wherein, the fourth material is part or all of the first material.

In a possible design, according to the ordering of preset rules, the types of scenes include: close-up, medium-range, and far-scenes; The distant view, the type of scene adjacent to the distant view is the close view.

In a possible design, the first application is a gallery application of the electronic device.

In a second aspect, the present application provides an electronic device, including: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory to enable the electronic device to execute the first aspect and any possibility of the first aspect The video generation method in the design.

In a third aspect, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen and a sensor; the chip system includes: a processor; when the processor executes computer instructions stored in the memory, the electronic device executes the first A video generation method in one aspect and any possible design of the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by the processor to enable the electronic device to realize the video generation in the first aspect and any possible design of the first aspect. method.

In a fifth aspect, the present application provides a computer program product, comprising: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and at least one processor Executing the execution instruction enables the electronic device to implement the first aspect and the video generation method in any possible design of the first aspect.

Description of drawings

FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

2 is a block diagram of a software structure of an electronic device provided by an embodiment of the present application;

3A-3T are schematic diagrams of human-computer interaction interfaces provided by an embodiment of the present application;

4A-4J are schematic diagrams of the effect of using a mirror for a picture material provided by an embodiment of the present application;

5 is a schematic diagram of the effect of using different speeds for a picture material provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of the effect of using a transition for a picture material provided by an embodiment of the present application;

7 is a schematic diagram of a scene type of a character type material provided by an embodiment of the present application;

8A-8E are schematic diagrams of playback of a video generated based on a material provided by an embodiment of the application;

FIG. 9 is a schematic diagram of a video generation method provided by an embodiment of the present application.

Detailed ways

In the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And/or", which describes the association relationship of the associated objects, indicates that there can be three kinds of relationships, for example, A and/or B, which can indicate: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects are an "or" relationship. "At least one item(s) below" or similar expressions thereof refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (a) of a single a, a single b or a single c can mean: a single a, a single b, a single c, a combination of a and b, a combination of a and c, a combination of b and c, or a combination of a, b and c, where a, b, c can be single or multiple. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed to indicate or imply relative importance.

Please refer to FIG. 1 , which is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 1 , the electronic device 100 may include a processor 110 , an external memory interface 120 , an internal memory 121 , a universal serial bus (USB) interface 130 , a charging management module 140 , a power management module 141 , and a battery 142 , Antenna 1, Antenna 2, Mobile Communication Module 150, Wireless Communication Module 160, Audio Module 170, Speaker 170A, Receiver 170B, Microphone 170C, Headphone Interface 170D, Sensor Module 180, Key 190, Motor 191, Indicator 192, Camera 193 , a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structures illustrated in this application do not constitute a specific limitation on the electronic device 100 . In other embodiments, the electronic device 100 may include more or fewer components than shown, or some components may be combined, or some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may contain multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

The PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the wireless communication module 160 . For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in this application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

The electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The earphone jack 170D is used to connect wired earphones. The earphone interface 170D can be the USB interface 130, or can be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenarios.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance. The electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light to the outside through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 at a different location than the display screen 194.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as call and data communication. In some embodiments, the electronic device 100 employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 . Wherein, the embodiment of the present application does not limit the type of the operating system of the electronic device. For example, Android system, Linux system, Windows system, iOS system, Hongmeng operating system (harmony operating system, Hongmeng OS), etc.

Please refer to FIG. 2 , which is a block diagram of a software structure of an electronic device according to an embodiment of the present application. As shown in Figure 2, the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, they are an application layer (APP), an application framework layer (APP framework), an Android runtime (Android runtime) and a system library (libraries), And the kernel layer (kernel).

The application layer can include a series of application packages.

As shown in Figure 2, the application package can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, game, chat, shopping, travel, instant messaging (such as SMS), smart home, Device control and other applications (application, APP).

Among them, smart home applications can be used to control or manage home devices with networking capabilities. For example, household equipment may include lights, televisions, and air conditioners. For another example, household equipment may also include anti-theft door locks, speakers, sweeping robots, sockets, body fat scales, desk lamps, air purifiers, refrigerators, washing machines, water heaters, microwave ovens, rice cookers, curtains, fans, TVs, set-top boxes, windows, etc.

In addition, the application package may also include: home screen (ie desktop), negative screen, control center, notification center and other applications.

The negative one screen, also known as "-1 screen", refers to sliding the screen to the right on the main screen of the electronic device until it slides to the user interface (UI) of the leftmost split screen. For example, the negative screen can be used to place some quick service functions and notification messages, such as global search, quick entry to a certain page of the application (payment code, WeChat, etc.), instant information and reminders (courier information, expense information, commuting road conditions) , taxi travel information, schedule information, etc.) and attention to trends (football stands, basketball stands, stock information, etc.). The control center is the slide-up message notification bar of the electronic device, that is, the user interface displayed by the electronic device when the user starts to perform the slide-up operation at the bottom of the electronic device. The notification center is a drop-down message notification bar of the electronic device, that is, a user interface displayed by the electronic device when the user starts to perform downward operations on the top of the electronic device.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

Window manager (window manager) is used to manage window programs, such as managing window state, properties, view addition, deletion, update, window order, message collection and processing, etc. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc. And, the window manager is the entry for the outside world to access the window.

Content providers are used to store and retrieve data and make this data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager (resource manager) provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

The Android runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of the Android system.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: surface manager (surface manager), media library (media library), 3D graphics processing library (eg: OpenGLES), 2D graphics engine (eg: SGL), etc.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

The software and hardware workflows of the electronic device 100 are exemplarily described below in conjunction with a scenario in which a smart speaker is used to play sound.

When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and the control corresponding to the click operation is the control of the smart speaker icon, for example, the smart speaker application calls the interface of the application framework layer to start the smart speaker application, and then starts the audio driver by calling the kernel layer. , the audio electrical signal is converted into a sound signal through the speaker 170A.

The technical solutions involved in the following embodiments can all be implemented in the electronic device 100 having the above-mentioned hardware architecture and software architecture.

Embodiments of the present application provide a method and electronic device for generating a video. The electronic device identifies the scene type of the material, matches a suitable video template, adjusts the arrangement order of the material based on the scene type set in the video template, and combines The camera movement, speed and transition set in the video template can automatically generate video, so that the generated video has a coherent line of sight and high quality, which enhances the sense of footage and film of the video, and improves the user experience. Adjust the video duration, filters, frame and other parameters to meet the actual user needs and enrich the types of videos.

The electronic device may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR)/virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer) , UMPC), netbooks, personal digital assistants (personal digital assistants, PDAs), smart TVs, smart screens, high-definition TVs, 4K TVs, smart speakers, smart projectors, etc. The embodiments of this application do not impose any restrictions on the specific types of electronic devices .

Hereinafter, some terms involved in the embodiments of the present application will be explained, so as to facilitate the understanding of those skilled in the art.

1. The material can be understood as the picture material or video material stored in the electronic device. It should be noted that the picture material mentioned in the embodiments of the present application has the same meaning as the photo material. The picture material may be photographed by an electronic device, downloaded by the electronic device from a server, or received by the electronic device from other electronic devices, which is not limited in this embodiment of the present application.

2. Scene recognition can be understood as the difference in the size of the range presented by the subject in the subject due to the difference in the distance between the subject and the subject. The photographing body may be an electronic device, or may be a device in communication connection with the electronic device, which is not limited in this embodiment of the present application.

In this embodiment of the present application, the division of scene types may include various implementation manners. It should be noted that the scene types mentioned in the embodiments of the present application refer to scene types.

In some embodiments, the classification of scene types can be divided into three types, from near to far, they are close-range, medium-range, and long-range. For example, close shot refers to above the human chest, medium shot refers to above the human thigh, and long shot refers to situations other than close and medium shots.

In other embodiments, the classification of scene types may be five, from near to far, they are close-up, close-up, medium-range, panorama, and long-range. For example, close-up refers to the body above the shoulders, close-up refers to the body above the chest, medium shot refers to the body above the knee, panoramic refers to the entire human body and part of the surrounding environment, and long-range refers to the environment where the subject is located.

In this embodiment of the present application, the scene type corresponding to the video material may be regarded as a collection of the respective scene types of a plurality of picture materials. Usually, the electronic device can record the start time and duration, or the start time and the end time, or the start time, the duration and the end time of each scene type. In addition, electronic equipment adopts technologies such as face recognition, semantic recognition, salient feature recognition, semantic segmentation, etc., which can classify the scene type of the material, that is, determine the scene type of the picture material.

In the following, a specific implementation manner of determining the scene type of any material by an electronic device is described with reference to the embodiments.

A, face close-up and face close-up

The electronic device determines the face recognition frame of any material based on the face recognition technology.

When the area of the face recognition frame is larger than the threshold value A1, the electronic device determines that the scene of the material is a close-up of the face.

When the area of the face recognition frame is larger than the threshold value A2 and smaller than the threshold value A1, the electronic device determines that the scene of the material is a close-up scene of a face.

The specific values of the threshold value A1 and the threshold value A2 may be set according to factors such as empirical values and face recognition technical means.

B. Character close-up and character close-up

Electronic devices can perform face recognition on any material based on face recognition technology.

When the recognition result indicates that there is no face, and the semantic style of the person (such as the profile/back of the person in the material) exists, the electronic device can use the area of the head*threshold to obtain the person recognition frame.

When the area of the person identification frame is greater than the threshold B1, the electronic device determines that the scene of the material is a close-up of a person.

When the area of the person identification frame is larger than the threshold B2 and smaller than the threshold B1, the electronic device determines that the scene of the material is a close-up scene of a person.

The specific values of the threshold value B1 and the threshold value B2 may be set according to factors such as empirical values.

C, food close-up and food close-up

The electronic device determines the semantic recognition result and the salient feature recognition result of any material based on the speech segmentation recognition technology and the salient feature recognition technology.

When the semantic recognition result indicates that the area of the food is larger than the threshold value C1, the saliency feature result indicates that the area of significance is larger than the threshold value C2, and the area of the food coincides with the area of salience, the electronic device determines that the scene of the material is a close-up of the food.

When the semantic recognition result indicates that the area of the food is larger than the threshold C1, and the saliency feature result indicates that the saliency area is smaller than the threshold C2, the electronic device determines that the scene of the material is a close-up scene of the food.

The specific values of the threshold C1 and the threshold C2 can be set according to factors such as empirical values.

D, non-person large aperture close-up

When it is detected that any material has a photo with a large aperture mode or that a large defocused image exists in the material, the electronic device determines that the scene of the material is a close-up view of a non-person with a large aperture.

E. Close-up of prominent flowers and close-up of prominent pets

When the semantic recognition result indicates that the area of the flower is larger than the threshold D1, the saliency feature result indicates that the area of salience is larger than the threshold D2, and the area of the flower coincides with the area of saliency, the electronic device judges the scene of the material as a close-up view of a saliency flower .

The specific values of the threshold D1 and the threshold D2 may be set according to factors such as empirical values.

When the semantic recognition result indicates that the area of the pet is greater than the threshold E1, the saliency feature result indicates that the area of salience is greater than the threshold E2, and the area of the pet coincides with the area of salience, the electronic device determines that the scene of the material is a close-up view of a salient pet .

The specific values of the threshold E1 and the threshold E2 may be set according to factors such as empirical values.

F. Character in the middle shot

When the recognition result indicates that there is no face or decomposition result that matches the close-up of the character, or, there is no complete character completely entering the frame (as follows: the torso leaves the edge of the frame and the face or head is smaller than a threshold), the electronic device determines the material. The scene is not the character medium scene.

G. Significant prospect

The electronic device determines the salient feature recognition result of any material based on the salient feature recognition technology.

When a significant result exists and the significant area indicates that the significant area is smaller than the threshold value F (for example, the material is a picture material of a camel in the desert, where the camel is a significant result), the electronic device determines the scene of the material as a significant distant view .

H. Landscape

The electronic device determines the screen segmentation result of any material based on the semantic segmentation technology.

When the screen segmentation result indicates that the area of the material is greater than the threshold G as the preset target, the electronic device determines that the scene of the material is a landscape and distant scene.

The threshold value G may be set to be greater than or equal to 90%, and the specific value of the threshold value G is not limited in this embodiment of the present application. The preset targets can be landscape features, such as sea, sky, mountain peaks, etc.

I. Other

When the electronic device cannot identify the scene of the material based on the aforementioned technology, the electronic device determines that the scene of the material is a medium scene.

Among them, A to E are the close-up range, and the deterministic order is close-up of person=close-up of human face>close-up of food>close-up of person=close-up of human face>close-up of non-person with large aperture>close-up of food=close-up of prominent flowers and close-up of prominent pets, G and H are long shots, the deterministic order is H>G, and F and I are medium shots.

3. Mirror movement is also called sports lens, which mainly refers to the movement of the lens itself. In this embodiment of the present application, the movement of the mirror is related to the type of the material, that is, the movement of the mirror corresponding to the picture material and the movement of the mirror corresponding to the video material may be the same or different.

4. Transitions can be understood as transitions or transitions between paragraphs and paragraphs, scenes and scenes. Among them, each paragraph (the smallest unit that constitutes a video is a shot, a sequence of shots formed by connecting shots together) has a single, relatively complete meaning, such as expressing an action process, expressing a correlation, expressing a meaning and so on. It is a complete narrative level in a video, just like an act in a drama or a chapter in a novel. The paragraphs are connected together to form a complete video. Therefore, paragraph is the most basic structural form of video, and the structural level of video content is expressed through paragraphs.

5. The type of scene, movement, speed and transition set in the video template

A video template can be understood as the theme or style of the video. The types of video templates may include but are not limited to: travel, parent-child, party, sports, food, scenery, retro, city, night, humanities, etc.

The parameters in any video template may include, but are not limited to: scene type, camera movement, speed, transition, and the like. Generally, different video templates have different at least one parameter in scene type, camera movement, speed and transition.

In this embodiment of the present application, the electronic device has a function of generating a video from stored materials, so that one or more picture materials and/or video materials in the electronic device generate a video. In addition, the electronic device provides the user with a variety of entry ways to generate video, which facilitates the user to generate the video in a timely and rapid manner, and improves the convenience of the user.

In the following, taking the gallery application of the electronic device as an example for generating a video, the method for generating a video from the stored material by the electronic device according to the embodiment of the present application will be described in detail in combination with the first, second and third modes. It should be noted that the embodiments of the present application include, but are not limited to, the gallery application as an entry method for generating videos, and include but are not limited to the above three methods.

method one

Please refer to FIGS. 3A-3F . FIGS. 3A-3F are schematic diagrams of human-computer interaction interfaces provided by an embodiment of the present application. For convenience of description, in FIGS. 3A-3F , the electronic device is a mobile phone as an example for exemplary illustration.

The handset may display a user interface 11 as exemplarily shown in Figure 3A. Wherein, the user interface 11 may be the home screen of the desktop, and the user interface 11 may include but not limited to: a status bar, a navigation bar, a calendar indicator, a weather indicator, and multiple application icons. The application icon may include: the icon 301 of the gallery application, and the application icon may also include: such as the icon of the Huawei video application, the icon of the music application, the icon of the mobile phone manager application, the icon of the setting application, the icon of the Huawei mall application, the icon of the smart life Icons of applications, icons of sports health applications, icons of calling applications, icons of instant messaging applications, icons of browser applications, icons of camera applications, etc.

After detecting that the user performs an operation of opening the gallery application in the user interface 11 shown in FIG. 3A (for example, clicking the icon 301 of the gallery application), the mobile phone can display the user interface 12 exemplarily shown in FIG. 3B . The user interface 12 is used for Displays the page corresponding to the album category in the Gallery app.

Among them, the user interface 12 may include: a control 3021, which is used to enter a display interface containing all picture materials and/or video materials in the mobile phone, and a control 3023, which is used to enter the photo album in the gallery application The display interface corresponding to the category.

In this embodiment of the present application, the specific implementation manner of the user interface 12 may include various manners. For ease of illustration, in FIG. 3B, the user interface 12 is divided into two groups.

The first group consists of two parts. The title of the first group is illustrated by taking the text "album" as an example in FIG. 3B .

In the first part, a search box is used to provide the user with a way to search for picture material and/or video material according to keywords such as photos, people, places, etc.

The second part includes controls 3021, and controls for entering a display interface containing only video material.

The second group displays pictures obtained by means of screenshots or an application. The title of the third part is illustrated in FIG. 3B by using the text “other albums (3)” and a rounded rectangular frame as an example.

In addition, the user interface 12 further includes: a control 3022 , a control 3024 and a control 3025 . The control 3022 is used to enter the display interface corresponding to the photo category in the gallery application. The control 3024 is used to enter the display interface corresponding to the moment category in the gallery application. The control 3025 is used to enter the display interface corresponding to the discovery category in the gallery application.

In addition, the user interface 12 may further include: controls for implementing functions such as deleting an existing group in the user interface 12, changing the name of an existing group, and controls for adding a new group in the user interface 12 .

After detecting that the user performs an operation such as clicking on the control 3021 in the user interface 12 shown in FIG. 3B, the mobile phone can display the user interface 13 exemplarily shown in FIG. 3C, and the user interface 13 is all the picture materials and/or the mobile phone. Display interface for video footage. Among them, the embodiments of the present application display the number of picture materials in the user interface 13, the display area of picture materials, the display position of picture materials, the display content of video materials, the display quantity of video materials, the display area of video materials, the display area of video materials, the The parameters such as the display position and the order of each type of material are not limited.

For the convenience of description, in FIG. 3C , in the chronological order from the current moment to the farthest, the user interface 13 displays: video material 3031 , picture material 3032 , picture material 3033 , video material 3034 , picture material 3035 , picture material 3036 , picture material 3037 and video material 3038. Wherein, for any video material, the electronic device may select an image displayed in any frame in the video material as the screen displayed by the electronic device to the user. Therefore, in FIG. 3C , the images displayed by the video material 3031 , the video material 3034 and the video material 3038 are the images displayed by any frame in the respective video materials.

After detecting that the user performs an operation (such as a long-press operation) for selecting a picture material and/or a video material in the user interface 13 shown in FIG. 3C, the mobile phone may display the user interface 14 exemplarily shown in FIG. The interface 14 is used to display a display interface where the user selects the picture material and/or video material used to generate the video.

In the embodiment of the present application, the specific implementation manner of the user interface 14 may include various. For the convenience of description, in FIG. 3D , the user interface 14 includes the user interface 13 and an editing interface overlaid on the user interface 13 .

For the picture material and/or video material not selected by the user (in Figure 3D, in addition to the video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038, For example, other picture materials/video materials other than the ones shown below are shown as an example), in the editing interface, a control can be displayed in the upper left corner of each picture material/video material for magnifying and displaying the picture material/video material (Figure 3D Two diagonal and oppositely pointing arrows are used as an example for illustration), and a control for selecting the picture material/video material is displayed in the lower right corner of each picture material/video material (a circle is used in Fig. 3D The corner rectangle box is used as an example to illustrate).

For the picture material and/or video material selected by the user (in Figure 3D, video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038 are used as examples For illustration), in the editing interface, a control for magnifying and displaying the picture material/video material can be displayed in the upper left corner of each picture material/video material (two diagonal and opposite arrows are used in Fig. 3D An example is shown for illustration), and a control for selecting the image material/video material is displayed in the lower right corner of each picture material/video material (a rounded rectangular box is used as an example for illustration in FIG. 3D).

Moreover, the editing interface may include: a control 304, where the control 304 is used to create the picture material and/or video material selected by the user. In addition, the editing interface may further include: controls for performing operations such as sharing, selecting all, deleting, and more on the picture material and/or video material selected by the user, which is not limited in this embodiment of the present application.

After detecting that the user performs an operation such as clicking on the control 304 in the user interface 14 shown in FIG. 3D, the mobile phone can display the window 305 exemplarily shown in FIG. 3E on the user interface 14 (FIG. "Puzzle" and a rounded rectangle as an example).

Wherein, when the user selects a picture material, or a video material, or a picture material and a video material, after the user enters the text "video" in the window 305, such as clicking and other operations, the mobile phone can display a screen for performing operations on the new video. Edited user interface.

Wherein, when the user selects a picture material, after the user enters the word "puzzle" in the window 305, such as clicking and other operations, the mobile phone can display a user interface for editing a new picture.

Wherein, when the user selects a video material, or a picture material and a video material, after the user enters the text "Puzzle" in the window 305, such as clicking and other operations, the mobile phone cannot display a user interface for editing a new picture , the text "Puzzle does not support video" can be displayed to prompt the user to deselect the video material.

After the mobile phone detects that the user performs an operation such as clicking the text "video" in the window 305 shown in FIG. 3E, the mobile phone selects the video material 3031, the picture material 3032, the picture material 3033, the video material 3034, the picture material 3035, and the picture based on the user's selection. Material 3036, picture material 3037, and video material 3038, the type of the video template can be determined as parent-child type, so based on the parent-child type video template, the video material 3031, picture material 3032, picture material 3033, video material 3034, picture material selected by the user The material 3035, the picture material 3036, the picture material 3037 and the video material 3038 generate a video and can display the user interface 15 exemplarily shown in FIG. 3F for displaying the video generated by the mobile phone.

The segments in the generated video correspond to segments in the parent-child video template. Video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037, and video material 3038 appear at least once in the generated video, and any two adjacent to each other in the generated video Clips do not place the same footage.

To sum up, the electronic device can automatically generate a video based on the picture elements and/or video material selected by the user in the gallery application. In addition, the user interface 15 is used to display controls for editing the generated video.

The user interface 15 may include: a preview area 306 , a progress bar 307 , a control 3081 , a control 3082 , a control 3083 , a control 3084 , a control 3085 , a control 30811 , a control 30812 , a control 30813 , a control 30814 , and a control 309 .

The preview area 306 is used to display the generated video, which is convenient for the user to watch and adjust the video.

The progress bar 307 is used to represent the duration of the video under any video template (FIG. 3F uses “00:00” to exemplarily represent the start time of the video, “00:32” to exemplarily represent the end time of the video, and a sliding bar exemplarily represents the progress of the video).

Control 3081, used to provide different types of video templates. The control 30811 is used to represent the parent-child type of video template (Figure 3F uses the text "parent-child" and a bold rounded rectangle to exemplarily represent the parent-child type of video template), and the control 30812 is used to represent the travel type of video template (Figure 3F). 3F uses the text "travel" and a normally displayed rounded rectangular box to exemplarily represent a video template of travel type), and control 30813 is used to represent a video template of food type (Figure 3F uses the text "food" and a normal display of rounded corners) The rectangular box exemplarily represents the video template of gourmet type), and the control 30814 is used to represent the video template of sports type (Fig. 3F uses the word "sports" and a normally displayed rounded rectangular box exemplarily represents the video template of sports type). Therefore, when it is recognized that the material matches a certain type of video body template, the electronic device can also provide the user with generating other types of video templates other than this type, which is beneficial to meet various needs of the user.

The control 3082 is used for functions such as editing the frame of the video, changing the duration of the video, adding new pictures and/or videos in the video, and deleting pictures and/or videos in the video. Therefore, a video of a corresponding length and/or a corresponding material is generated based on user requirements, taking into account the flexibility of video generation.

Control 3083, used to change the music matched by the video template.

Control 3084, used to change the filter of the video.

The control 3085 is used to add text in the video, such as adding text at the beginning and end, etc.

The control 309 is used to save the generated video for convenient use or viewing of the saved video.

Based on the above description, the electronic device may display the generated video to the user through the preview area 306 .

In addition, since the type of the video template determined by the electronic device is the parent-child type, the electronic device displays the rounded rectangular box in the control 3081 in bold, which is convenient and quick to inform the user.

And, based on other controls in the user interface 15, the user can perform operations such as selecting the type of video template, adjusting the frame of the video, adjusting the duration of the video, adding new picture material and/or video material to the video, selecting music that matches the video , select the filter of the video, add text in the video, etc., so that the electronic device can determine the video template that meets the user's wishes and generate the corresponding video.

For example, after detecting that the user performs an operation such as clicking on the control 3081 in the user interface 15 shown in FIG. 3F, the mobile phone can display the user interface 15 exemplarily shown in FIG. 3F, so that the user can click the control 30811, the control 30812 and the control Choose a video template from 30813.

For another example, after detecting that the user performs an operation such as clicking on the control 3082 in the user interface 15 shown in FIG. 3F, the mobile phone can display the user interface 21 exemplarily shown in FIG. Factors such as frame size, duration, and materials included in playback.

The user interface 21 may include: a video playback area 3171 , a control 3172 , a control 3173 , a control 3174 , a control 3175 , a material playback area 3176 , and a control 3177 . Among them, the video playing area 3171 is used to display the effect played by the video to be generated. The control 3172 is used to enter the user interface for changing the frame of the video, where the frame of the video can be 16:9, 1:1, or 9:16, etc. Control 3173 is used to enter the user interface for changing the duration of the video. Control 3174 is used to enter the user interface for adding new material to the video. Control 3175 is used to access material that already exists in the video. The material playback area 3176 is used to display the playback effect of each material in the video. Control 3177 is used to exit user interface 21 .

For another example, after detecting that the user performs an operation such as clicking on the control 3083 in the user interface 15 shown in FIG. 3F, the mobile phone can display the user interface 22 exemplarily shown in FIG. 3H, and the user interface 22 is used to display the corresponding editing video. music.

The user interface 22 may include: a video playing area 3181 , a progress bar 3182 , a control 3183 , a control 3184 , and a control 3185 . Among them, the video play area 3181 is used to display the effect played by the video to be generated. The progress bar 3182 is used to display or change the playback progress of the video to be generated. The control 3183 is used to display various types of video templates, such as the text "parent-child", "travel", "food", "sports" and other types. The control 3184 is used to display the corresponding music under a certain type of video template, for example, "Song 1", "Song 2", and "Song 3" are displayed. Control 3185 is used to exit user interface 22 .

For another example, after detecting that the user performs an operation such as clicking on the control 3084 in the user interface 15 shown in FIG. 3F, the mobile phone can display the user interface 23 exemplarily shown in FIG. mirror. For example, in Figure 3H, the text "Parent-Child" is displayed in bold and there is a check mark in the display column corresponding to the text "Song 1" to indicate that the mobile phone currently selects a parent-child video template, and the music corresponding to the video template is Song 1 . It should be noted that the corresponding rounded rectangular boxes respectively located before the words "song 1", "song 2", and "song 3" are used to display the images of the corresponding songs. The specific display content of the image is not limited in this embodiment of the present application. For convenience of description, the embodiments of the present application take filling white as an example for illustration.

The user interface 23 may include: a video playing area 3191 , a progress bar 3192 , a control 3183 , a control 3184 , and a control 3185 . Among them, the video play area 3191 is used to display the effect played by the video to be generated. The progress bar 3192 is used to display or change the playback progress of the video to be generated. The control 3183 is used to display each filter, such as the text "Filter 1", "Filter 2", "Filter 3", "Filter 4", "Filter 5" and other types. Among them, different filters, the video has different display effects, such as softening, black and white, color deepening and other effects. Control 3194 is used to exit user interface 23 . For example, in FIG. 3I , the text “Filter 1” displayed in bold may indicate that the filter of the video currently selected by the mobile phone is Filter 1 .

For another example, after detecting that the user performs an operation such as clicking on the control 3085 in the user interface 15 shown in FIG. 3F, the mobile phone can display the user interface 24 exemplarily shown in FIG. mirror.

The user interface 24 may include: a video playing area 3201 , a control 3202 , a control 3203 , a control 3204 , and a control 3185 . Among them, the video play area 3191 is used to display the effect played by the video to be generated. Control 3202 is used to select to add a title to the intro or end credit. The control 3193 is used to display each title, such as the text "Title 1", "Title 2", "Title 3", "Title 4", "Title 5" and so on. For any two different titles (eg, title 1 and title 2), if the content of title 1 and title 2 can be the same, then title 1 and title 2 can be displayed on any screen of the video with different playback effects. The playback effect can be understood as the effect formed by changing parameters such as font, thickness, and color of the text in the title. For example, Heading 1 may be the text "Hours of the Weekend" and Heading 1 is in italics. Title 2 is the text "Weekend Hours", and Title 1 is in Song Dynasty. If the contents of Title 1 and Title 2 are different, Title 1 and Title 2 may be displayed on any screen of the video with the same or different playback effects. For example, Title 1 may be the text "Hours of the Weekend." Title 2 is the text "Good day". Control 3194 is used to exit user interface 24 . For example, in FIG. 3J, the bold display of the words "title 1" and "title 1" may indicate that the mobile phone currently chooses to add title 1 to the title 1 of the video.

In conclusion, the electronic device can provide the user with the function of manually editing the generated video, which is convenient for the user to configure parameters such as the duration, frame, video template, included materials and filters of the video based on their own wishes, thus enriching the style of the video.

In addition, the mobile phone can save the video after detecting that the user performs an operation such as clicking the control 309 in the user interface 15 shown in FIG. 3F .

Method 2

Please refer to FIG. 3A-FIG. 3B, FIG. 3K-FIG. 3N, and FIG. 3F. FIGS. 3K-FIG. 3N are schematic diagrams of human-computer interaction interfaces provided by an embodiment of the present application.

In some embodiments, after detecting that the user performs an operation such as clicking on the control 3025 in the user interface 12 shown in FIG. 3K, the mobile phone may display the user interface 16 exemplarily shown in FIG. 3L, and the user interface 16 is used to display the gallery The page corresponding to the category found in the application. In FIG. 3L, the control 3023 is changed from the bold display to the normal display, and the control 3025 is changed from the normal display to the bold display.

In other embodiments, after detecting the operation of opening the gallery application indicated by the user (such as clicking on the icon 301 of the gallery application), the mobile phone may display the user interface 16 exemplarily shown in FIG. 3L, and the user interface 16 is used to display the gallery The page corresponding to the category found in the application. In Figure 3L, control 3025 is shown in bold.

The user interface 16 may include: a control 312, where the control 312 is used to enter the display page of the picture material and/or video material stored in the mobile phone.

In the embodiment of the present application, the specific implementation manner of the user interface 16 may include various. For ease of illustration, in FIG. 3L, the user interface 16 is divided into five parts.

The first part includes a search box for providing the user with a way to search for picture material and/or video material according to keywords such as photos, people, places, etc.

The second part includes controls for entering into creating a new video by using a template method (Fig. 3L uses the text "template creation" and an icon as an example for illustration), and a control 312, and a control for entering into creating a new video using a puzzle method. The control of the video (Fig. 3L uses the text "template creation" and an icon as an example for illustration).

The third section shows pictures divided by portraits. Wherein, the title of the third part is illustrated in FIG. 3L by taking the text “portrait” and the text “more” as examples.

The fourth part includes pictures and/or videos divided by location, as shown in Figure 3L, pictures and/or videos with the location of "Shenzhen", pictures and/or videos with the location of "Guilin City", and pictures and/or videos with the location of Pictures and/or videos of "Chengdu City". The title of the fourth part is illustrated in FIG. 3L by taking the words "location" and the words "more" as examples.

Control 3022, Control 3023, Control 3024, and Control 3025 are displayed in the fifth section.

In addition, the title of the user interface 16 is illustrated in FIG. 3L by taking the word “discovery” as an example. The user interface 16 may further include: controls for implementing editing of the user interface 16 such as adding a new group or deleting an existing group in the user interface 16 ( FIG. 3L uses three black dots as an example for illustration).

After detecting that the user performs an operation such as clicking on the control 312 in the user interface 16 shown in FIG. 3L, the mobile phone can display the user interface 17 exemplarily shown in FIG. The picture material and/or video material of the video.

In this embodiment of the present application, the specific implementation manner of the user interface 17 may include various manners. For convenience of description, in FIG. 3M , the user interface 17 includes a display area 313 and a window 314 overlying the display area 313 .

The display area 313 includes picture material and/or video material, and in the upper left corner of each picture material/video material is displayed a control for magnifying and displaying the picture material/video material (two oblique directions are used in FIG. arrows pointing to the opposite direction as an example), and a control for selecting the picture material/video material is displayed in the lower right corner of each picture material/video material (a rounded rectangular box is used as an example in FIG. 3M for illustration ).

Among them, in the embodiment of the present application, the displayed number of picture materials in the display area 313, the display area of picture materials, the display position of picture materials, the display content of video materials, the display quantity of video materials, the display area of video materials, the display area of video materials, the The parameters such as the display position and the order of each type of material are not limited. For the convenience of description, in FIG. 3M, the display area 313 displays: video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038. For details, please refer to The description in Mode 1 will not be repeated here.

The window 314 may include: a control 3141 (in FIG. 3M, the icon "0/50" is used as an example for illustration, where "0" indicates that no picture material/video material is selected, and "50" indicates that there are 50 pictures in the mobile phone. material/video material), control 3141 is used to indicate the total number of picture material/video material stored in the mobile phone and the number of picture material/video material currently selected by the user, and control 3142, control 3142 is used to enter to start making new The display interface of the video, and the preview area 3143, the preview area 3143 is used to display the picture material and/or video material selected by the user.

After detecting that the user performs an operation of selecting a picture material/video material in the display area 313 shown in FIG. 3M , the mobile phone can display the display changes based on the user operation in the user interface 17 exemplarily shown in FIG. 3N .

For pictures and/or videos that are not selected by the user (in FIG. 3N, pictures other than video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038 are used Taking other picture materials/other video materials as an example for illustration), other picture materials/other material videos in the display area in the user interface 17 maintain the same display screen.

For the picture material and/or video material selected by the user (in FIG. 3N, video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038 are used as examples For illustration), the video material 3031, the picture material 3032, the picture material 3033, the video material 3034, the picture material 3035, the picture material 3036, the picture material 3037 and the video material 3038 in the display area 313 in the user interface 17 are located in each The control for selecting the picture material/video material in the lower right corner of each picture material/video material changes in display (in FIG. 3N, a check mark is added to a rounded rectangular box as an example for illustration).

The control 3141 in the user interface 17 displays that the number of picture materials/video materials selected by the user changes (Fig. 3N uses the icon "8/50" as an example for illustration, where "8" indicates that the user selects eight picture materials/video materials, "50" means that there are 50 picture materials/video materials in the mobile phone, which can be freely created to generate new videos).

The preview area 2143 in the user interface 17 displays that the selected picture material/video material changes (in FIG. 3N, the video material 3031, the picture material 3032, the picture material 3033 and the video material 3034 are displayed, and the picture material is displayed by dragging the slider bar. 3035, picture material 3036, picture material 3037, and video material 3038 are taken as examples for illustration).

After the mobile phone detects that the user performs an operation of generating a new video in the user interface 17 shown in FIG. 3N (for example, clicking on the control 3142 in the user interface 17 ), the mobile phone selects the video material 3031 , the picture material 3032 , and the picture material 3033 based on the user’s selection. , video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038, the type of the video template can be determined to be parent-child type, so based on the parent-child type video template, the video material 3031, picture material 3032 selected by the user , picture material 3033 , video material 3034 , picture material 3035 , picture material 3036 , picture material 3037 , and video material 3038 generate a video, and may display the user interface 15 exemplarily shown in FIG. 3F . For a specific implementation manner of the generated video mentioned here, reference may be made to the description of the generated video in Mode 1.

To sum up, the electronic device can automatically generate a video based on the picture elements and/or video material selected by the user in the gallery application.

The specific implementation manner of the user interface 15 may refer to the content described above, which will not be repeated here. Therefore, the electronic device can display the generated video to the user through the preview area 306 .

In addition, the user interface 15 is used to display controls for editing the generated video. Therefore, the electronic device can provide the user with the function of manually editing the generated video, which is convenient for the user to configure parameters such as the duration, frame, video template, included materials and filters of the video based on their own wishes, thus enriching the style of the video. In addition, the mobile phone can save the video after detecting that the user performs an operation such as clicking the control 309 in the user interface 15 shown in FIG. 3F .

way three

Please refer to FIG. 3A-FIG. 3B, FIG. 3O-FIG. 3Q, FIG. 3T, and FIG. 3F. FIGS. 3O-FIG. 3Q and FIG. 3T are schematic diagrams of human-computer interaction interfaces provided by an embodiment of the present application.

In some embodiments, after detecting that the user performs an operation such as clicking on the control 3024 in the user interface 12 shown in FIG. 3O, the mobile phone may display the user interface 18 exemplarily shown in FIG. 3P, and the user interface 18 is used to display the gallery The page corresponding to the moment category in the application. In FIG. 3P, the control 3023 changes from the bold display to the normal display, and the control 3024 changes from the normal display to the bold display.

In other embodiments, the mobile phone may display the user interface 18 exemplarily shown in FIG. 3P after detecting the operation of opening the gallery application (such as clicking the icon 301 of the gallery application) instructed by the user, and the user interface 18 is used to display the gallery The page corresponding to the moment category in the application. In Figure 3P, control 3024 is shown in bold.

The user interface 18 may include: a control 3151, where the control 3151 is used to enter a display page for creating a new video in the manner provided by the embodiment of the present application.

In the embodiment of the present application, the specific implementation manner of the user interface 18 may include various. For ease of illustration, in Figure 3P, the user interface 18 is divided into three parts.

The second part includes a control 3152 (Figure 3P uses the text "hours of the weekend", the date "September 2020" and a picture material as examples), and the control 3152 is used to display the picture material within a period of time in the mobile phone Video 1 generated by and/or video material, and a control 3153 for displaying video 2 generated by picture material and/or video material in a period of time in the mobile phone (Fig. May 2020" and a picture material as an example), and a control 3154 for displaying video 3 generated from picture material and/or video material in a period of time in the mobile phone (Figure 3P uses the text "hours of the weekend" light", the date "April 2020" and a picture material for illustration). It should be noted that, the picture material/video material in Video 1, Video 2, and Video 3 may or may not be repeated, which is not limited in this embodiment of the present application.

It should be noted that the video 1, the video 2 and the video 3 are all generated by the electronic device according to the solution provided in this application.

Control 3022, Control 3023, Control 3024, and Control 3025 are displayed in the third section.

The title of the user interface 18 is illustrated by taking the word "moment" as an example in FIG. 3P .

In some embodiments, after detecting that the user performs an operation such as clicking on the control 3151 in the user interface 18 shown in FIG. 3P, the mobile phone may display the window 316 exemplarily shown in FIG. 3Q on the user interface 18. The window 316 uses It is used to display picture material and/or video material that can generate movies or puzzles.

After the mobile phone detects that the user performs an operation such as clicking the text "create a movie" in the window 316 shown in FIG. 3Q, the mobile phone can display the user interface 17 exemplarily shown in FIG. 3M. The specific implementation manner of the user interface 17 may refer to the content described above, which will not be repeated here.

After detecting that the user performs an operation of selecting a picture material/video material in the display area 313 shown in FIG. 3M , the mobile phone can display the display changes based on the user operation in the user interface 17 exemplarily shown in FIG. 3N . The specific implementation of the display change of the user interface 17 may refer to the content described above, which will not be repeated here.

In other embodiments, after detecting that the user performs an operation such as clicking on the control 3152 in the user interface 18 shown in FIG. 3P, the mobile phone may display the user interface 19 exemplarily shown in FIG. 3T.

Wherein, the user interface 19 may include a control 317, and the control 317 is used to enter an interface that can play the video 1, and the video 1 mentioned here is the video generated based on the solution of the present application.

After detecting that the user performs an operation such as clicking the control 317 in the user interface 19 shown in FIG. 3Q , the mobile phone may display the user interface 15 exemplarily shown in FIG. 3F .

In addition, the user interface 15 is used to display controls for editing the generated video. Therefore, the electronic device can provide the user with the function of manually editing the generated video, which is convenient for the user to configure parameters such as the duration, frame, video template, included materials and filters of the video based on their own wishes, thus enriching the style of the video.

It should be noted that the parameters such as the control size, control position, display content, and jumping mode of the user interface mentioned in the first, second, and third modes include but are not limited to the foregoing descriptions.

Based on the descriptions of Mode 1, Mode 2, and Mode 3, the mobile phone can save the generated video in the gallery application.

Please refer to FIGS. 3A and 3R to 3S. FIGS. 3R to 3S are schematic diagrams of a human-computer interaction interface provided by an embodiment of the present application.

After detecting that the user performs an operation of opening the gallery application in the user interface 11 shown in FIG. 3A (for example, clicking on the icon 301 of the gallery application), the mobile phone can display the user interface 12' exemplarily shown in FIG. 3R, and the user interface 12' The page used to display albums in the Gallery app.

The interface layout of the user interface 12 ′ is basically the same as that of the user interface 12 shown in FIG. 3B , and the specific implementation can refer to the description of the user interface 12 shown in FIG. 3B in the first mode, which will not be repeated here. Unlike the user interface 12 shown in Figure 3B, the number of videos stored in the user interface 12' is incremented by 1, so the user interface 12' in Figure 3R shows that the number of all photos has increased from "182" to "183" , the number of videos increased from "49" to "50".

After detecting that the user performs an operation such as clicking on the control 3021 in the user interface 12' shown in FIG. 3R, the mobile phone can display the user interface 13' exemplarily shown in FIG. 3S, and the user interface 13' is the pictures and videos in the mobile phone. display interface.

The interface layout of the user interface 13 is basically the same as that of the user interface 13 shown in FIG. 3C . For a specific implementation manner, refer to the description of the user interface 13 in FIG. 3C in the first mode, which will not be repeated here. Similar to the user interface 13 shown in FIG. 3C, the pictures/videos stored in the user interface 13' move to the next as a whole. Therefore, according to the time sequence from near to far from the current moment, the user interface 13' in FIG. The first clip displayed is the newly generated video 3039.

After detecting that the user performs an operation such as clicking on the video 3039 in the user interface 13' shown in FIG. 3S, the mobile phone can play the video 3039.

In this embodiment of the present application, each video template may correspond to a piece of music. Usually, the music corresponding to different video templates is different. The electronic device can keep the music corresponding to each video template unchanged by default, or can change the music corresponding to each video template based on user selection, so that the electronic device can flexibly set according to the actual situation. The music may be preset by the electronic device, or manually added by the user, which is not limited in this embodiment of the present application.

On the one hand, video templates are also related to movement, speed, and transitions. Generally, no matter whether the music corresponding to the video templates is the same or not, different video templates correspond to at least one of camera movement, speed and transition.

For the music corresponding to any video template, each segment of the music can match the set motion, speed and transition. The motion and transition may be related to the type of material, the motion used for the video material may be the same or different from the motion used for the picture material, and the transition used for the video material may be the same or different from that used for the picture material. In addition, the video material can usually be set with a playback effect corresponding to the speed.

Please refer to FIG. 4A-FIG. 4J, FIG. 4A-FIG. 4J are schematic diagrams showing the effect of the picture material 3033 after the mirror movement is adopted.

The mobile phone stores the picture material 3033 exemplarily shown in FIG. 4A , where the picture material 3033 can be referred to the description of the embodiment in FIG. 3C , which is not repeated here.

When the mobile phone displays the picture pixels 3033 using the mirror movement effect moving diagonally, the mobile phone can change from displaying the interface 11 exemplarily shown in FIG. 4B to displaying the interface 12 exemplarily shown in FIG. 4C , wherein the interface 11 is a picture The area a1 of the material 3033, the interface 12 is the area a2 of the picture material 3033, and the area a1 and the area a2 are located at different positions of the picture material 3033.

Wherein, in addition to the mirror movement effect of diagonal movement, the electronic device may also adopt the mirror movement effect of upward, upward, leftward, rightward, etc., which is not limited in this embodiment of the present application.

When the mobile phone uses the magnified mirror motion effect to display the picture pixels 3033 , the mobile phone can change from displaying the interface 1 exemplarily shown in FIG. 4B to displaying the interface 13 exemplarily shown in FIG. 4D , wherein the interface 11 is the area of the picture material 3033 a1, the interface 13 is an enlarged view of the area a3 of the picture material 3033.

Wherein, in addition to the magnified mirror movement effect, the electronic device may also use the zoomed mirror movement effect, which is not limited in this embodiment of the present application.

In addition, when the picture material 20 is a vertical picture exemplarily shown in FIG. 4E , and the generated video adopts a banner format, the electronic device can display the picture material 20 by moving the mirror from top to bottom. For example, the mobile phone may change from displaying the interface 21 exemplarily shown in FIG. 4F to displaying the interface 22 exemplarily shown in FIG. 4G , wherein the interface 21 is the area b1 of the picture material 20 , the interface 22 is the area b2 of the picture material 20 , And the area b1 and the area b2 are located at different positions of the picture material 20 . Optionally, the shape of the region composed of the region b1 and the region b2 may be set to be a square. Wherein, if the picture material 20 includes characters, faces, etc., the area formed by the area b1 and the area b2 by the electronic device may include as many areas as possible corresponding to the characters and faces in the material.

When the picture material 30 is the banner picture exemplarily shown in FIG. 4H , and the generated video adopts a vertical frame, the electronic device can display the picture material 30 by using a mirror movement effect that moves from left to right. For example, the mobile phone can change from displaying the interface 31 exemplarily shown in FIG. 4I to displaying the interface 32 exemplarily shown in FIG. 4J , wherein the interface 31 is the area c1 of the picture material 30 , the interface 32 is the area c2 of the picture material 303 , And the area c1 and the area c2 are located at different positions of the picture material 30 . Optionally, the shape of the region composed of the region c1 and the region c2 may be set to be a square. Wherein, if the picture material 30 includes characters, faces, etc., the area formed by the area c1 and the area c2 by the electronic device may include as many areas as possible corresponding to the characters and faces in the material.

Therefore, it is beneficial for the video generated by the electronic device to maximize the display material, enrich the content of the video, and ensure the cinematic and picture sense brought by the video.

Please refer to FIG. 5 , which shows a schematic diagram of the effect of the video material 3038 at different speeds. For the video material 3038, reference may be made to the description of the embodiment in FIG. 3C, which will not be repeated here.

As shown in FIG. 5 , it is assumed that the video generated by the electronic device based on the material plays the video material 3038 in both the time period from t0 to t1 and the time period from t2 to t3, and the time period from t2 to t3 is equal to the length of the time period from t0 to t1. Three times, the speed of the electronic device playing the video material 3038 in the time period t0 to t1 is three times the speed of playing the video material 3038 in the time period t2 to t3.

It should be noted that, in addition to the triple speed, the speed may also include a speed of any ratio, which is not limited in this embodiment of the present application.

Please refer to FIG. 6 , which shows a schematic diagram of the effect of the transition between the picture material 3033 and the picture material 3032 . The picture material 3033 and the picture material 3032 may refer to the description of the embodiment in FIG. 3C , which will not be repeated here.

As shown in FIG. 6 , it is assumed that the electronic device plays the picture material 3033 in the time period from t4 to t5, plays the picture material 3032 in the time period from t6 to t7, and the picture material 3033 uses the The transition effect of superimposing and blurring transitions to the picture material 3032, then the electronic device plays the picture material 3033 in the time period from t4 to t5, and plays the gradually enlarged picture material superimposed on the blurred picture material 3033 in the time period from t5 to t6. 3032. Play the picture material 3032 in the time period from t6 to t7.

It should be noted that, in addition to the "superimposed blur" effect, the transition may also include effects such as focus blur, which is not limited in this embodiment of the present application.

Therefore, the electronic device can realize the scene scheduling and shot scheduling of the material according to the set movement, speed and transition.

On the other hand, video templates are related to scene types. Generally, regardless of whether the music corresponding to the video templates is the same, different types of video templates correspond to different scene types; video templates of the same type correspond to the same scene type.

When the user selects the music corresponding to the default video template, the electronic device does not need to adjust the duration of each segment of the video template, so that the music jam point can be realized. When the user selects other music as the music corresponding to the video template, the electronic device needs to perform beat detection on the music selected by the user, obtain the tempo of the music selected by the user, and then determine whether the duration of each segment of the video template is equal to the obtained beat Integer multiples of the tempo, adjust the durations of segments whose durations are not equal to the integer multiples of the obtained tempo, so that the duration of each segment in the video template is an integer multiple of the tempo.

Wherein, for any piece of music, in the embodiment of the present application, the BPM (Beat Per Minute, beat number) detection method can be used to detect the beat of the music to obtain the tempo (bpm), wherein the electronic device uses digital signal processing (digital signal processing). , DSP) method to analyze the audio to get the beat of the music. The usual algorithm divides the original audio into several segments, then obtains the spectrum through fast Fourier transform, and finally performs filter analysis based on the sound energy to obtain the beat of the music.

It should be noted that the scene type corresponding to each segment in each video template is based on actual experience in advance (such as the user's scene type corresponding to a single segment at certain locations and the scene type corresponding to multiple consecutive segments). are all perceptually strong) to set.

For the music corresponding to any video template, the embodiment of the present application may use the beat of the music as the dividing line, divide the entire piece of music into multiple segments, and match each segment with the set scene type.

Wherein, each segment is an integer multiple of the tempo of the music, so as to realize the music timing of each segment. It can be understood that the beat of music is the beat or beat, which refers to the combination rule of upbeat and downbeat, and specifically refers to the total length of notes in each measure in the score. These notes can be, for example, half notes, quarter notes, Eighth notes, etc. Usually, a piece of music can be composed of multiple beats, and the beats of a piece of music are usually fixed.

It is understandable that the selection of materials is random, and in actual situations it is very likely that the materials cannot all meet the scene type set for each clip. Therefore, when the aforementioned problem occurs, the electronic device can adjust the arrangement order of the materials in various ways.

In some embodiments, the electronic device may prioritize each segment. The high-priority segments may include, but are not limited to, segments such as the beginning part, the chorus part, the ending part, or the accent part of the music. Further, the electronic device can give priority to satisfying the scene type set by the high-priority segment, and place the material corresponding to the scene type set by the high-priority segment to the high-priority segment, and then follow the settings of the remaining segments. The scene type of the remaining material is placed in the remaining clips. At this time, the scene type of the remaining material and the scene type set in the remaining clips can be the same or different.

In other embodiments, the electronic device may preferentially satisfy the scene type set by the segment at the front, and place the material corresponding to the type of the scene set by the segment at the front in the segment at the front, and then follow the The scene type set for the remaining clips, the remaining material is placed in the remaining clips. At this time, the scene type of the remaining material and the scene type set for the remaining clips can be the same or different.

Wherein, the electronic device may also preferentially satisfy the scene type set by the segment located at the front among the remaining segments.

For the music corresponding to any video template, in this embodiment of the present application, the beat of the music can be used as the dividing line to divide the entire piece of music into multiple segments, and the multiple consecutive segments are matched with the set scene types, and the remaining segments are The type of scene is not limited. Thereby, the lens sense and cinematic sense of the generated video are enhanced. The multiple continuous segments may be segments such as the beginning part, the ending part, or the chorus part of the music.

Taking the classification of scene types into three types exemplarily shown in FIG. 7 , namely, close-range, medium-range, and long-range, as an example, the scene types corresponding to each of the multiple continuous segments are introduced. Among them, A represents the type of scene corresponding to the close shot, B represents the type of scene corresponding to the medium shot, and C represents the type of scene corresponding to the long shot.

For example, in this embodiment of the present application, the scene types corresponding to the five consecutive segments corresponding to the beginning and/or the end of the music may be CCCBA, respectively, so that the generated video has a suspenseful effect at the beginning, or has unfinished content at the end. continued effect.

For another example, in this embodiment of the present application, the scene types corresponding to the four consecutive segments corresponding to the beginning and/or the end of the music may be ABBC, respectively, so that the generated video is prepared for the narration of the video at the beginning or at the end. Effect.

For another example, in this embodiment of the present application, the scene types corresponding to the five consecutive segments corresponding to the segment after the beginning and/or the segment before the end of the music are respectively BBBBB, so that the generated video has a narration in the corresponding segment. Effect.

For another example, in this embodiment of the present application, the scene types corresponding to the five consecutive segments corresponding to the chorus part of the music can be CCCCA respectively, so that the generated video has the effect of enhancing the narration of the video to a climax at the chorus.

It should be noted that the embodiments of the present application include, but are not limited to, specific implementations of scene types corresponding to the above-mentioned multiple consecutive pieces of music.

Therefore, the electronic device can adjust the arrangement order of the materials according to the scene type of the segment set at the beat point of the music.

To sum up, the electronic equipment arranges the materials according to the sequence of scenes set in the video template, and adds the scene and lens sense of the materials according to the movement, speed and transition set in the video template, and generates a video template with The video with the corresponding playback effect makes the generated video more expressive and intense in terms of the narrative of the film's plot, the expression of characters' thoughts and feelings, and the handling of character relationships, thereby enhancing the artistic appeal of the generated video.

For convenience of description, the specific implementation of the video template is introduced by taking the parent-child type video template and the travel type video template as examples in conjunction with Table 1 and Table 2. In Tables 1 and 2, the types of scene types are taken as examples of the three types of close-up, medium-range and long-range shown in Figure 7. For ease of expression, A represents the type of scenery corresponding to the close-up, and B represents the type of scenery corresponding to the medium-range. Type, C represents the type of scene corresponding to the vision.

Table 1 Parent-child video templates

In Table 1, at the beginning of the video, for the video material, the transition uses the effect of "white fading" and the effect of "fading in and out of title". For the picture material, the transition uses the "white fade" effect.

At 6x time, for video material, the transition uses a "quick down" effect. For the picture material, the transition adopts the effect of "up and down blurred oblique angle transition".

At 14x time, for the video material, the transition uses a "stretch in" effect. For the picture material, the transition adopts the effect of "left and right blur and push".

At 22x time, for the video material, the transition uses a "fast up" effect. For image assets, transitions use a "push up and focus blur/zoom behind the scenes" effect.

At 32x time, for the video material, the transition uses the effect of "speed left". For the picture material, the transition adopts the effect of "rotation blur on the right axis".

At 34x time, for the video material, the transition uses a "rotate right" effect. For the picture material, the transition uses the effect of "rotate to the left to blur".

At 36x time, for the video material, the transition adopts the effect of "quick left slide". For the picture material, the transition uses the "perspective blur" effect.

At 38x time, for the video material, the transition uses the effect of "blur and dissolve". For image materials, transitions do not use any effects.

At 40x time, for the video material, the transition uses the effect of "speed left". For image materials, transitions do not use any effects.

At 42x time, for the video material, the transition uses the effect of "fade out" and "rotate right". For the picture material, the transition adopts the effect of "whitening and fading out".

At 44x time, for the video material, the transition adopts the effect of "quick left slide". For image materials, transitions do not use any effects.

At 46x time, for the video material, the transition uses a "blurred and dissolve" effect. For image materials, transitions do not use any effects.

At 48x time, for the video material, the transition adopts the effect of "whitening and fading out". For the picture material, the transition adopts the effect of "whitening and fading out".

At 49x time, for the video material, the transition uses a "rotate left" effect. For the picture material, the transition uses the "perspective blur" effect.

At 52x time, for the video material, the transition uses a "left rotation" effect. For image materials, transitions do not use any effects.

At 56x time, for the video material, the transition uses a "fast left" effect. For image materials, transitions do not use any effects.

At 60x, for video footage, the transition uses a "fast left" effect. For image materials, transitions do not use any effects.

At 62x time, for the video material, the transition uses a "stretch in" effect. For image materials, transitions do not use any effects.

Table 2 Video templates for travel types

For a specific implementation manner of the transition field in Table 2, reference may be made to the description manner of the transition field in Table 1, which will not be repeated here.

It should be noted that the video template includes, but is not limited to, parameters related to scene type, camera movement, speed, and transition.

In addition, the video template can also adaptively adjust the movement of the video lens based on the frame of the material to achieve the best playback effect. For example, when generating a banner video, the electronic device can use the moving method of the lens from top to bottom to achieve the maximum regionalized display effect of the vertical material; when generating a vertical video, the electronic device can use the lens to move from left to bottom. The right movement method realizes the maximum regional display effect of the banner material. Therefore, it is beneficial to maximize the display of materials in the video, enrich the content of the video, and ensure the cinematic and picture sense brought by the video.

In the embodiment of the present application, each scene type in the video template corresponds to a segment, and the duration of the segment may be the same or different. The electronic device may first place the material selected by the user based on the duration of each segment in the video template. Typically, video footage can be placed in longer clips in preference to image footage. The electronic device then adjusts the arrangement order of the placed materials based on the scene type corresponding to each clip, so that the scene type of the material matches the scene type of the clip, thereby ensuring that the material selected by the user appears at least once in the generated video and adjacent clips do not place the same footage.

It should be noted that the embodiments of the present application are not limited to the above implementation manner to adjust the arrangement order of materials in the video.

In addition, when the number of materials selected by the user is large and the duration of the video is set to be small, the duration of the segment corresponding to the scene type can be set to be small, so that all the materials can appear in the video once. When the number of materials selected by the user is small and the duration of the video is set relatively large, the electronic device can select one or more segments from the video materials to repeat N times in the generated video, where N is a positive integer greater than 1 . If the length of the longer video still cannot be satisfied at this time, the electronic device may repeat all the arranged materials M times in the generated video, where M is a positive integer greater than 1.

Wherein, the electronic device can usually set a minimum duration and a maximum duration for the duration of the music corresponding to the video template, so as to ensure that the material selected by the user appears at least once in the generated video.

Based on the foregoing description, the playback effect of the video 3039 in FIG. 3S is related to the video template. Generally, the video 3039 will play differently depending on the video template. Wherein, when the user selects video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038, the video 3039 may include: video material 3031, picture material Material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and video material 3038.

In the embodiment of the present application, the electronic device can identify that the scene type corresponding to the video material 3031 is BCBBB, the scene type of the picture material 3032 is B, the scene type of the picture material 3033 is B, and the scene type corresponding to the video material 3034 The category is CCCC, the scene type of the picture material 3035 is B, the scene type of the picture material 3036 is A, the scene type of the picture material 3037 is A, and the scene type corresponding to the video material 3038 is BCCCC.

Based on the video material 3031, the picture material 3032, the picture material 3033, the video material 3034, the picture material 3035, the picture material 3036, the picture material 3037 and the video material 3038, the electronic device recognizes that a parent-child video template can be used to generate the video 3039.

In some embodiments, if the parent-child video template shown in Table 1 is used, the electronic device will be based on video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037 and the respective scene types of video material 3038, according to Table 1, the scene type corresponding to each segment of the music is given. , the picture material 3037 and the video material 3038 are respectively placed at the positions corresponding to the music, and a video 3039 is obtained.

In other embodiments, the electronic device can set scene types of multiple continuous segments corresponding to music according to the video template to enhance the playback effect of the generated video, which is beneficial to enhance the video's sense of shots and movies.

It should be noted that, in addition to the above two methods, the electronic device can set the scene type in the video template according to the actual situation and experience value, and the embodiment of the present application does not limit the setting method of the scene type in the video template.

Below, with reference to FIGS. 8A-8E , an example will be given to illustrate the playback effect of the video generated by the electronic device based on the material selected by the user.

Please refer to FIG. 8A-FIG. 8E. FIG. 8A-FIG. 8E are schematic diagrams showing the playback sequence of each material when the electronic device plays the generated video.

As shown in FIG. 8A , when the user selects picture material 11, picture material 12, picture material 13, picture material 14 and picture material 15, the electronic device determines that the scene type in the video template is CCCBA, and the scene type is The durations of CCCBA are 4x, 2x, 2x, x and 2x respectively, where x=0.48 seconds, and the scene type of picture material 11 is B, the scene type of picture material 12 is B, and the scene type of picture material 13 is C, the scene type of the picture material 14 is A and the scene type of the picture material 15 is C.

Then, based on the scene type in the video template, and the scene types of picture material 11, picture material 12, picture material 13, picture material 14, and picture material 15, the electronic device can learn that there is a 2x duration missing in all the materials. Scene type C, so that all the materials cannot accurately match the scene type in the video template. Since all the material needs to appear at least once, the electronic device can change the scene type CCCBA in the video template to CBCBA.

Therefore, based on the scene type CBCBA, the electronic device adjusts the arrangement order of picture material 11, picture material 12, picture material 13, picture material 14 and picture material 15 to generate the video exemplarily shown in FIG. 8A.

In FIG. 8A , the playback sequence of picture material 11, picture material 12, picture material 13, picture material 14 and picture material 15 in the generated video is:

Between 0-4x: picture material 13;

Between the 4x-6x: picture material 11;

Between the 6x-8x: picture material 15;

Between the 8x-9x: picture material 12;

Between 9x-11x: Image 14.

Furthermore, the scene types corresponding to the videos exemplarily shown in FIG. 8A are respectively CBCBA.

As shown in FIG. 8B , when the user selects picture material 21, picture material 22, picture material 23, picture material 24 and picture material 25, the electronic device determines that the scene type in the video template is CCCBA, and the scene type is The durations of CCCBA are 4x, 2x, 2x, x and 2x respectively, where x=0.48 seconds, and the scene type of picture material 21 is C, the scene type of picture material 22 is B, and the scene type of picture material 23 is C, the scene type of the picture material 24 is A and the scene type of the picture material 25 is C.

Based on the scene type in the video template and the scene types of picture material 21, picture material 22, picture material 23, picture material 24, and picture material 25, the electronic device can know that all the materials can accurately match the scene type in the video template. Scene type. Therefore, based on the scene type CCCBA, the electronic device adjusts the arrangement order of picture material 21, picture material 22, picture material 23, picture material 24, and picture material 25 to generate the video exemplarily shown in FIG. 8B.

In FIG. 8B , the playback order of picture material 21, picture material 22, picture material 23, picture material 24 and picture material 25 in the generated video is:

Between 0-4x: picture material 23;

Between the 4x-6x: picture material 21;

Between the 6x-8x: picture material 25;

Between the 8x-9x: picture material 22;

Between 9x-11x: Image 24.

Furthermore, the scene types corresponding to the videos exemplarily shown in FIG. 8B are respectively CCCBA.

As shown in FIG. 8C, when the user selects picture material 31, video material 31, video material 32, picture material 32 and picture material 33, the electronic device determines that the scene type in the video template is CCCBA, and the scene type is The durations of CCCBA are 4x, 2x, 2x, x, and 2x, respectively, where x=0.48 seconds, and the scene type of the picture material 31 is B, the scene type of the video material 31 is B, and the duration of the video material 31 is equal to x, The scene type of the video material 32 is C, the duration of the video material 32 is greater than or equal to 4x, the scene type of the picture material 32 is A, and the scene type of the picture material 33 is C.

Then, based on the scene type in the video template, and the scene types of picture material 31, video material 31, video material 32, picture material 32, and picture material 33, the electronic device can know that there is a missing one with a duration of 2x in all the materials. Scene type C, so that all the materials cannot accurately match the scene type in the video template. Since all the material needs to appear at least once, the electronic device can change the scene type CCCBA in the video template to CBCBA.

Therefore, the electronic device adjusts the arrangement order of picture material 31, video material 31, video material 32, picture material 32, and picture material 33 based on the scene type CBCBA to generate the video exemplarily shown in FIG. 8C.

In FIG. 8C , the playback order of picture material 31, video material 31, video material 32, picture material 32 and picture material 33 in the generated video is:

Between 0-4x: video material 32;

Between the 4x-6x: picture material 31;

Between the 6x-8x: picture material 33;

Between 8x-9x: video material 32;

Between 9x-11x: Image 32.

Furthermore, the scene types corresponding to the videos exemplarily shown in FIG. 8C are respectively CBCBA.

As shown in FIG. 8D, when the user selects picture material 41, video material 41, video material 42, picture material 42 and picture material 43, the electronic device determines that the scene type in the video template is CCCBA, and the scene type is The durations of CCCBA are 4x, 2x, 2x, x, and 2x, respectively, where x=0.48 seconds, and the scene type of the picture material 41 is C, the scene type of the video material 41 is B, and the duration of the video material 41 is equal to x, The scene type of the video material 42 is C, the duration of the video material 42 is greater than or equal to 4x, the scene type of the picture material 42 is A, and the scene type of the picture material 43 is C.

Based on the scene type in the video template, and the scene types of picture material 41, video material 41, video material 42, picture material 42, and picture material 43, the electronic device can know that all the materials can accurately match the scene type in the video template. Scene type. Therefore, the electronic device adjusts the arrangement order of picture material 41, video material 41, video material 42, picture material 42 and picture material 43 based on the scene type CCCBA to generate the video exemplarily shown in FIG. 8D.

In FIG. 8D, the playback sequence of the picture material 41, the video material 41, the video material 42, the picture material 42 and the picture material 43 in the generated video is:

Between 0-4x: video material 42;

Between the 4x-6x: picture material 41;

Between the 6x-8x: picture material 43;

Between 8x-9x: video material 42;

Between 9x-11x: Image 42.

Furthermore, the scene types corresponding to the videos exemplarily shown in FIG. 8D are respectively CCCBA.

As shown in FIG. 8E, when the user selects the picture material 51, the video material 51, the video material 52 and the picture material 52, the electronic device determines that the scene type in the video template is CCCBA, and the durations of the scene type CCCBA are respectively are 4x, 2x, 2x, x and 2x, where x=0.48 seconds, and the scene type of the picture material 51 is C, the scene type of the video material 51 is BC, and the scene type in the video material 51 is corresponding to C The duration of the segment is equal to 2x, the duration of the segment corresponding to the scene type B in the video material 51 is equal to x, the scene type of the video material 52 is C, the duration of the video material 42 is greater than or equal to 4x, and the scene type of the picture material 52. The other type is A.

Then, based on the scene type in the video template, and the scene types of the picture material 51, video material 51, video material 52, and picture material 52, the electronic device can know that all the materials can accurately match the scene type in the video template. Therefore, the electronic device adjusts the arrangement order of the picture material 51 , the video material 51 , the video material 52 and the picture material 52 based on the scene type CCCBA to generate the video exemplarily shown in FIG. 8E .

In FIG. 8E, the playback sequence of the picture material 51, the video material 51, the video material 52 and the picture material 52 in the generated video is:

Between 0-4x: video material 52;

Between the 4x-6x: the fragment corresponding to the scene type C in the picture material 51;

Between the 6x-8x: picture material 51;

Between the 8x-9x: the segment corresponding to the scene type B in the video material 51;

Between 9x-11x: Image 52.

Furthermore, the scene types corresponding to the videos exemplarily shown in FIG. 8E are respectively CCCBA.

Based on the foregoing description, after determining that the scene type in the video template is CCCBA, the electronic device can consider the playback effect of the video, the duration of the video, the scene type in the video, The use of the material, the quantity of the material, the scene type of the material, whether the material supports repeated use and other factors, the scene type of the material and the scene type in the video template are matched to a preset degree to generate the video. That is, the scene type corresponding to the video generated by the electronic device is completely or partially the same as the scene type in the video template. The preset degree may be 100% (ie, exact matching) or 90% (ie, fuzzy matching), and usually the preset degree is greater than or equal to 50%. In the embodiment of the present application, the electronic device adjusts the arrangement order of the materials in the generated video based on the arrangement order of the scene types in the video template, and then combines the techniques of mirror movement, speed, and transition in the video template to generate a coherent line of sight. and high-quality video.

To sum up, the video generation method of the embodiment of the present application enhances the sense of shots and movies of the video, which is beneficial to improve the user experience.

Based on the foregoing description, the embodiments of the present application may provide a video generation method.

Please refer to FIG. 9. FIG. 9 shows a schematic diagram of a video generation method provided by an embodiment of the present application. As shown in FIG. 9 , the video generation method of the embodiment of the present application may include:

S101. The electronic device displays a first interface of a first application, where the first interface includes a first control and a second control.

S102. After receiving the first operation acting on the first control, the electronic device determines that the arrangement order of the first material, the second material and the third material is the first order; The second material and the third material generate the first video.

S103. After receiving the second operation acting on the second control, the electronic device determines that the arrangement order of the first material, the second material and the third material is the second order, and the second order is different from the third order; In the second sequence, a second video is generated from the first material, the second material and the third material.

Wherein, the first material, the second material and the third material are different image materials stored in the electronic device, and the third order is the time sequence in which the first material, the second material and the third material are stored in the electronic device. The order is different from the third order.

In this embodiment of the present application, reference may be made to the foregoing description for specific implementation manners of the first material, the second material, and the third material. For the specific implementation of the first control, refer to any one of the

controls

30811, 30812, 30813, and 30814 exemplarily shown in FIG. 3F, and for the specific implementation of the second control, refer to the control exemplarily shown in FIG. 3F 30811, control 30812, control 30813, and control 30814, the first control is different from the second control. The first order and the second order may be the same or different, which is not limited in this embodiment of the present application. The playback effects of the first video and the second video are different. For details, please refer to the aforementioned video 1, video 2, video 3 and user-selected video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, picture material 3037, and video material 3038 generate videos. In some embodiments, the first application is a gallery application of the electronic device.

In the embodiment of the present application, the electronic device matches the appropriate video template by identifying the scene type of the material, and adjusts the arrangement order of the material based on the scene type that has been set for each segment in the video template. The camera movement, speed and transition that have been set for each clip can automatically generate a video with a coherent line of sight and high quality, without relying on the user's manual editing, which enhances the camera and cinematic sense of the video, and improves the user experience.

In some embodiments, the first video is divided into a plurality of segments with the beat point of the music as a dividing line; the first material, the second material and the third material appear at least once in the first video, and in any part of the first video The materials appearing in two adjacent segments are different; the first material, the second material and the third material appear at least once in the second video, and the materials appearing in any two adjacent segments of the second video are different.

In some embodiments, the method further includes: displaying the second interface of the first application by the electronic device; after the electronic device receives the third operation acting on the second interface, displaying the first material, the second material and the third material Generate the first video.

In this embodiment of the present application, for the specific implementation process of the second interface, reference may be made to the description of the user interface 13 exemplarily shown in FIG. 3E in Mode 1, or, reference may be made to the description of the user interface 17 exemplarily shown in FIG. 3N in Mode 2 , or, refer to the description of the user interface 17 exemplarily shown in FIG. 3N in the third way. For the specific implementation process of the third operation, please refer to the description of clicking on the text “video” in the window 305 of the user interface 13 in the first mode exemplarily shown in FIG. For the description of the control 3142 in the user interface 17, or, please refer to the description of the control 3142 in the click user interface 17 as exemplarily shown in FIG. 3N in the third way.

In some embodiments, the method further includes: the electronic device determines from the first material, the second material, the third material and the fourth material to generate the first video from the first material, the second material and the third material; wherein, The fourth material is an image material stored in the electronic device that is different from the first material, the second material, and the third material.

In this embodiment of the present application, for the specific implementation process of the above solution, reference may be made to the description of video 1, video 2, and video in the user interface 18 exemplarily shown in FIG. 3P in the third mode.

In some embodiments, the first interface further includes a third control; the method further includes: after the electronic device receives the fourth operation acting on the third control, displaying a third interface, where the third interface includes: the configuration information option, the configuration information includes: at least one parameter of duration, filter, frame, material or title; after the electronic device receives the fifth operation acting on the option of the configuration information, in the first order, based on the configuration information, The first material, the second material, and the third material generate a third video.

In this embodiment of the present application, for a specific implementation manner of the third control, reference may be made to the descriptions of the control 3082 , the control 3083 , the control 3084 , and the control 3085 exemplarily shown in FIG. 3F , which will not be repeated here. For the third interface, please refer to the description of the user interface 21 exemplarily shown in FIG. 3G , or the description of the user interface 22 exemplarily shown in FIG. 3H , or the description of the user interface 23 exemplarily shown in FIG. 3I , or, The description of the user interface 24 exemplarily shown in FIG. 3J will not be repeated here.

For example, the electronic device can adjust parameters such as the duration, frame, whether to add new material, whether to delete existing material, etc. of the video 1 through the user interface 21 exemplarily shown in FIG. 3G . As another example, the electronic device may adjust the music of the video 1 through the user interface 22 exemplarily shown in FIG. 3H . For another example, the electronic device may adjust the filter of the video 1 through the user interface 23 exemplarily shown in FIG. 3I . For another example, the electronic device can adjust whether to add a title to the video 1 through the user interface 24 exemplarily shown in FIG. 3J .

In some embodiments, the first interface further includes a fourth control; the method further includes: after the electronic device generates the first video, in response to a fourth operation acting on the fourth control, the electronic device saves the first video. In this embodiment of the present application, for a specific implementation manner of the fourth control, reference may be made to the description of the control 309 exemplarily shown in FIG. 3F , which is not repeated here.

In some embodiments, the method specifically includes: the electronic device determines the scene type corresponding to the first material, the scene type corresponding to the second material, and the scene type corresponding to the third material; the electronic device determines the scene type corresponding to the first material based on the scene type corresponding to the first material. type, the scene type corresponding to the second material, the scene type corresponding to the third material, and the scene type set for each segment in the first video template, determine the material that matches the scene type corresponding to the first segment, The first segment is any segment in the first video template; the arrangement order of the materials corresponding to all the segments in the first video template is the first order; the electronic device is based on the scene type corresponding to the first material, the second material The corresponding scene type, the scene type corresponding to the third material, and the scene type set for each segment in the second video template, determine the material that matches the scene type corresponding to the second segment, and the second segment is the first segment. Any one segment in the two video templates; and the arrangement order of the materials corresponding to all the segments in the second video template is the second order; wherein, the first video template is different from the second video template, and each The segments correspond to each segment in the second video template, and each segment in the second video corresponds to each segment in the second video template.

In this embodiment of the present application, for the above solutions, reference may be made to the aforementioned video material 1, video 2, video 3 and user-selected video material 3031, picture material 3032, picture material 3033, video material 3034, picture material 3035, picture material 3036, the description of the video generated by the picture material 3037 and the video material 3038, which will not be repeated here.

In some embodiments, the method further includes: the electronic device converts the first material, the second material and the first material according to the first order and the motion effect, speed effect and transition effect set for each segment in the first video template. The first video is generated from three materials; the electronic device generates the first, second and third materials according to the second order and the motion mirror effect, speed effect and transition effect set for each segment in the second video template Second video.

In the embodiment of the present application, the above-mentioned solution can refer to the description mentioned above, the specific implementation of the mirror movement effect can refer to the description exemplarily shown in FIG. 5 , and the specific implementation of the speed effect can refer to the description exemplarily shown in FIG. 6 . , the specific implementation of the transition effect may refer to the description exemplarily shown in FIG. 7 , which will not be repeated here.

In some embodiments, when the first material is a picture material, the method specifically includes: the scene type corresponding to the first material and the scene type corresponding to the first segment of the electronic device are the same, or the scene type corresponding to the first material is When the type is adjacent to the order of the scene type corresponding to the first clip according to the preset rule, the first material is determined to be a material matching the scene type corresponding to the first clip; the electronic device is in the scene type corresponding to the first material. The scene type corresponding to the second clip is the same, or, when the scene type corresponding to the first material is adjacent to the scene type corresponding to the second clip according to the preset rules, the first material is determined to be the same as the second clip. The corresponding scene type matches the material.

In this embodiment of the present application, for the specific implementation process of the above solution, reference may be made to the descriptions exemplarily shown in FIGS. 8A to 8E , which will not be repeated here. For the specific implementation of the first pixel, reference may be made to the picture pixels exemplarily mentioned in FIGS. 8A-8E .

In some embodiments, when the first material is a video material, the method specifically includes: in the electronic device, the scene type corresponding to the fourth material is the same as the scene type corresponding to the first segment or the scene type corresponding to the fourth material is according to When the preset rule is adjacent to the sequence of the scene types corresponding to the first clip, and the duration of the fourth material is equal to the duration of the first clip, the fourth material is cut out from the first material, and the fourth material is determined to be the same as the first clip. A material that matches the scene type corresponding to the clip; the scene type corresponding to the fourth material of the electronic device is the same as the scene type corresponding to the second clip or the scene type corresponding to the fourth material corresponds to the second clip according to preset rules. When the sequence of the scene types is adjacent, and the duration of the fourth material is equal to the duration of the second segment, the fourth material is cut out from the second material, and the fourth material is determined to match the scene type corresponding to the second segment. material; wherein, the fourth material is part or all of the first material.

In this embodiment of the present application, for the specific implementation process of the above solution, reference may be made to the descriptions exemplarily shown in FIGS. 8A to 8E , which will not be repeated here. For the specific implementation of the first pixel, reference may be made to the video pixels exemplarily mentioned in FIGS. 8A-8E , and for the specific implementation of the fourth pixel, reference may be made to the video pixel 51 or the video pixel 52 .

In some embodiments, according to the ordering of preset rules, the types of scenes include: close-up, medium-range, and distant, the types of scenes adjacent to the close-range are distant, and the types of scenes adjacent to the mid-range are close-range and distant. The type of scene adjacent to the distant scene is the close scene. In this embodiment of the present application, the division of scene types is not limited to the foregoing implementation manner, and for details, reference may be made to the descriptions mentioned above, which will not be repeated here.

Exemplarily, the present application provides an electronic device, including: a memory and a processor; the memory is used for storing program instructions; the processor is used for calling the program instructions in the memory to make the electronic device execute the video generation method in the foregoing embodiments.

Exemplarily, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen and a sensor; the chip system includes: a processor; when the processor executes the computer instructions stored in the memory, the electronic device executes the foregoing The video generation method in the embodiment.

Exemplarily, the present application provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to cause an electronic device to implement the video generation method in the foregoing embodiments.

Exemplarily, the present application provides a computer program product, comprising: execution instructions, the execution instructions are stored in a readable storage medium, at least one processor of an electronic device can read the execution instructions from the readable storage medium, and at least one processor Executing the execution instruction enables the electronic device to implement the video generation method in the foregoing embodiment.

In the above embodiments, all or part of the functions may be implemented by software, hardware, or a combination of software and hardware. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented. The process can be completed by instructing the relevant hardware by a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed , which may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random storage memory RAM, magnetic disk or optical disk and other mediums that can store program codes.

Claims

A method for generating video, comprising:

The electronic device displays a first interface of the first application, where the first interface includes a first control and a second control;

After receiving the first operation acting on the first control, the electronic device determines that the arrangement order of the first material, the second material and the third material is the first order, and the first material, the first material and the third material are arranged in the first order. The second material and the third material are different image materials stored in the electronic device, the first order is different from the third order, and the third order is the first material, the second material and the the time sequence in which the third material is stored in the electronic device; and generating a first video from the first material, the second material and the third material according to the first order;

After receiving the second operation acting on the second control, the electronic device determines that the arrangement order of the first material, the second material and the third material is the second order, and the first material The second order is different from the third order; and according to the second order, a second video is generated from the first material, the second material and the third material.
The method according to claim 1, wherein the first video is divided into a plurality of segments by taking the beat of the music as a dividing line;

The first material, the second material and the third material appear at least once in the first video, and the materials appearing in any two adjacent segments of the first video are different;

The first material, the second material and the third material appear at least once in the second video, and the materials appearing in any two adjacent segments of the second video are different.
The method according to claim 1 or 2, wherein the method further comprises:

the electronic device displays the second interface of the first application;

After receiving the third operation acting on the second interface, the electronic device generates the first video from the first material, the second material and the third material.
The method according to claim 1 or 2, wherein the method further comprises:

The electronic device determines from the first material, the second material, the third material and the fourth material to generate the first material, the second material and the third material into the first video;

The fourth material is an image material stored in the electronic device that is different from the first material, the second material and the third material.
The method according to any one of claims 1-4, wherein the first interface further includes a third control; the method further includes:

After receiving the fourth operation acting on the third control, the electronic device displays a third interface, where the third interface includes: options of configuration information, and the configuration information includes: duration, filter, frame , at least one parameter in the material or title;

After the electronic device receives the fifth operation acting on the options of the configuration information, in the first order, based on the configuration information, the first material, the second material and the The third material generates a third video.
The method according to any one of claims 1-5, wherein the first interface further includes a fourth control; the method further includes:

After generating the first video, the electronic device saves the first video in response to a fourth operation acting on the fourth control.
The method according to any one of claims 1-6, wherein the method specifically comprises:

The electronic device determines the scene category type corresponding to the first material, the scene category type corresponding to the second material, and the scene category type corresponding to the third material;

The electronic device is set based on the scene type corresponding to the first material, the scene type corresponding to the second material, the scene type corresponding to the third material, and each segment in the first video template. Scene type, determine the material matching the scene type corresponding to the first segment, and the first segment is any segment in the first video template; and assign all segments corresponding to the first video template The arrangement order of the materials is the first order;

The electronic device is set based on the scene type corresponding to the first material, the scene type corresponding to the second material, the scene type corresponding to the third material, and each segment in the second video template. Scene type, determine the material that matches the scene type corresponding to the second segment, and the second segment is any segment in the second video template; The arrangement order of the materials is the second order;

Wherein, the first video template is different from the second video template, each segment in the second video is opposite to each segment in the second video template, and each segment in the second video A segment corresponds to each segment in the second video template.
The method according to claim 7, wherein the method further comprises:

The electronic device converts the first material, the second material, and all of the first material, the second material, and all of them according to the first sequence and the motion effect, speed effect, and transition effect set for each segment in the first video template. generating the first video from the third material;

The electronic device converts the first material, the second material, and all of the first material, the second material, and all of them into The third material generates the second video.
The method according to claim 7 or 8, wherein when the first material is a picture material, the method specifically comprises:

The scene type corresponding to the first material of the electronic device is the same as the scene type corresponding to the first segment, or the scene type corresponding to the first material is the same as the first segment according to a preset rule. When the ordering of the corresponding scene types is adjacent, determining the first material as a material matching the scene type corresponding to the first segment;

The scene type corresponding to the electronic device in the first material is the same as the scene type corresponding to the second segment, or the scene type corresponding to the first material is the same as the second segment according to a preset rule. When the sequence of the corresponding scene types is adjacent, the first material is determined to be a material matching the scene type corresponding to the second segment.
The method according to claim 7 or 8, wherein when the first material is a video material, the method specifically comprises:

The scene type corresponding to the fourth material of the electronic device is the same as the scene type corresponding to the first segment, or the scene type corresponding to the fourth material is the scene type corresponding to the first segment according to the preset rule. When the order of types is adjacent, and the duration of the fourth material is equal to the duration of the first segment, the fourth material is cut out from the first material, and the fourth material is determined to be the same as the first segment. The material that matches the scene type corresponding to the clip;

The scene type corresponding to the electronic device in the fourth material is the same as the scene type corresponding to the second segment, or the scene type corresponding to the fourth material is the scene type corresponding to the second segment according to the preset rule. When the order of types is adjacent, and the duration of the fourth material is equal to the duration of the second segment, the fourth material is cut out from the second material, and the fourth material is determined to be the same as the second segment. The material that matches the scene type corresponding to the clip;

Wherein, the fourth material is part or all of the first material.
The method according to claim 10, characterized in that, according to the sorting of the preset rules, the types of scenes include: close-up, medium-range, and distant, and the type of scene adjacent to the close-range is a distant-scape, which is the same as the close-range. The types of scenes adjacent to the medium shot are close shots and distant shots, and the types of scenes adjacent to the distant shots are close shots.
The method according to any one of claims 1-11, wherein the first application is a gallery application of the electronic device.
An electronic device, comprising: a memory and a processor;

the memory is used to store program instructions;

The processor is configured to invoke program instructions in the memory to cause the electronic device to execute the video generation method according to any one of claims 1-12.
A chip system, characterized in that, the chip system is applied to an electronic device including a memory, a display screen and a sensor; the chip system includes: a processor; when the processor executes computer instructions stored in the memory , the electronic device executes the video generation method according to any one of claims 1-12.
A computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to cause the electronic device to implement the video generation method according to any one of claims 1-12.
A program product, characterized in that the program product comprises a computer program, the computer program is stored in a readable storage medium, and at least one processor of a communication device can read the computer program from the readable storage medium , the at least one processor executing the computer program causes the communication device to implement the method according to any one of claims 1-12.