CN115706853A - Video processing method and device, electronic equipment and storage medium - Google Patents

Abstract

The embodiment of the application provides a video processing method and device, electronic equipment and a storage medium, relates to the technical field of video shooting, and can solve the problem of video interruption when video frame rates are changed in different scenes. The video processing method comprises the following steps: acquiring a video image captured by a camera; switching a video mode according to the image parameters of the current video image, wherein the switching the video mode according to the image parameters of the current video image comprises: if the overexposure ratio of the current video image is larger than a first overexposure ratio threshold value, entering a first video mode, and if the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value, entering a second video mode; in a first video mode, a camera captures video images by using a first frame rate, and the video images captured in any two adjacent frames have different exposure time; in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

Description

Video processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video shooting technologies, and in particular, to a video processing method and apparatus, an electronic device, and a storage medium.

Background

With the development of the technology, the requirements of users on the video effect shot by a terminal such as a mobile phone are higher and higher, and for video shooting, different frame rate requirements are provided based on different scenes, however, at present, in different scene, for different frame rate switching processes, restarting is needed to capture video streams with different frame rates, and video interruption is conducted.

Disclosure of Invention

A video processing method, a video processing device, an electronic device and a storage medium can solve the problem of video interruption when video frame rates are changed in different scenes.

In a first aspect, a video processing method is provided, including: acquiring a video image captured by a camera; switching the video mode according to the image parameters of the current video image, wherein the switching the video mode according to the image parameters of the current video image comprises the following steps: if the overexposure ratio of the current video image is larger than a first overexposure ratio threshold value, entering a first video mode, if the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value, entering a second video mode, and if the second overexposure ratio threshold value is smaller than the first overexposure ratio threshold value; in a first video mode, a camera captures video images at a first frame rate, the video images captured in any two adjacent frames have different exposure time, and the video images in each two adjacent frames are fused into an image of one frame to obtain a video image at a second frame rate; in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

In a possible implementation manner, if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, the process of entering the first video mode includes: if the brightness value of the current video image is larger than the first brightness threshold value and the overexposure ratio of the current video image is larger than the first overexposure ratio threshold value, entering a first video mode; if the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold, the process of entering the second video mode comprises the following steps: if the brightness value of the current video image is larger than the first brightness threshold value and the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold value, entering a second video mode, wherein the third frame rate is larger than the second frame rate; switching the video mode according to the image parameters of the current video image further comprises: if the brightness value of the current video image is smaller than the second brightness threshold value, entering a third video mode; in the third video mode, the camera captures video images using a fourth frame rate, and the video images captured in any two adjacent frames have the same exposure time, and the fourth frame rate is less than the third frame rate. Switching of video modes is performed according to different image parameters of a current video, and a frame rate can be dynamically adjusted according to a current video shooting scene to adapt to different scenes, for example, when a brightness value of a current video image is greater than a first brightness threshold value B1 and an overexposure ratio of the current video image is greater than a first overexposure ratio threshold value A1, the current video image belongs to a medium-high brightness HDR scene, so that the current video mode enters a first video mode and HDR video image processing at a lower frame rate (a second frame rate) is realized; when the brightness value of the current video image is greater than a first brightness threshold value B1 and the overexposure ratio of the current video image is less than a second overexposure ratio threshold value A2, the current video image belongs to a medium-high brightness non-HDR scene, so that a second video mode is entered, common video image processing with a higher frame rate (a third frame rate) is realized, and the smoothness of the video is improved by shortening the exposure time and improving the frame rate; the brightness value of the current video image is smaller than the second brightness threshold value B2 and belongs to a low-brightness scene, so that the third video mode is entered, the common video image processing with a lower frame rate (fourth frame rate) is realized, the light input quantity is improved by prolonging the exposure time and reducing the frame rate, and the picture effect under the low-brightness scene is improved.

In one possible embodiment, the third frame rate is equal to the first frame rate and the fourth frame rate is equal to the second frame rate. Therefore, no matter what video mode, the multiple frame rate is used for switching, so that the jitter problem caused by the non-multiple frame rate is solved.

In one possible embodiment, the first overexposure percentage threshold is 25% and the second overexposure percentage threshold is 20%; the first luminance threshold is 500nit and the second luminance threshold is 300nit.

In a possible implementation, before switching the video mode according to the image parameters of the current video image, the method further includes: if the current video image has the preset pattern, switching the video mode according to the preset pattern of the current video image, and if the current video image has no preset pattern, entering a process of switching the video mode according to the image parameters of the current video image; the process of switching the video mode according to the preset pattern of the current video image comprises the following steps: if the current video image has the preset pattern of the ticker, entering a second video mode; if the current video image has a preset pattern of a non-ticker and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold value, entering a first video mode; and if the current video image has the preset pattern which is not a horse race and the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold value, entering a third video mode. In addition to the scenes of the actual captured video, the video recording process of the electronic device may also be used in some test processes, which use a specific preset pattern for testing, so as to better match the specific test process, therefore, the video mode may be switched based on the preset pattern, combining the scenes and the over-exposure ratio, and the preset pattern of the ticker is a pattern for circular display of the test video frame rate, such as a circular lighting bulb pattern.

In a possible embodiment, if the current video image has no preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows: and if the current video image has no preset pattern and the electronic equipment is in a handheld state, entering a process of switching the video mode according to the image parameters of the current video image.

In a second aspect, a video processing apparatus is provided, including: a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the video processing method described above.

In a third aspect, an electronic device is provided, including: a camera; the video processing device is provided.

In a fourth aspect, a computer-readable storage medium is provided, in which a computer program is stored which, when run on a computer, causes the computer to perform the above-described video processing method.

In the video processing method, the video processing device, the electronic device and the storage medium in the embodiment of the application, in the first video mode, video images of adjacent frames with different exposure times, which are obtained by capturing with the first frame rate, are fused into one frame to obtain a video image with the second frame rate, and an HDR video image with a lower frame rate can be obtained.

Drawings

Fig. 1 is a block diagram of an electronic device according to an embodiment of the present application;

FIG. 2 is a flowchart of a video processing method according to an embodiment of the present application;

FIG. 3 is a schematic diagram illustrating a switching condition of different video modes according to an embodiment of the present application;

fig. 4 is a flowchart of a video processing method in a first video mode according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a video image capturing process in a first video mode according to an embodiment of the present application;

fig. 6 is a flowchart of a video processing method in the second video mode according to an embodiment of the present application;

FIG. 7 is a diagram illustrating a video image capture in a second video mode according to an embodiment of the present application;

fig. 8 is a flowchart of a video processing method in a third video mode according to an embodiment of the present application;

FIG. 9 is a diagram illustrating a video image capturing operation in a third video mode according to an embodiment of the present application;

FIG. 10 is a flow chart of another video processing method in an embodiment of the present application;

FIG. 11 is a block diagram of a software structure of an electronic device according to an embodiment of the present application;

FIG. 12 is a diagram illustrating a user interface in movie mode according to an embodiment of the present application;

fig. 13 is a schematic diagram of a user interface in the professional mode according to an embodiment of the present application.

Detailed Description

The terminology used in the description of the embodiments section of the present application is for the purpose of describing particular embodiments of the present application only and is not intended to be limiting of the present application.

Before describing the embodiments of the present application, first, referring to the electronic device according to the embodiments of the present application, as shown in fig. 1, the electronic device 100 may include a processor 110, a camera 193, a display screen 194, and the like. It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), and the like. Wherein, the different processing units may be independent devices or may be integrated in one or more processors. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution. A memory may also be provided in the processor 110 for storing instructions and data.

The electronic device 100 implements display functions via the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The electronic device 100 may implement a photographing function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV and other formats. In some embodiments, electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

As shown in fig. 2 and fig. 3, an embodiment of the present application provides a video processing method, where an execution subject of the video processing method may be a processor 110, and specifically may be an ISP or a combination of the ISP and another processor, and the video processing method includes:

step 101, acquiring a video image captured by a camera;

102, switching a video mode according to the image parameters of the current video image, wherein the switching the video mode according to the image parameters of the current video image comprises the following steps: if the overexposure ratio of the current video image is greater than a first overexposure ratio threshold value A1, entering a first video mode, if the overexposure ratio of the current video image is less than a second overexposure ratio threshold value A2, entering a second video mode, and if the second overexposure ratio threshold value A2 is less than the first overexposure ratio threshold value A1;

the overexposure ratio is a ratio of the number of pixels in the video image larger than a preset pixel value to the total number of pixels, for example, an 8-bit video image is taken as an example, the pixel value range of each pixel is from 0 to 255, the preset pixel value is 250, and the overexposure ratio is a ratio of the number of pixels with the pixel value larger than 250 to the total number of pixels. For example, the overexposure ratio of the current video image is greater than a first overexposure ratio threshold value A1, at this time, the video mode belongs to a first video mode, if the overexposure ratio of the video image is gradually reduced to be less than the first overexposure ratio threshold value A1 and greater than a second overexposure ratio threshold value A2, the video mode is not switched, and is still the first video mode, until the overexposure ratio of the video image is reduced to be less than the second overexposure ratio threshold value A2, the video mode is switched to enter a second video mode; similarly, in the second video mode, if the overexposure ratio of the video image is gradually increased to be smaller than the first overexposure ratio threshold value A1 and larger than the second overexposure ratio threshold value A2, the video mode is not switched, and is still the second video mode, and the video mode is switched to enter the first video mode until the overexposure ratio of the video image is increased to be larger than the first overexposure ratio threshold value A1. With such switching logic, ping-pong switching of different video modes at a single threshold can be avoided. It should be noted that, for the initial video mode determination logic, the embodiment of the present application is not limited, for example, an initial overexposure ratio threshold may be set between the first overexposure ratio threshold A1 and the second overexposure ratio threshold A2, before recording a video, if the overexposure ratio of the video image in the preview screen is greater than the initial overexposure ratio threshold, the first video mode is entered, and if the overexposure ratio of the video image in the preview screen is not greater than the initial overexposure ratio threshold, the second video mode is entered.

As shown in fig. 4 and 5, in the first video mode, the camera captures video images using a first frame rate, the video images captured in any two adjacent frames have different exposure times, and the video images in each two adjacent frames are fused into an image of one frame, so as to obtain a video image at a second frame rate;

for example, the electronic device may specifically include a camera 193, a Global Tone Mapping (GTM) module 20, a first warping module 31, a second warping module 32, a third warping module 33, a fourth warping module 34, a first fusion module 41, a second fusion module 42, a first gamma module 51, and a second gamma module 52, in the first video mode, assuming that the first frame rate is 60FPS, that is, the camera 193 captures video images using a frame rate of 60FPS, and alternately captures a first exposure frame video image F1 and a second exposure frame video image F2, where exposure times of any two adjacent frame video images are different, that is, exposure times of F1 and F2 are different, an exposure time of F1 is greater than that of F2, for example, 33ms is a length of two frame times, where the exposure time is a longer F1 and the exposure time is a shorter F2; the GTM module 20 performs GTM processing on the video Image captured by the camera 193, divides the video Image subjected to GTM processing into a video stream and a preview stream, in the video stream, the first exposure frame video Image F1 is subjected to Electronic Image Stabilization (EIS) processing by the first warping module 31, the second exposure frame video Image F2 is subjected to EIS processing by the second warping module 32, and the first exposure frame video Image F1 and the second exposure frame video Image F2 subjected to EIS processing are subjected to High-Dynamic Range Image (High-Dynamic Range, HDR), fusing the first exposure frame video Image F1 and the second exposure frame video Image F2 in each two adjacent frames into a frame Image, so that the frame rate is reduced to half of the original frame rate after fusing, that is, the frame rate is changed into an Image of 30FPS, the fused video Image is subjected to gamma mapping processing by the first gamma module 51, and the processed Image of the video stream can be stored as a video file, that is, the HDR video stream processing process of 30FPS is realized, that is, a video Image with the second frame rate of 30FPS is obtained; in addition, in the preview stream, similar to the processing process of the video stream, the first exposure frame video image F1 is subjected to the EIS processing by the third warping module 33, and the second exposure frame video image F2 is subjected to the EIS processing by the fourth warping module 34, it should be noted that, since the preview stream is only used for previewing, the EIS processing performed in the preview stream may be different from the EIS processing performed in the video stream, for example, a higher version of the EIS processing is performed in the video stream, and a lower version of the EIS processing is performed in the preview stream, and the first exposure frame video image F1 and the second exposure frame video image F2 subjected to the EIS processing in the preview stream are subjected to the HDR fusion processing in the second fusion module 42, and the first exposure frame video image F1 and the second exposure frame video image F2 in every two adjacent frames are fused into one frame image, so that the frame rate after the fusion is reduced to half of the original frame rate, that the first exposure frame image is changed into an image of the FPS, and the video image after the fusion is subjected to the second mapping processing 52, and the gamma processing of the video stream, that the video stream is subjected to the video stream, and the HDR processing of the video stream, that the gamma processing of the video stream is implemented as the HDR processing of the video stream, that the video stream, that the gamma processing of the HDR processing of the video stream can be implemented.

As shown in fig. 6 and 7, in the second video mode, the camera captures video images using the third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

For example, in the second video mode, assuming that the third frame rate is 60FPS, that is, the camera 193 captures video images using a frame rate of 60FPS, each frame of the captured exposure frame video images F has an equal exposure time, for example, 33ms is a length of two frame times, which includes two exposure frame video images F with equal exposure times; the GTM module 20 performs GTM processing on a video Image captured by the camera 193, and splits the video Image subjected to GTM processing, wherein one of the streams is a video stream, the other stream is a preview stream, in the video stream, an exposure frame video Image F is subjected to Electronic Image Stabilization (EIS) processing by the first warping module 31, the exposure frame video Image F subjected to the EIS processing no longer passes through the first fusion module 41, and is subjected to gamma mapping processing by the first gamma module 51, and an Image of the processed video stream can be stored as a video file, that is, a common video stream processing process of 60FPS is realized, so as to obtain a video Image of 60 FPS; in addition, in the preview stream, similar to the processing procedure of the video stream, the exposure frame video image F is subjected to the EIS processing by the third morphing module 33, it should be noted that, since the preview stream is only used for previewing, the EIS processing performed in the preview stream may be different from the EIS processing performed in the video stream, for example, the EIS processing of a higher version is performed in the video stream, while the EIS processing of a lower version is performed in the preview stream, the exposure video image F after the EIS processing in the preview stream bypasses the second fusion module 42, and then is subjected to the gamma mapping processing by the second gamma module 52, and the image of the processed video stream may be previewed, that is, the preview stream processing procedure of 60FPS is implemented. In other realizable embodiments, the third frame rate may be adjusted between the first frame rate and the second frame rate, for example, assuming that the first frame rate is 60FPS, the second frame rate is 30FPS, and the third frame rate may be 30FPS, 60FPS, or between 30FPS and 60 FPS.

In the video processing method in the embodiment of the application, in the first video mode, video images of adjacent frames with different exposure times, which are captured by using the first frame rate, are fused into one frame to obtain a video image of the second frame rate, so that an HDR video image of a lower frame rate can be obtained, in the second video mode, the video image with the same exposure time, which is captured by using the third frame rate, is taken as a video stream to be processed, so that a non-fused ordinary video image can be obtained, the different video modes can be switched according to image parameters of the current video image, and in the switching process, the setting when the video image is captured by the camera does not need to be changed, so that the video stream does not need to be captured by restarting the camera, and the problem of video recording interruption when the video frame rate is changed in different scenes is avoided.

In a possible embodiment, as shown in fig. 3, if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold A1, the entering into the first video mode includes: if the brightness value of the current video image is greater than a first brightness threshold B1 and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold A1, entering a first video mode;

if the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value A2, the process of entering the second video mode comprises the following steps: if the brightness value of the current video image is greater than a first brightness threshold value B1 and the overexposure ratio of the current video image is less than a second overexposure ratio threshold value A2, entering a second video mode, wherein a third frame rate is greater than a second frame rate;

switching the video mode according to the image parameters of the current video image further comprises:

if the brightness value of the current video image is smaller than a second brightness threshold value B2, entering a third video mode;

as shown in fig. 8 and 9, in the third video mode, the camera captures video images using a fourth frame rate, and the video images captured in any two adjacent frames have the same exposure time, the fourth frame rate being less than the third frame rate.

For example, in the third video mode, assuming that the fourth frame rate is 30FPS, that is, the camera 193 captures video images using a frame rate of 30FPS, and each frame of the captured exposure frame video images F has an equal exposure time, for example, 33ms is a length of one frame time, that is, a video image F with a normal 30FPS frame rate is obtained; the GTM module 20 performs GTM processing on a video Image F captured by the camera 193, and splits the video Image subjected to GTM processing, where one of the streams is a video stream, and the other stream is a preview stream, and in the video stream, the video Image F is subjected to Electronic Image Stabilization (EIS) processing by the first deformation module 31, the video Image F after the EIS processing does not pass through the first fusion module 41 any more, and is subjected to gamma mapping processing by the first gamma module 51, and an Image of the processed video stream can be stored as a video file, that is, a normal video stream processing process of 30FPS is implemented, so as to obtain a normal video Image with a frame rate of 30 FPS; in addition, in the preview stream, similar to the processing procedure of the video stream, the video image F is subjected to the EIS processing by the third morphing module 33, it should be noted that, since the preview stream is only used for previewing, the EIS processing performed in the preview stream may be different from the EIS processing performed in the video stream, for example, a higher version of EIS processing is performed in the video stream, and a lower version of EIS processing is performed in the preview stream, and the video image F after the EIS processing in the preview stream bypasses the second fusion module 42 and then undergoes the gamma mapping processing by the second gamma module 52, and the image of the processed video stream may be previewed, that is, the preview stream processing procedure of 30FPS is implemented.

It should be further noted that, for the initial video mode determination logic, embodiments of the present application are not limited, for example, an initial brightness threshold may be set between the first brightness threshold B1 and the second brightness threshold B2, before recording a video, if the brightness of the video image in the preview picture is greater than the initial brightness threshold, the first video mode or the second video mode is entered, specifically, determination is performed in combination with an overexposure ratio, and if the brightness of the video image in the preview picture is not greater than the initial brightness threshold, the third video mode is entered.

Switching of video modes is performed according to different image parameters of a current video, and a frame rate can be dynamically adjusted according to a current video shooting scene to adapt to different scenes, for example, when a brightness value of a current video image is greater than a first brightness threshold value B1 and an overexposure ratio of the current video image is greater than a first overexposure ratio threshold value A1, the current video image belongs to a medium-high brightness HDR scene, so that the current video mode enters a first video mode and HDR video image processing at a lower frame rate (a second frame rate) is realized; when the brightness value of the current video image is greater than a first brightness threshold value B1 and the overexposure ratio of the current video image is less than a second overexposure ratio threshold value A2, the current video image belongs to a medium-high brightness non-HDR scene, so that a second video mode is entered, common video image processing with a higher frame rate (a third frame rate) is realized, and the smoothness of the video is improved by shortening the exposure time and improving the frame rate; the brightness value of the current video image is smaller than the second brightness threshold value B2 and belongs to a low-brightness scene, so that the third video mode is entered, the common video image processing with a lower frame rate (fourth frame rate) is realized, the light input quantity is increased by prolonging the exposure time and reducing the frame rate, and the picture effect under the low-brightness scene is improved. And the different video modes are smoothly switched to avoid the problem of frequent ping-pong switching.

In one possible embodiment, the third frame rate is equal to the first frame rate and the fourth frame rate is equal to the second frame rate. For example, the first and fourth frame rates are 60FPS, and the second and third frame rates are 30FPS. In this way, no matter what video mode, the multiple frame rate is used for switching, so that the jitter problem caused by the non-multiple frame rate is improved.

In a possible embodiment, the first overexposure proportion threshold value A1 is 25% and the second overexposure proportion threshold value A2 is 20%; the first luminance threshold B1 is 500nit and the second luminance threshold B2 is 300nit.

In a possible implementation manner, as shown in fig. 10, before switching the video mode according to the image parameters of the current video image in step 102, the method further includes:

step 103, determining whether the current video image has a preset pattern, if so, switching to a video mode according to the preset pattern of the current video image, otherwise, switching to a video mode according to the image parameter of the current video image, if not, switching to a video mode according to the preset pattern of the current video image;

step 104, the process of switching the video mode according to the preset pattern of the current video image comprises:

if the current video image has the preset pattern of the ticker, entering a second video mode;

if the current video image has a preset pattern of a non-ticker and the overexposure ratio of the current video image is greater than a first overexposure ratio threshold value, entering a first video mode;

and if the current video image has the preset pattern of the non-ticker and the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold value, entering a third video mode.

Specifically, besides the scene of the actual shot video, the video recording process of the electronic device may also be used in some test processes, the test process may use a specific preset pattern for testing, and in order to better match the specific test process, therefore, the video mode may be switched based on the preset pattern in combination with the scene and the overexposure ratio, and the preset pattern of the ticker is a pattern for circular display of the test video frame rate, such as a circular lighting bulb pattern.

In a possible embodiment, if the current video image has no preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows: if the current video image has no preset pattern and the electronic device is in a handheld state, the process of switching the video mode according to the image parameters of the current video image is performed in step 102. The determination of the handheld state may be performed according to data of a sensor in the electronic device, such as a gyroscope, and because the electronic device cannot be kept stable in the handheld state, if it is determined that the electronic device is in a shaking process according to the relevant sensor, it is determined that the electronic device is in the handheld state, and in this state, the above step 102 is performed. In a handheld state, through the frame rate dynamic adjustment in the embodiment of the application, the video can have higher smoothness.

In a possible implementation manner, the electronic device includes a main camera, a telephoto camera, and an ultra-wide angle camera, and if the current camera is the main camera and is in a non-zoom state, that is, in a single-zoom state, step 102 is performed, that is, dynamic frame rate adjustment is performed, and if the current camera is the main camera and is in the non-single-zoom state, or if the current camera is the non-main camera, a video mode with a fixed frame rate is used.

The embodiments of the present application are described below with reference to a software architecture, and the embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe a software structure of the electronic device 100. Fig. 11 is a software configuration block diagram of the electronic device 100 according to the embodiment of the present application.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into five layers, which are an Application Layer, an Application framework Layer, a system library, a Hardware Abstraction Layer (HAL), and a kernel Layer from top to bottom.

The application layer may include a camera or like application.

The Application framework layer may include an Application Programming Interface (API), a media recorder, a surface view, and the like. Media recording is used to record video or picture data and make the data accessible to applications. The surface view is used to display a preview screen.

The system library may include a plurality of functional modules. For example: camera service cameraview, etc.

The hardware abstraction layer is used to provide interface support, for example, including camera flow CameraPipeline for the camera service to Call the Call.

The kernel layer is a layer between hardware and software. The core layer includes a display driver, a camera driver, and the like.

In combination with a specific scene of a captured video, an application layer issues a capture request CaptureRequest which corresponds to a video stream and a preview stream. And meanwhile, a media codec instance is created, and the coding of the video stream is received. HAL recalls the two streams according to the data flow described above. Wherein, the streaming display is previewed, and the mediacodec streaming is recorded.

The video processing method provided by the embodiment of the application can be expressed as a plurality of functions in two shooting modes, wherein the two shooting modes can be as follows: movie mode, professional mode.

The movie mode is a shooting mode related to a movie theme in which the images displayed by the electronic device 100 can perceptually give the user an effect of watching a movie, and the electronic device 100 further provides a plurality of video style templates related to the movie theme, and the user can obtain the image or video with adjusted color tone by using the video style templates, and the color tone of the image or video is similar to or identical to the color tone of the movie. In the following embodiments of the present application, the movie mode may provide at least an interface for the user to trigger the LUT function, the HDR10 function. For a description of the LUT function, the HDR10 function in particular, see the following embodiments.

For example, assuming that the electronic device 100 is a mobile phone, in one possible embodiment, as shown in fig. 12, the electronic device may enter a movie mode in response to a user operation. For example, the electronic device 100 may detect a touch operation by a user on a camera application, and in response to the operation, the electronic device 100 displays a default photographing interface of the camera application. The default photographing interface may include: preview boxes, shooting mode lists, gallery shortcut keys, shutter controls, and the like. Wherein:

the preview pane may be used to display images acquired by the camera 193 in real time. The electronic device 100 may refresh the display content therein in real-time to facilitate the user to preview the image currently captured by the camera 193.

One or more shooting mode options may be displayed in the shooting mode list. The one or more shooting mode options may include: portrait mode option, video mode option, photo mode option, movie mode option, professional option. The one or more shooting mode options may be presented on the interface as textual information, such as "portrait", "video", "take", "movie", "professional". Without limitation, the one or more shooting mode options may also appear as icons or other forms of Interactive Elements (IEs) on the interface.

The gallery shortcut may be used to open a gallery application. The gallery application is an application for managing pictures on electronic devices such as smart phones and tablet computers, and may also be referred to as "albums," and this embodiment does not limit the name of the application. The gallery application may support various operations, such as browsing, editing, deleting, selecting, etc., by the user on the pictures stored on the electronic device 100.

The shutter control can be used to monitor user operations that trigger a photograph. The electronic device 100 may detect a user operation acting on the shutter control, in response to which the electronic device 100 may save the image in the preview box as a picture in the gallery application. In addition, the electronic device 100 may also display thumbnails of the saved images in the gallery shortcut. That is, the user may click on the shutter control to trigger the taking of a picture. The shutter control may be a button or other form of control, among others.

The electronic device 100 may detect a touch operation by the user on the movie mode option, and in response to the operation, the electronic device displays a user interface as shown in fig. 12.

In some embodiments, the electronic device 100 may default to the movie mode on after launching the camera application. Without limitation, the electronic device 100 may also turn on the movie mode in other manners, for example, the electronic device 100 may also turn on the movie mode according to a voice instruction of a user, which is not limited in this embodiment of the application.

The electronic device 100 may detect a touch operation by the user on the movie mode option, and in response to the operation, the electronic device displays a user interface as shown in fig. 12.

The user interface as shown in fig. 12 includes function options including HDR10 options, flash options, LUT options, settings options. The plurality of function options may detect a touch operation by a user, and in response to the operation, turn on or off a corresponding photographing function, for example, an HDR10 function, a flash function, an LUT function, a setting function.

The electronic device may turn on a LUT function that may change the display effect of the preview image. In essence, the LUT function introduces a color lookup table, which corresponds to a color conversion model that is capable of outputting adjusted color values based on input color values. The color value of the image collected by the camera is equivalent to the input value, and different color values can all correspondingly obtain an output value after passing through the color conversion model. Finally, the image displayed in the preview frame is the image adjusted by the color conversion model. The electronic device 100 displays an image composed of color values adjusted by the color conversion model using the LUT function, thereby achieving an effect of adjusting the color tone of the image. After turning on the LUT function, the electronic device 100 may provide a plurality of video style templates, where one video style template corresponds to one color conversion model, and different video style templates may bring different display effects to the preview image. Moreover, the video style templates can be associated with the theme of the movie, and the tone adjustment effect brought to the preview image by the video style templates can be close to or the same as the tone in the movie, so that the atmosphere feeling of shooting the movie is created for the user.

In addition, after the electronic device 100 turns on the LUT function, the electronic device 100 may determine a video style template from a plurality of video style templates according to the current preview video frame, and the determined video style template may be displayed in the interface, so that the user can know the currently determined video style template, for example, the plurality of video style templates includes an "a" movie style template, a "B" movie style template, and a "C" movie style template, and LUTs corresponding to different movie style templates may be generated in advance based on corresponding movie color matching styles, and color conversions of the LUTs have style characteristics of corresponding movies. It can be extracted from the movie genre in advance to generate a LUT suitable for the mobile electronic device. Turning on the LUT function changes the color tone of the preview video picture. As illustrated in fig. 12, the electronic device 100 determines and displays an "a" movie genre template.

In some embodiments, the electronic device 100 may select the video style template according to a sliding operation by the user. Specifically, after the electronic device 100 detects a user operation of turning on the LUT function by the user and displays the LUT preview window, the electronic device 100 may default to select a first video style template located in the LUT preview window as the video style template selected by the electronic device 100. After that, the electronic device 100 may detect a left-right sliding operation performed by the user on the LUT preview window, move the position of each video style template in the LUT preview window, and when the electronic device 100 no longer detects the sliding operation by the user, the electronic device 100 may use the first video style template displayed in the LUT preview window as the video style template selected by the electronic device 100.

In some embodiments, in addition to changing the display effect of the preview image by using the video style template, the electronic device 100 may detect a user operation of starting to record the video after adding the video style template, and in response to the user operation, the electronic device 100 starts to record the video, so as to obtain the video with the display effect adjusted by using the video style template. In addition, during the process of recording the video, the electronic device 100 may further detect a user operation of taking a picture, and in response to the user operation, the electronic device 100 saves the preview image with the video style template added to the preview frame as a picture, thereby obtaining an image with the display effect adjusted by using the video style template.

The electronic device can start an HDR10 function, in the HDR10 mode, the HDR is a High-Dynamic Range image (HDR), compared with an ordinary image, the HDR can provide more Dynamic ranges and image details and better reflect visual effects in a real environment, 10 of the HDR10 is 10 bits, and the HDR10 can record videos in 10-bit High Dynamic ranges.

The electronic device 100 may detect a touch operation performed by the user on the professional mode option, and enter the professional mode. As shown in fig. 13, when the electronic device is in the professional mode, the function options included in the user interface may be, for example: LOG option, flash option, LUT option, setup option, and in addition, the user interface also includes parameter adjustment options, such as: photometry M option, ISO option, shutter S option, exposure compensation EV option, focusing mode AF option, and white balance WB option.

In some embodiments, device 100 may default to the professional mode upon launching the camera application. Without limitation, the electronic device 100 may also turn on the professional mode in other manners, for example, the electronic device 100 may also turn on the professional mode according to a voice instruction of a user, which is not limited in this embodiment of the present application.

The electronic apparatus 100 may detect a user operation of the user on the LOG option, and in response to the operation, the electronic apparatus 100 turns on the LOG function. The LOG function can apply a logarithmic function to the exposure curve, so that details of highlight and shadow parts in an image acquired by a camera are retained to the maximum extent, and the saturation of a finally presented preview image is low. Among them, a video recorded using the LOG function is called a LOG video.

The electronic device 100 may record, through the professional mode, not only the video to which the video style template is added, but also add the video style template to the video after recording the video to which the video style template is not added, or record the LOG video after starting the LOG function, and then add the video style template to the LOG video. In this way, the electronic device 100 can not only adjust the display effect of the picture before recording the video, but also adjust the display effect of the recorded video after the video is recorded, thereby increasing the flexibility and the degree of freedom of image adjustment.

An embodiment of the present application further provides a video processing apparatus, including: the video acquisition module is used for acquiring a video image captured by the camera; the switching module is used for switching the video mode according to the image parameters of the current video image, and the switching the video mode according to the image parameters of the current video image comprises the following steps: if the overexposure ratio of the current video image is larger than a first overexposure ratio threshold value, entering a first video mode, if the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value, entering a second video mode, and if the second overexposure ratio threshold value is smaller than the first overexposure ratio threshold value; in a first video mode, a camera captures video images at a first frame rate, the video images captured in any two adjacent frames have different exposure time, and the video images in each two adjacent frames are fused into an image of one frame to obtain a video image at a second frame rate; in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

It should be understood that the above division of the modules of the video processing apparatus is only a logical division, and the actual implementation may be wholly or partially integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or may be implemented entirely in hardware; and part of the modules can be realized in the form of software called by the processing element, and part of the modules can be realized in the form of hardware. For example, either one of the video obtaining module and the switching module may be a processing element that is set up separately, or may be integrated in the video processing apparatus, for example, be implemented in a chip of the video processing apparatus, or may be stored in a memory of the video processing apparatus in the form of a program, and a processing element of the video processing apparatus calls and executes the functions of the above modules. Other modules are implemented similarly. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.

For example, the video acquisition module and the switching module may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), among others. For another example, when some of the above modules are implemented in the form of a Processing element scheduler, the Processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling programs. As another example, these modules may be integrated together, implemented in the form of a system-on-a-chip (SOC).

An embodiment of the present application further provides a video processing apparatus, including: a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the video processing method of any of the above embodiments.

The video processing apparatus may apply the video processing method, and the detailed process and principle are not described herein again.

The number of processors may be one or more, and the processors and memory may be connected by a bus or other means. The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the video processing apparatus in the embodiments of the present application. The processor executes various functional applications and data processing by executing non-transitory software programs, instructions and modules stored in the memory, i.e., implements the methods in any of the above-described method embodiments. The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; and necessary data, etc. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device.

As shown in fig. 1, an embodiment of the present application further provides an electronic device, including: a camera 193 and the video processing device described above, the video processing device including the processor 110.

The specific principle and operation process of the video processing apparatus are the same as those of the above embodiments, and are not described herein again. The electronic device can be any product or component with a video shooting function, such as a mobile phone, a television, a tablet computer, a watch, a bracelet and the like.

Embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored, and when the computer program runs on a computer, the computer is caused to execute the video processing method in any of the above embodiments.

In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, and indicates that three relationships may exist, for example, a and/or B, and may indicate that a exists alone, a and B exist simultaneously, and B exists alone. Wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (9)

1. A video processing method, comprising:

acquiring a video image captured by a camera;

switching a video mode according to an image parameter of a current video image, wherein the switching the video mode according to the image parameter of the current video image comprises:

if the overexposure ratio of the current video image is larger than a first overexposure ratio threshold value, entering a first video mode, and if the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value, entering a second video mode, wherein the second overexposure ratio threshold value is smaller than the first overexposure ratio threshold value;

in the first video mode, the camera captures video images at a first frame rate, the video images captured in any two adjacent frames have different exposure time, and the video images in each two adjacent frames are fused into an image of one frame to obtain a video image at a second frame rate;

in the second video mode, the camera captures video images using a third frame rate, and the video images captured in any two adjacent frames have the same exposure time.

2. The video processing method according to claim 1,

if the overexposure ratio of the current video image is greater than the first overexposure ratio threshold, the process of entering the first video mode includes:

if the brightness value of the current video image is larger than a first brightness threshold value and the overexposure ratio of the current video image is larger than a first overexposure ratio threshold value, entering the first video mode;

if the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold, the process of entering the second video mode comprises the following steps:

if the brightness value of the current video image is larger than the first brightness threshold value and the overexposure ratio of the current video image is smaller than a second overexposure ratio threshold value, entering a second video mode, wherein the third frame rate is larger than the second frame rate;

the switching the video mode according to the image parameters of the current video image further comprises:

if the brightness value of the current video image is smaller than the second brightness threshold value, entering a third video mode;

in the third video mode, the camera captures video images at a fourth frame rate, the video images captured in any two adjacent frames have the same exposure time, and the fourth frame rate is smaller than the third frame rate.

3. The video processing method according to claim 2,

the third frame rate is equal to the first frame rate, and the fourth frame rate is equal to the second frame rate.

4. The video processing method according to claim 2,

the first overexposure proportion threshold value is 25%, and the second overexposure proportion threshold value is 20%;

the first luminance threshold value is 500nit, and the second luminance threshold value is 300nit.

5. The video processing method according to claim 2,

before the switching the video mode according to the image parameters of the current video image, the method further comprises the following steps:

if the current video image has the preset pattern, switching the video mode according to the preset pattern of the current video image, and if the current video image has no preset pattern, entering the process of switching the video mode according to the image parameters of the current video image;

the process of switching the video mode according to the preset pattern of the current video image comprises the following steps:

if the current video image has the preset pattern of the ticker, entering the second video mode;

if the current video image has a preset pattern of a non-ticker and the overexposure ratio of the current video image is greater than the first overexposure ratio threshold value, entering the first video mode;

and if the current video image has a non-horse race preset pattern and the overexposure ratio of the current video image is smaller than the second overexposure ratio threshold value, entering the third video mode.

6. The video processing method according to claim 5,

if the current video image has no preset pattern, the process of switching the video mode according to the image parameters of the current video image is as follows:

and if the current video image has no preset pattern and the electronic equipment is in a handheld state, entering a process of switching the video mode according to the image parameters of the current video image.

7. A video processing apparatus, comprising:

a processor and a memory for storing at least one instruction which is loaded and executed by the processor to implement the video processing method of any of claims 1 to 6.

8. An electronic device, comprising:

a camera;

the video processing apparatus of claim 7.

9. A computer-readable storage medium, in which a computer program is stored which, when run on a computer, causes the computer to perform the video processing method according to any one of claims 1 to 6.

Images