WO2023016014A1

WO2023016014A1 - Video editing method and electronic device

Info

Publication number: WO2023016014A1
Application number: PCT/CN2022/093042
Authority: WO
Inventors: 吴孟函
Original assignee: 荣耀终端有限公司
Priority date: 2021-08-12
Filing date: 2022-05-16
Publication date: 2023-02-16
Also published as: CN114222187A; CN114222187B

Abstract

A video editing method. The method can be applied to electronic devices capable of image processing, such as smart phones and tablet computers. During editing an HDR video using an SDR video editor, when the SDR video editor cannot display video frames of the HDR video normally, for example, the display is not clear, an electronic device can use color correspondences constructed on the basis of a color lookup table to convert the color gamut of the HDR video being edited into a color gamut that the SDR video editor can normally display, such that the SDR video editor can display the HDR video frames more normally.

Description

Video editing method and electronic device

This application claims the priority of the Chinese patent application submitted to the China Patent Office on August 12, 2021, with the application number 202110927488.2 and the application name "A Method for Converting HDR Video to SDR Video for Video Editing", and on November 2021 The priority of the Chinese patent application with the application number 202111329478.5 and the application title "Video Editing Method and Electronic Device" submitted to the China Patent Office on March 10. The entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the field of terminals, in particular to a video editing method and electronic equipment.

Background technique

At present, most smart terminals such as mobile phones and tablet computers can support shooting high-dynamic range (High-Dynamic Range, HDR) video. HDR video includes richer color effects and can record more image details, so that the video can present an excellent viewing effect. However, due to the limitation of the graphics processing capability of the chip platform, many smart terminals cannot support editing HDR video. This makes it impossible for the user to edit the shot video after shooting the HDR video, such as adding a video filter, etc., thereby affecting the user experience.

Contents of the invention

The present application provides a video editing method and electronic equipment, implementing the above video editing method, when the video to be edited is an HDR video, the electronic equipment can convert the HDR video of the BT2020 color gamut into the SDR video of the BT709 color gamut, so that When using the SDR video editor to edit HDR video, the video to be edited can also be displayed normally.

In a first aspect, the present application provides a video editing method, which is applied to an electronic device, and the method includes: detecting a first user operation, the first user operation corresponds to an editing control, and is used to trigger a video editing service; responding Based on the first user operation, the first video is decoded into N first video frames, and the color gamut of the N first video frames is the first color gamut; the color gamut conversion is performed on the N first video frames to obtain N first video frames Two video frames, the color gamut of the N second video frames is the second color gamut, the first color gamut is different from the second color gamut; any one of the N second video frames is displayed on the first interface.

By implementing the video editing method provided in the first aspect, the electronic device can convert the video in the first color gamut to the video in the second color gamut, and then display the video in the second color gamut. In this way, when the video editor of the electronic device cannot normally display the video to be edited in the first color gamut, the electronic device can convert the color gamut of the video to be edited from the first color gamut to the second color gamut, so that the editing The video browser can normally display the video frame of the video to be edited without problems such as unclear display affecting the user experience.

In combination with the method provided in the first aspect, in some embodiments, the range of colors that can be represented by the second color gamut is smaller than the range of colors that can be represented by the first color gamut.

In combination with the method provided in the first aspect, in some embodiments, the color gamut conversion is performed on the N first video frames, specifically: performing color gamut conversion on the N first video frames by using the first color table, and the first color table Contains several color values used to alter the color gamut of the video frame.

By implementing the methods provided in the foregoing embodiments, the electronic device can use the first color table that provides a color conversion relationship to convert a video in the first color gamut into a video in the second color gamut. A color table whose color values are within the range of the second color gamut can be used to convert the video of the first color gamut to the video of the second color gamut. Therefore, there are various types of the first color table, and there are also various methods for using the first color table to realize color value correspondence and thereby realize color gamut conversion.

With reference to the method provided in the first aspect, in some embodiments, after displaying N second video frames on the first interface, the method further includes: detecting a second user operation, where the second user operation is to change the second video frame selected by the user. An editing operation of a video display effect; in response to a second user operation, increase or decrease the number of second video frames, and/or change the number of pixels of one or more second video frames in the N second video frames , and/or, change the color values of the pixels of one or more second video frames in N second video frames to obtain M third video frames; M and N are equal or not equal; M is displayed on the first interface Any one of the third video frames.

Implementing the method provided by the above embodiment, the electronic device can also detect editing operations selected by the user to change the display effect of the first video, such as splitting the video, deleting video frames, adding a title or credits, cropping the screen size, adding filters, etc. wait. At the same time, the electronic device can perform the above-mentioned editing operation, and display the video frame after the above-mentioned editing and copying. In this way, whenever the user selects an editing operation, the user can immediately see the style of the video after the editing operation is performed.

In combination with the method provided in the first aspect, in some embodiments, the method further includes: detecting a third user operation, where the third user operation corresponds to a save control; in response to the third user operation, saving M third video frames as The second video, where the second video is an edited video obtained by editing the first video; the second video is displayed on the second interface.

Implementing the method provided by the above embodiment, the electronic device can detect the user operation of saving the video, and in response to the operation, the electronic device can package the edited video frame into a video and save it in the local storage space, For users to browse, forward, etc. at any time.

In combination with the method provided in the first aspect, in some embodiments, the editing operation selected by the user to change the display effect of the first video includes: splitting the video, deleting video frames, adding a title or credits, cropping the screen size, adding filters, One or more of the operations of adding text or graphics; among them, the operation of splitting the video, deleting the video frame, adding the beginning or end of the credits is used to increase or decrease the number of the second video frame, and the operation of cropping the screen size is used to change N The number of pixels in one or more second video frames in the second video frame, the operation of adding a filter, adding text or graphics is used to change the pixel points of one or more second video frames in the N second video frames color value.

In combination with the method provided in the first aspect, in some embodiments, when the second user operation includes an operation to increase or decrease the number of second video frames, then M and N are not equal; when the second user operation does not include an increase or decrease When operating on the number of second video frames, then M and N are equal.

In combination with the method provided in the first aspect, in some embodiments, the first color gamut is BT2020, and the second color gamut is BT709.

In this way, the electronic device can convert the video to be edited with a color gamut of BT2020 into a video with a color gamut of BT709. When the video editor used by the electronic device does not support the display of the video to be edited with a color gamut of BT2020, the electronic device can display the converted video with a color gamut of BT709, so that the electronic device can normally display the video of the video to be edited Frames, and there will be no problems such as unclear display that affect the user experience.

With reference to the method provided in the first aspect, in some embodiments, the first video is a high dynamic range HDR video, and the second video is a standard dynamic range SDR video.

In this way, the electronic device can convert the HDR video to be edited into the SDR video. When the video editor used by the electronic device does not support the display of HDR video, the electronic device can display the above-mentioned converted SDR video, so that the electronic device can normally display the video frame of the video to be edited without unclear display, etc. Issues affecting user experience.

In combination with the method provided in the first aspect, in some embodiments, the electronic device includes a video editing application APP, a decoder, and a first memory, where the first memory is a storage space for buffering video input by the APP in the encoder, and the first Decoding the video into N first video frames specifically includes: the APP sends the first video to the first memory; the decoder reads the first video from the first memory, and decomposes the first video into N first video frames.

In combination with the method provided in the first aspect, in some embodiments, the electronic device further includes an open graphics library OpenGL and a graphics processor GPU, and uses the first color table to perform color gamut conversion on the N first video frames to obtain N second video frames. The video frame specifically includes: OpenGL receives the N first video frames sent by the decoder, the color coding format of the N first video frames is YUV format, and the data type of the data representing the color value is an integer; OpenGL converts the N first video frames The color coding format of the first video frame is changed to RGB format, and the data type of the data representing the color value is changed to floating point type; OpenGL calls the first color table from the GPU, and uses the color value conversion relationship provided by the first color table to modify N The color values of the pixels of the first video frames are obtained to obtain N second video frames; the color coding format of the N second video frames is RGB format, and the data type of the data representing the color value is a floating point type.

With reference to the method provided in the first aspect, in some embodiments, before OpenGL calls the first color table from the GPU, the method further includes: OpenGL obtains the first color table from the APP, and loads the first color table into the GPU.

In combination with the method provided in the first aspect, in some embodiments, using the color value conversion relationship provided by the first color table to modify the color values of the pixels of the N first video frames specifically includes: OpenGL determining the current color value of the first pixel The color value Q1, the data type of Q1 is an integer, the first pixel is any pixel in any one of the N first video frames; OpenGL takes the position of Q1 in the three-dimensional color space as the origin, and determines the same value as Q1 The 7 auxiliary color values that constitute the space cube; OpenGL determines the respective index values of Q1 and the 7 auxiliary color values in the first color table; OpenGL queries the corresponding Q1 and 7 auxiliary colors in the first color table according to the index values The target color value of the value; OpenGL interpolates the target color value of Q1 and the 7 auxiliary color values to obtain the changed color value, and sets the color value of the first pixel as the changed color value.

In combination with the method provided in the first aspect, in some embodiments, the calculation formula of the index value W of a color value (R, G, B) in the first color table is: W=R*33 ² +G*33 ¹ + B*33 ⁰ .

In conjunction with the method provided in the first aspect, in some embodiments, displaying any one of the N second video frames on the first interface specifically includes: the GPU sends the first video memory address to OpenGL, and the first video memory address is to store N The video memory address of the second video frame; OpenGL sends the first video memory address to the APP; the APP obtains N second video frames according to the first video memory address; the APP displays the N second video frames obtained through the first video memory address on the first interface. Any one of the video frames.

In combination with the method provided in the first aspect, in some embodiments, in response to a second user operation, increase or decrease the number of second video frames, and/or change one or more second video frames in the N second video frames The number of pixels of the frame, and/or, change the color value of the pixels of one or more second video frames in the N second video frames to obtain M third video frames, specifically including: the APP converts the second user The operation is sent to OpenGL; OpenGL determines according to the second user operation: realize the calculation logic of the video display effect indicated by the second user operation; OpenGL sends the calculation logic to the GPU; the GPU increases or decreases the number of second video frames according to the calculation logic, and /or, change the number of pixels of one or more second video frames in the N second video frames, and/or, change the color of the pixels of one or more second video frames in the N second video frames value, to obtain M third video frames; the video composed of M third video frames has the display effect indicated by the second user operation, and the color coding format of the M third video frames is RGB format, which represents the data of the color value data The type is float.

In conjunction with the method provided in the first aspect, in some embodiments, displaying any one of the M third video frames on the first interface specifically includes: the GPU sends the second video memory address to OpenGL, and the second video memory address is to store M The video memory address of the third video frame; OpenGL sends the second video memory address to APP; APP obtains M third video frames according to the second video memory address; APP displays on the first interface that M third video frames are obtained through the second video memory address Any one of the video frames in the frame.

In combination with the method provided in the first aspect, in some embodiments, the electronic device further includes an encoder and a second memory, and saves M third video frames as the second video, specifically including: the APP sends a request to OpenGL to call the C2D engine; In response to the request, OpenGL calls the C2D engine to obtain M third video frames from the GPU; the color encoding format of the M third video frames obtained by OpenGL is YUV format, and the data type representing the color value is an integer; OpenGL will The M third video frames are sent to the second memory; the encoder reads the M third video frames from the second memory, and encapsulates the M third video frames into the second video.

In combination with the method provided in the first aspect, in some embodiments, before the APP sends the first video to the first memory, the method further includes: the APP calls mediacodec to create an encoder; the encoder applies to the memory for the first memory; OpenGL receives the video sent by the APP. The memory identification ID and/or address of the first memory, and determine the first memory for receiving the video frame output by OpenGL according to the ID and/or address; the APP calls mediacodec to create a decoder; the decoder applies to the memory for a second memory.

In a second aspect, the present application provides an electronic device, which includes one or more processors and one or more memories; wherein, one or more memories are coupled with one or more processors, and one or more The memory is used to store computer program codes. The computer program codes include computer instructions. When one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation manner of the first aspect.

In a third aspect, the present application provides a computer-readable storage medium, including instructions. When the above-mentioned instructions are run on an electronic device, the above-mentioned electronic device executes the method described in the first aspect and any possible implementation manner of the first aspect. method.

In a fourth aspect, the present application provides a computer program product containing instructions. When the above-mentioned computer program product is run on an electronic device, the above-mentioned electronic device is executed as described in the first aspect and any possible implementation manner of the first aspect. method.

It can be understood that the electronic device provided in the second aspect above, the computer storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in the first aspect of the present application. Therefore, the beneficial effects that it can achieve can refer to the beneficial effects in the corresponding method, and will not be repeated here.

Description of drawings

Figure 1A-Figure 1K are a set of schematic diagrams of user interfaces provided by the embodiment of the present application;

FIG. 2 is a software architecture diagram of an electronic device provided by an embodiment of the present application;

Fig. 3 is a flow chart of the video editing method provided by the embodiment of the present application;

FIG. 4 is a flowchart of an electronic device initializing a video editing environment provided by an embodiment of the present application;

Fig. 5 is a flow chart of the electronic device provided in the embodiment of the present application to transform the color gamut of the video to be edited and process the video to be edited according to the editing operation selected by the user;

FIG. 6 is a flow chart of saving an edited video by an electronic device provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of video color gamut conversion implemented by an electronic device using LUT resources according to an embodiment of the present application;

FIG. 8 is a hardware structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

The color gamut represents the range of colors that can be displayed when video is encoded. Standard Dynamic Range (SDR) video uses BT709 color gamut; High Dynamic Range (High Dynamic Range, HDR) video uses BT2020 color gamut. Therefore, compared with SDR video, HDR video can use more colors, a wider range of color representation, and a higher display brightness range. Further, HDR video can support richer image color performance and more vivid images. Image detail performance. This also enables HDR video to provide users with viewing effects, thereby improving the user experience.

Not limited to BT2020 color gamut, BT709 color gamut, HDR video and SDR video may also use other types of color gamut. But in general, the color gamut used by HDR video is wider than that used by SDR video, and the color and details are richer.

Usually, electronic devices such as mobile phones and tablet computers (hereinafter referred to as the electronic device 100 for short) support the shooting of SDR videos. With the development of shooting technology and image technology, the electronic device 100 not only supports shooting SDR video, but also supports shooting HDR video. In this way, the user's demand for editing HDR video also emerges.

The editor used by the electronic device 100 to edit the HDR video is also an SDR video editor. At this time, when using the SDR video editor to edit the HDR video, since the color gamut (BT2020) of the HDR video includes a wider color range than the SDR video (BT709), in the process of using the SDR editor to edit the HDR video, the SDR The editor cannot properly display the HDR video to be edited, for example, the display is not clear, contains noise points, and so on.

In order to solve the above problems, an embodiment of the present application provides a video editing method. The method can be applied to an electronic device capable of processing the image, such as a mobile phone, a tablet computer, and the like (ie, the electronic device 100 ).

Implementing the video editing method provided in the embodiment of the present application, the electronic device 100 can convert the HDR video with a color gamut of BT2020 into an SDR video with a color gamut of BT709, so that the electronic device 100 can normally process the video when using an editor to edit the video And display the video frame of the video to be edited.

Wherein, the conversion of the HDR video to be edited by the electronic device 100 into a corresponding SDR video may be implemented by using a LUT filter for rendering. The above-mentioned LUT filter is a special filter based on a color lookup table (look up table, LUT) algorithm. This specific LUT filter can be used to convert video frames with a color gamut of BT2020 to video frames with a color gamut of BT709, thereby realizing the function of converting HDR video to SDR video.

Optionally, when the HDR video and/or the SDR video adopts other color gamut modes, the aforementioned LUT filter can be adjusted accordingly so that it can perform adaptive color gamut conversion. For example, when the color gamut of HDR video is BT2020 and the color gamut of SDR video is sRGB, the above LUT filter can be used to convert the video frame of BT2020 color gamut to the video frame of sRGB color gamut.

In this way, the user can edit the above-mentioned SDR video converted from the HDR video, add filters, etc. to the video. Video editing operations include, but are not limited to, adding filters. For example, the above editing operations also include cropping, image inversion, zooming, adding text, adding filters, adding headers (or endings or other pages), adding video watermarks or stickers, and so on.

Not limited to mobile phones and tablet computers, the electronic device 100 can also be a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital Assistant (personal digital assistant, PDA), augmented reality (augmented reality, AR) equipment, virtual reality (virtual reality, VR) equipment, artificial intelligence (artificial intelligence, AI) equipment, wearable equipment, vehicle equipment, smart home equipment and/or smart city devices, the graphics processors of the aforementioned electronic devices are not capable of editing and saving the edited video as HDR video. The embodiment of the present application does not specifically limit the specific type of the electronic device.

1A-1K schematically show a group of user interfaces on the electronic device 100. The application scenarios for implementing the video editing method provided by the embodiment of the present application will be described in detail below with reference to FIGS. 1A-1K.

First, FIG. 1A exemplarily shows a user interface displaying installed application programs on the electronic device 100, that is, a home page. As shown in FIG. 1A , one or more application program icons are displayed on the main page, such as a "clock" application program icon, a "calendar" application program icon, a "weather" application program icon, and the like.

The above-mentioned one or more application program icons include an icon of a “Gallery” application program (hereinafter referred to as “Gallery”), that is, an icon 111 . The electronic device 100 may detect a user operation acting on the icon 111 . The above operation is, for example, a click operation or the like. In response to the above operations, the electronic device 100 may display the user interface shown in FIG. 1B .

FIG. 1B exemplarily shows the main interface of the "Gallery" when the "Gallery" is running on the electronic device 100 . This interface can display one or more pictures or videos. Wherein, the above-mentioned one or more videos include HDR video, LOG video, and other types of video, such as SDR video. The above-mentioned LOG video refers to a low-saturation, low-brightness video shot in the LOG gray mode, and may also be called a LOG gray film.

As shown in FIG. 1B , the video indicated by the icon 121 may be a LOG video; the video indicated by the icon 122 may be an HDR video; the video indicated by the icon 123 may be an SDR video. When the electronic device 100 displays HDR video or LOG video, the icon indicating the video can display the type of the video. In this way, the user can know the type of the video through the information displayed in the icon. For example, LOG is displayed in the lower left corner of the icon 121 ; HDR is displayed in the lower left corner of the icon 122 . Videos not labeled HDR or LOG in Figure 1B are SDR videos.

The electronic device 100 may detect a user operation acting on the icon 122, and in response to the operation, the electronic device 100 may display the user interface shown in FIG. 1C. FIG. 1C is a user interface of the electronic device 100 specifically displaying a certain picture or video.

As shown in FIG. 1C , the user interface may include a window 131 . The window 131 can be used to display the videos that the user chooses to browse. For example, in FIG. 1B , the video that the user selects to browse is the HDR video indicated by the icon 122 ("Video A"). Then, "Video A" can be displayed in the window 131 .

The user interface also includes icons 132 and controls 133 . Icons 132 may be used to represent the type of video displayed in window 131 . For example, "HDR" displayed in the current icon 132 may indicate that "Video A" is an HDR type video.

The control 133 can be used to receive a user's operation of editing a video (or picture), and display a user interface for editing a video (or picture). Generally, if the chip platform used by the electronic device 100 does not support the processing of HDR video, the electronic device 100 will not provide the user with the function of editing the HDR video. Therefore, FIG. 1C generally includes a control 133, that is, the electronic device 100 does not provide the user with a control for editing video, because the electronic device 100 cannot output and save the edited HDR video.

In the embodiment of the present application, the electronic device 100 can convert the HDR video into an SDR video, and then provide the user with the function of editing the SDR video, so as to meet the user's editing needs and improve the user experience. Therefore, in the user interface shown in FIG. 1C , the electronic device 100 can display the control 133 and can respond to a user operation acting on the control 133 .

The user interface may also include controls 134, share controls (135), favorite controls (136), delete controls (137), and the like.

The control 134 can be used to display detailed information of the video, such as shooting time, shooting location, color coding format, bit rate, frame rate, pixel size and so on.

The sharing control (135) can be used to send the video A to other applications for use. For example, after detecting a user operation acting on the sharing control, in response to the operation, the electronic device 100 may display icons of one or more applications, and the icons of the one or more applications include an icon of social software A. After detecting the application icon acting on the social software A, in response to the operation, the electronic device 100 can send the video A to the social software A, and further, the user can share the video to friends through the social software.

A favorite control (136) can be used to tag videos. In the user interface shown in FIG. 1C , when a user operation acting on the favorite control is detected, in response to the operation, the electronic device 100 may mark video A as the user's favorite video. The electronic device 100 can generate an album for displaying videos marked as favorite by the user. In this way, in the case that video A is marked as the user's favorite video, the user can quickly view video A through the above-mentioned photo album showing the user's favorite videos.

A delete control (137) can be used to delete video A.

When a user operation on the control 133 is detected, the electronic device 100 may display the user interface shown in FIG. 1D . FIG. 1D exemplarily shows a user interface for a user to edit a video (or picture). As shown in FIG. 1D , the user interface may include a window 141 , a window 142 , an operation bar 143 , and an operation bar 144 .

The window 141 may be used to display a preview image of the edited HDR video. Generally, window 141 will display the cover video frame of the video. When a user operation acting on the play button 145 is detected, the window 141 may sequentially display the video frame stream of the video, that is, play the video.

Window 142 may be used to display a stream of video frames of the video being edited. The user can drag the window 142 to adjust the video frame displayed in the window 141 . Specifically, a scale 147 is also shown in FIG. 1D . The electronic device 100 can detect the user operation of sliding left or right on the window 142, and in response to the above user operation, the position of the video frame stream where the ruler 147 is located is different. At this time, the electronic device 100 can display the current Frame of video where ruler 147 is located.

A plurality of icons for video editing operations may be displayed in the operation bar 143 and the operation bar 144 . Generally, an icon displayed in the operation bar 143 indicates a category of editing operations. The operation bar 144 can display video editing operations belonging to the selected operation category in the current operation bar 143 . For example, "Clip" is included in the operation column 143 . The "clip" displayed in bold may indicate that the type of video editing operation currently selected by the user is "clip". At this time, some operations belonging to the "Clip" category are displayed in the operation bar 144, such as "Split", "Crop", "Volume", "Frame" and so on.

Exemplarily, the electronic device 100 may detect a user operation on the "split" control, and in response to the operation, the electronic device 100 may display one or more operation controls for splitting the video. The electronic device 100 can record the segmentation operation of the user, such as the start time and end time of the first video segment, the start time and end time of the second video segment, and so on.

For another example, the electronic device 100 may detect a user operation acting on the "frame" control, and in response to the operation, the electronic device 100 may record the size of the video frame set by the user, and then crop the original video frame.

The operation bar 144 corresponding to the "clip" operation also includes other editing controls belonging to the "clip" category. In response to user operations acting on the above-mentioned controls, the electronic device 100 may record and execute video editing operations corresponding to the above-mentioned controls, which will not be exemplified here.

The user interface also includes a save control 146 . When a user operation acting on the save control 146 is detected, in response to the operation, the electronic device 100 may save the video in the current state. The video in the current state may be a video with editing operations added, or a video without editing operations.

The electronic device 100 may detect a user operation on the "filter" control in the operation bar 143, and in response to the operation, the electronic device 100 may display the user interface shown in FIG. 1E. FIG. 1E exemplarily shows a user interface where the electronic device 100 displays a filter provided for a user to adjust the color of a video image.

"Filter" includes several filter options. Each filter option corresponds to an image processing method to adjust the display effect of the video screen. A user may select one of a plurality of filters provided by the electronic device 100 . In response to the above-mentioned user operation of selecting a filter, the electronic device 100 may perform image processing indicated by the above-mentioned filter selected by the user on the edited video, so that the screen of the processed video has a display consistent with the display effect of the above-mentioned filter Effect.

As shown in FIG. 1E , multiple filter controls may be displayed in the filter selection interface, such as filter control 151 , filter control 152 , filter control 153 , filter control 154 , filter control 155 and so on. Each of the filter controls above indicates an image processing method that uses a filter to render an image.

First, when displaying the user interface shown in FIG. 1E , the electronic device 100 may set the currently used filter as the filter 151 (the filter indicated by the filter control 151 ) by default. Then, when a user's operation on a certain filter control is detected, in response to the operation, the electronic device 100 may display a user interface for editing a video using a filter indicated by the above-mentioned filter control.

For example, the electronic device 100 may detect a user operation on the filter control 155, and in response to the operation, the electronic device 100 may display the user interface described in FIG. 1F. As shown in FIG. 1F , the electronic device 100 can highlight the display effect of the selected filter control, for example, enlarge the filter control, thicken the border of the control, or set the highlight of the control, etc. Examples are not limited to this.

At the same time, the electronic device 100 will also display in the window 141 a preview image after using the filter control 155 to render the video to be edited. For example, the picture color of "Video A" displayed in window 141 is different from the picture color in window 141 in Fig. 1E at this time, and the "Video A" in window 141 in Fig. Mirror consistent display effect.

Generally, in order to save computing resources, the electronic device 100 often only renders the video frame displayed in the current window, or, in some embodiments, the electronic device 100 can also use other simple image processing means to process the above-mentioned cover video frame, so that after processing The image of is previewed with the effect of the above filter.

The user interface also displays a confirmation control 147 ("√"), a cancel control 148 ("ⅹ"). When it is determined that the currently selected filter meets (filter 155 ) the user's needs, the user can click the confirmation control 147 .

Of course, when it is determined that the currently selected filter does not meet the user's needs, the user may click other filter controls to select other filters. In response to a user operation acting on any filter control, the electronic device 100 may display in the window 141 a video rendered using the filter indicated by any of the above filter controls. When none of the filters provided by the electronic device 100 meets the user's needs, or when the user suspends editing the filters, the user can click the cancel control 148 . In response to the above user operations, the electronic device 100 may display the user interface shown in FIG. 1D .

The electronic device 100 may detect a user operation acting on the "music" control in the operation bar 143, and in response to the above operation, the electronic device 100 may display the user interface shown in FIG. 1G.

At this time, the "Music" control in the operation bar 143 can be thickened to indicate that the type of editing operation currently selected by the user is "Music". At the same time, the editing controls displayed in the operation bar 144 will be replaced with corresponding operation controls under the "music" operation, such as "add music" and "extract music".

When a user operation acting on the 'add music' control is detected, in response to the operation, the electronic device 100 may display a user interface for adding music. As shown in FIG. 1H , the user interface may display multiple music options, such as "Music 1", "Music 2", "Music 3", "Music 4" and so on. Each music option is followed by playback controls and usage controls. The playback control can be used to play the music corresponding to the music option. Use controls can be used to apply the above music to the edited video.

The "Extract Music" control can be used to extract audio from the video to be edited. For example, in the scenario shown in FIG. 1G , after detecting a user operation on the control, in response to the operation, the electronic device 100 may extract the audio from "Video A". Not limited to controls such as "add music" and "extract music", the operation bar 144 may also include more music operation controls. The embodiment of the present application does not limit this.

The electronic device 100 may detect a user operation acting on the "text" control in the operation bar 143, and in response to the above operation, the electronic device 100 may display the user interface shown in FIG. 1I.

At this time, the "text" control in the operation bar 143 can be thickened to indicate that the type of editing operation currently selected by the user is "text". At the same time, the editing controls displayed in the operation bar 144 will be replaced with the corresponding operating controls under the "text" operation, including "title" and "title". Wherein, "title" and "title" include multiple text templates.

As shown in FIG. 1I , first, the electronic device 100 can display a text template of "Title", such as "None" 151, "Title 1", "Title 2", "Title 3", "Title 4", "Title 5" etc. The electronic device 100 can detect a user operation acting on any of the above templates, for example, detect a user operation acting on "Title 5", and in response to the above operation, the electronic device 100 can display the title of "Title 5" in the window 141 Effect.

Then, the electronic device 100 may detect a user operation on the confirmation control 161 . At this time, the electronic device 100 may confirm that the user selects an editing operation using the title shown in "Title 5". Meanwhile, the electronic device 100 may display the user interface described in FIG. 1K.

The electronic device 100 detects that the user edits the "credits credits" for the process of adding a credits credits to the edited video, refer to the above process of adding a "credits credits", and will not be repeated here. In addition, the electronic device 100 can also provide more editing capabilities, which will not be listed one by one here.

1D-1J exemplarily show a process in which the electronic device 100 receives an operation of editing a video by a user. After detecting the operation of saving the video, the electronic device 100 can convert the HDR video displayed in the window 133 in FIG. Edit operation shown in 1J, and then save the edited SDR video.

Referring to FIGS. 1J and 1K , in response to user operations acting on the save control 146 , the electronic device 100 may perform calculations for editing and saving the SDR video. After the saving is completed, the electronic device 100 may display the user interface shown in FIG. 1K. Compared with the user interface shown in FIG. 1C , at this moment, the video displayed in window 131 is the edited SDR video. For example, the cover of the video shown in FIG. 1K is the title added in the editing operation.

Optionally, the electronic device 100 may save the edited video as a new video. In this way, the electronic device 100 can not only provide the user with an HDR video before editing, but also provide the user with an edited personalized SDR video.

Implementing the methods described in FIGS. 1A-1K , when the electronic device 100 does not support the output of HDR video, the electronic device 100 can convert the HDR video to be edited into an SDR video. Then, on the basis that the electronic device 100 supports outputting the SDR video, the electronic device 100 may provide the user with a function of editing the SDR video. In this way, from the user's point of view: the user can edit the above-mentioned HDR video, and save the edited video.

In the scenario where the chip platform used by the electronic device 100 does not support the output of HDR video, when the user shoots an HDR video and wants to edit the HDR video, the electronic device 100 can convert the above-mentioned HDR video into a corresponding SDR video, and then Provide users with video editing services to firstly meet users' needs for editing videos.

The specific process for the electronic device 100 to realize the video editing capability shown in FIG. 1A-FIG. 1K is specifically introduced below.

First, FIG. 2 exemplarily shows the software architecture of the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the software structure of the electronic device 100 is exemplarily described by taking an Android system with a layered architecture as an example.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the Android system is divided into four layers, which are respectively the application program layer, the application program framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer from top to bottom.

The application layer can consist of a series of application packages. As shown in Figure 2, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message. In the embodiment of the present application, the application program layer also includes a video editing application. The video editing application has video data processing capabilities, and can provide users with video editing functions, including cropping, rendering, and other video data processing. The user interfaces shown in FIGS. 1D-1J can be regarded as the user interfaces provided by the above-mentioned video editing application.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 2, the application framework layer can include window managers, content providers, view systems, phone managers, resource managers, notification managers, and so on.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. Content providers are used to store and retrieve data and make it accessible to applications. Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide communication functions of the electronic device 100 . For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, prompting text information in the status bar, issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.

In the embodiment of the present application, the application framework layer also includes a media framework. The media framework provides multiple tools for editing video and audio. Among them, the above tools include MediaCodec. MediaCodec is a class provided by Android for encoding and decoding audio and video. It includes encoder, decoder.

Among them, the encoder can convert one form of video or audio input to the encoder into another form through compression technology, and the decoder performs the reverse process of encoding, and can convert one form of video input to the decoder Or the audio is converted to another form by decompression techniques.

For example, the video input to the encoder may be HDR video. The above HDR video is composed of N video frames whose color gamut is BT2020. The aforementioned N is an integer greater than 1. After receiving the above-mentioned HDR video, the decoder may split the above-mentioned N video frames composed of BT2020 color gamut into N independent video frames for the subsequent electronic device 100 to perform image processing on each video frame.

Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

A system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc. 2D graphics engine is a drawing engine for 2D drawing.

Open Graphics Library (OpenGL) provides multiple image rendering functions, which can be used to draw from simple graphics to complex three-dimensional scenes. In this embodiment of the present application, the OpenGL provided by the system library can be used to provide graphics and image editing operations for video editing applications, such as video cropping operations and filter adding operations described in the foregoing embodiments.

The kernel layer is the layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

FIG. 3 exemplarily shows a flow chart of editing HDR video by the electronic device 100 . Combining the user interface shown in FIG. 1A-FIG. 1K and the software architecture of the electronic device 100 shown in FIG. 2 , the following embodiments of the present application will specifically introduce the process for the electronic device 100 to provide users with editing HDR video.

S101: The electronic device 100 determines that the user chooses to edit an HDR video.

When displaying image resources such as pictures and videos stored in the gallery for the user to browse, the electronic device 100 may display editing controls. The edit control can provide users with the service of editing the currently displayed image resource. The video editing method provided in the embodiment of the present application is mainly applied to video image resources. Subsequent embodiments will use video as an example to introduce the video editing method provided by the embodiment of the present application.

Referring to the user interface shown in FIG. 1C , the window 131 can provide the user with browsing image resources stored in the electronic device 100 ; the control 133 , that is, the editing control, can provide the user with the service of editing video. At this time, the edited video is the video displayed in the window 131 .

After detecting a user operation acting on the edit control, in response to the above operation, the electronic device 100 may first determine the type of the edited video, so as to determine whether the edited video is processed and output by the chip platform. In the embodiment of the present application, when it is detected that the video to be edited is an HDR video, the electronic device 100 may determine that when editing the HDR video, the electronic device 100 cannot correctly display the HDR video frame. This is because the color gamut adopted by the HDR video is BT2020, and the chip platform used by the electronic device 100 does not support displaying the HDR video frame whose color gamut is BT2020 in the editing scene. As a result, in an editing scene, the electronic device 100 displays abnormal HDR video frames, thereby affecting user experience. The abnormal display of the HDR video frame above includes unclear display, wrong pixel color, and so on.

When detecting that the video to be edited is an SDR video, the electronic device 100 may determine that: when editing the SDR video, the electronic device 100 can normally display the SDR video to be edited.

After determining the type of the video to be edited, the electronic device 100 may implement different editing strategies according to the type of the video, so as to meet the user's editing needs.

Specifically, when the video to be edited is an SDR video, the electronic device 100 may determine: call the processing and output capabilities provided by the chip, and provide the user with a service of editing the video. When the edited video is an HDR video, that is, the electronic device 100 cannot display the HDR video frame normally, at this time, the electronic device 100 can determine to use the image editing method provided in the embodiment of this application: use LUT resources to convert the HDR video frame into a corresponding The SDR video frame, and then the electronic device 100 can display the above SDR video frame, so as to avoid abnormal display. The above LUT resource is a data resource established based on a color lookup table (look up table, LUT) algorithm for converting a video frame with a color gamut of BT2020 into a video frame with a color gamut of BT709.

Further, based on the existing ability to edit SDR video, the user is provided with the service of editing the above-mentioned HDR video. Here, what the user actually edits is the SDR video corresponding to the above HDR video. In this way, the electronic device 100 can provide users with the ability to edit HDR videos.

In this way, when a user operation for editing an HDR video is detected, the electronic device 100 may execute the image editing method provided in the embodiment of the present application to provide the user with a video editing service, so as to meet the user's demand for editing personalized videos.

Referring to the user interface shown in FIG. 1C , after detecting a user operation acting on the editing control 133 , in response to the above operation, the electronic device 100 may first determine the type of the displayed video (“Video A”) in the window 131 . At this time, "Video A" is an HDR video, therefore, the electronic device 100 may determine that the video to be edited ("Video A") is an HDR video. Subsequently, the electronic device 100 may determine to use the image editing method provided in the embodiment of the present application to provide the user with the ability to edit the HDR video.

S102: The electronic device 100 initializes a video editing environment.

After detecting a user operation on an editing control, the electronic device 100 may initialize an editing environment. Initializing the editing environment refers to creating or applying for tools and storage space required for editing a video, so that the electronic device 100 can perform data processing for editing the video.

Initializing the video editing environment includes: creating encoders, decoders, OpenGL, applying for memory for user cache video frames and display memory provided by GPU. A decoder can be used to split a video to be edited into a sequence of video frames; an encoder can be used to combine edited video frames into a video. OpenGL can be used to adjust the video frame, and/or modify the pixels in the video frame, so as to change the image content included in the video, that is, to render the video frame. The aforementioned adjustment of the video frame includes adjusting to increase or decrease the video frame, and to modify the size of the video frame.

Among them, the above memory includes surface and BufferQueue. A Surface can be used to cache rendered video frames output by the GPU. The encoder can encapsulate the sequence of video frames stored in the Surface into a video. BufferQueue can be used to cache the video to be edited input by the video editing application. The decoder can split the video to be edited stored in the BufferQueue into a sequence of video frames to be edited.

Specifically, FIG. 4 exemplarily shows a flow chart of electronic device 100 initializing a video editing environment. As shown in Figure 4, APP can be used to represent a video editing application.

First, (1) the electronic device 100 may detect a user operation of clicking an edit control. Referring to the user interface shown in FIG. 1C , a user operation acting on the edit control 133 may be referred to as a user operation of clicking the edit control.

(2) In response to the above user operations, the APP can determine the type of the video to be edited and the type of the edited video. In this embodiment of the application, the video to be edited is an HDR video. Among them, the color encoding format adopted by the HDR video is YUV format, the data type of the color value of the color channel is integer (INT), and the color gamut is BT2020. At this time, the electronic device 100 can determine to convert the HDR video into an SDR video, and the color gamut is BT709, so as to ensure that the electronic device 100 can support the display of the video frame of the HDR video to be edited in the editing environment (that is, display the SDR video corresponding to the HDR video) , while providing users with the ability to edit HDR videos.

If the video to be edited is an SDR video, the APP can determine that the video to be edited is an SDR video, and the edited video is also an SDR video.

Then, (3) APP can send a request to MediaCodec to create an encoder. The request can carry output video type information. The above output video type information may be used to indicate the type of the edited video output by the encoder. Specifically, the above output video type information may include: the color gamut and color coding format of the edited video. In the embodiment of this application, the output video type information is specifically BT709 and YUV. The above BT709 may be used to indicate that the color gamut of the edited video is the BT709 color gamut; the above YUV may indicate that the color coding format of the edited video is YUV. Not limited to the color gamut and color coding format, the above output video type information may also include other more parameters describing the output video type. When the above-mentioned output video type information is specifically BT709 or YUV, the video encoded by the encoder is the SDR video.

In response to the above request, (4) MediaCodec may create the encoder indicated by the above output video type information. For example, when recognizing the output video type information (BT709, YUV) carried in the above request, MediaCodec can create an encoder for encoding SDR video (BT709, YUV). The encoder can encapsulate the input SDR video frame with BT709 color gamut and YUV color coding format into an SDR video.

After the encoder is created, (5) MediaCodec may return confirmation information indicating the creation to the APP. The above confirmation information indicating the completion of creation is, for example, the confirmation character ACK and the like.

After receiving the above-mentioned confirmation information, (6) the APP may send a request to the above-mentioned encoder to create a surface. Surface is a memory storage space with a specific data structure. Generally, a surface is dedicated to buffering video frames to be encoded. (7) In response to the above request, the encoder can apply for a surface from the memory. (8) In response to the above application, a piece of storage space can be divided into the memory as the surface requested by the encoder for use by the encoder.

The memory can provide multiple surfaces. Each surface carries an identity (ID) indicating the surface. For any surface, the ID of the surface is in one-to-one correspondence with the address of the surface. For example, suppose the ID of surface-01 is 01; the address is 0011-0100. When it is recognized that the ID of a certain surface is 01, the electronic device 100 can determine that the above-mentioned surface is surface-01, and at the same time can also determine that the address of the above-mentioned surface is 0011-0100; otherwise, when it is recognized that the address used by a certain surface is From 0011 to 0100, the electronic device 100 may determine that the surface is surface-01.

After finishing allocating the surface to the encoder, (9) the memory may return the ID and/or address of the above-mentioned surface to the encoder. After receiving the above return information, the encoder can confirm that the application for the surface is successful, and determine the ID and/or address of the surface that can be used. Further, (10) the encoder can return the ID and/or address of the above-mentioned surface to the APP. In this way, the APP can determine that the encoder has completed the process of applying for a surface from the memory, and can determine the ID and/or address of the usable surface requested by the encoder.

Then, (11) APP can send an initialization request to OpenGL. The above request may carry surface information. The above surface information may be used to indicate the ID and/or address of the surface in the OpenGL decoder for receiving the edited video.

(12) According to the surface information carried in the above request, OpenGL can determine the surface used by the encoder, that is, when the edited video frame is output after performing the calculation and processing of the video frame, OpenGL can determine Which cache (surface) is written to the above-mentioned edited video frame.

In addition, after receiving the above initialization request, (13) OpenGL will also apply for a piece of video memory from the GPU, which is recorded as video memory A. Video memory A can be used to cache video frames to be edited. The above-mentioned video memory A can be a texture (texture) or a frame buffer object (Frame Buffer Object) in OpenGL.

In response to the above application, the GPU will allocate a storage space for OpenGL as the video memory (video memory A) requested by OpenGL. Then, (14) GPU can return the address of video memory A to OpenGL. After receiving the address of the video memory A, OpenGL can locate the video memory A through the above address, and then OpenGL can use the video memory A. Then (15) OpenGL can return confirmation information to APP. The confirmation information may indicate to the APP: OpenGL has been initialized.

After confirming that OpenGL has been initialized, (16) APP can send a request for creating a decoder to MediaCodec. (17) In response to the above request, MediaCodec may create a decoder. When creating a decoder, MediaCodec does not need to specify the type of video that the decoder supports decoding. The decoder can determine the type of the video to be decoded after receiving the video to be decoded from the APP. In the embodiment of the present application, the video to be edited is an HDR video, therefore, the above decoder can be used to decode the HDR video.

After the MediaCodec creates the decoder, (18) the decoder can send a request for a storage space (BufferQueue) to the memory. BufferQueue can be used to receive the video to be decoded input by APP. (19) In response to the above request, the memory can allocate a BufferQueue for the decoder. Subsequently, (20) the memory will return the address of the above-mentioned BufferQueue to the decoder.

After receiving the address of the BufferQueue returned by the memory, the decoder can locate the usable BufferQueue in the memory according to the above address. Subsequently, (21) the decoder may return confirmation information indicating successful creation of the decoder to the APP.

In other embodiments, the process of APP calling MediaCodec to create a decoder (step (16)-step (21)) may also occur before creating an encoder. This embodiment of the present application does not limit this.

The process shown in steps (1) to (21) in FIG. 4 shows the process of the electronic device 100 initializing the video editing environment. After the initialization of the editing environment is completed, the electronic device 100 can start to perform editing operations selected by the user on the video to be edited.

S103: The electronic device 100 converts the HDR video frame into an SDR video frame.

After completing the process of initializing the video editing environment, the electronic device 100 can use the above-mentioned video editing environment to split the HDR video (color space BT2020) to be edited into SDR video frames (color space BT709).

Specifically, the electronic device 100 can first use a decoder to split the above-mentioned HDR video (color gamut BT2020) to be edited into HDR video frames (color gamut BT2020), and then, the electronic device 100 can use the color gamut conversion filter resource (LUT resource) to convert the above HDR video frame (gamut BT2020) to SDR video frame. Steps (1) to (15) in FIG. 5 show a specific process for the electronic device 100 to convert HDR video frames into SDR video frames.

First, (1) the APP can send an instruction to OpenGL to load the LUT resource into the GPU memory. The APP can determine the storage address of the LUT resource. The above indication may carry the above storage address, such as 0100-1000. In this way, the APP can instruct OpenGL to obtain the storage space of the LUT resource.

(2) After receiving the above instruction, OpenGL may first locate the storage space for storing the LUT resource according to the storage address carried in the instruction. Then, OpenGL can read the LUT resources from the above storage space, and write the above LUT resources into the GPU.

Wherein, the address of the video memory storing the LUT resource in the GPU may be specified by the APP or by OpenGL. When the address of the video memory storing the LUT resource is specified by the APP, the above instruction should also carry the address of the video memory storing the LUT resource.

(3) After the writing is successful, OpenGL can return the writing success indication information to the APP. After receiving the above-mentioned write success indication information, the APP can confirm that OpenGL has completed the operation of loading the LUT resources required to convert the HDR video to the SDR video into the GPU memory.

Then, (4) APP can input the HDR video to be edited into the decoder. Specifically, according to the initialization process of the editing environment, the APP can determine the address of the BufferQueue requested by the decoder for caching the video to be decoded. After determining the above address, the APP can write the HDR video to be edited to the above BufferQueue. At this time, the color encoding format adopted by the HDR video to be edited is YUV format, the data type of the color value of the color channel is integer (INT), and the color gamut is BT2020.

(5) When detecting that the video is written into the BufferQueue, the decoder can decode the video stored in the BufferQueue, so as to obtain the video frame sequence of the video. Therefore, after writing the HDR video to be edited into the BufferQueue, the decoder can output the video frames of the HDR video to be edited, that is, N HDR video frames (HDR video frames to be edited). At this time, the color coding format, data type and color gamut of the HDR video frame are still YUV, INT, BT2020.

Understandably, a video may also include audio. Therefore, a decoder also includes an audio decoder. In the video editing method provided in the embodiment of the present application, the processing related to audio is a prior art, and will not be repeated here.

In the case of including audio data, after decoding by the decoder, the electronic device 100 can respectively obtain N HDR video frames and audio data of the HDR video to be edited. It can be understood that when the HDR video to be edited does not include audio data, the electronic device 100 does not need to perform audio decoding on the HDR video to be edited, and therefore, the decoded data does not include audio data.

(6) After the decoding is completed, the decoder can sequentially send the decoded HDR video frames (YUV, INT, BT2020) to be edited to OpenGL. Correspondingly, OpenGL can successively receive N HDR video frames (YUV, INT, BT2020) to be edited.

(7) After receiving the HDR video frame to be edited (YUV, INT, BT2020) sent by the decoder, first, OpenGL can change the color coding format of the HDR video frame to be edited and the data type of the color value in the color channel. In the embodiment of this application, OpenGL will set the color coding format of the HDR video frame to be edited to RGB, and the data type of the color value in the color channel to FLOAT, that is, the HDR video frame to be edited in the original (YUV, INT) format Change the HDR video frame to be edited in (RGB, FLOAT) format. This is because the color coding format supported by OpenGL when drawing and/or rendering the video frame is RGB, and the data type of the color value in the color channel is floating point.

Therefore, after the above N HDR video frames are input into OpenGL, their color coding format is changed to RGB, and the data type of the color value in the color channel is changed to floating point. At this time, the color gamut of the HDR video frame is still BT2020, that is, the above process does not involve converting the HDR video frame into an SDR video frame.

(8) After converting the HDR video frame to be edited in (YUV, INT, BT2020) format to the HDR video frame to be edited in (RGB, FLOAT, BT2020) format, OpenGL can write the changed HDR video frame to be edited GPU. Specifically, in steps (13) and (14) shown in FIG. 4, OpenGL applies for video memory A to the GPU. At this time, OpenGL can write the HDR video frame (RGB, FLOAT, BT2020) to be edited after the above changes into the GPU.

(9) After the writing is successful, OpenGL can call the LUT resource written in the GPU in advance. The above LUT resources are used to convert HDR video frames to SDR video frames.

(10) After acquiring the above-mentioned LUT resource, OpenGL can determine the calculation logic for converting the HDR video frame to be edited into an SDR video frame (RGB, FLOAT, BT709) using the above-mentioned LUT resource. (11) Then, OpenGL can send the above calculation logic to the GPU to instruct the GPU to use the above LUT resource to convert the HDR video frame to be edited into an SDR video frame (RGB, FLOAT, BT709).

(12) In response to the calculation logic issued by OpenGL, the GPU can sequentially modify the color value of each pixel in the HDR video frame to be edited, so as to realize the conversion of pixels with a color gamut of BT2020 into pixels with a color gamut of BT709, Then realize the conversion of HDR video frame into SDR video. Subsequent embodiments will introduce in detail how the GPU converts pixels with a color gamut of BT2020 into pixels with a color gamut of BT709 according to the calculation logic issued by OpenGL, so as to realize the specific process of converting HDR video frames into SDR video frames, which will not be expanded here .

(13) After using the LUT resource to convert the HDR video frame to be edited into an SDR video frame (RGB, FLOAT, BT709), the GPU can send the address of the video memory B storing the above SDR video frame to OpenGL. (14) Further, OpenGL may return the address of the above-mentioned video memory B to the APP. The video memory B above can be the same as or different from the video memory A.

(15) After receiving the address of the above video memory B, the APP can confirm that OpenGL has been completed: convert the HDR video frame to be edited into an SDR video frame. At this time, the electronic device 100 can display the above-mentioned SDR video frame (RGB, FLOAT, BT709). In this way, when the HDR video frame cannot be displayed normally, the electronic device 100 can display the SDR video frame converted according to the HDR video frame, thereby avoiding the situation that the electronic device 100 cannot normally display the HDR video frame to be edited.

Referring to the user interface shown in Figure 1D, after implementing the conversion method shown in steps (1) to (14) in Figure 5, the electronic device 100 can obtain the SDR video frame (BT709) converted according to the HDR video frame (BT2020) . Then, the electronic device 100 can display the above SDR video frame in the preview window 141 . At this time, when the preview window 141 cannot normally display the original HDR video frame, the SDR video frame displayed in the preview window 141 can provide a better preview effect for the user.

S104: The electronic device 100 modifies the SDR video frame to be edited according to the detected editing operation.

After detecting a user operation for editing a video, the electronic device 100 may display a user interface for editing the video. The user interface shown in FIGS. 1D-1J may be referred to as a user interface for editing a video. While the preview window 141 displays the SDR video frame to be edited (BT709) converted according to the HDR video frame to be edited (BT2020), multiple controls for editing video can be displayed in the user interface, such as the operation bar 143, The operation bar 144 provides multiple editing controls and the like for the user to edit the video. The electronic device 100 may detect a user operation acting on the editing control, and then perform image processing on the SDR video frame to be edited, so that the edited SDR video frame has the display effect indicated by the editing control.

Taking the operation of adding filter 155 shown in FIG. 1F as an example, and combining steps (16)-(20) in FIG. 5 , the process of modifying the SDR video frame to be edited by the electronic device 100 according to the detected editing operation will be described.

After converting the HDR video frame to be edited (BT2020) into the SDR video frame to be edited (BT709), the electronic device 100 can detect the user operation acting on the filter 155, and then execute the filter 155 on the SDR video frame to be edited (BT709). Indicated image processing to generate edited SDR video frames (BT709). The edited SDR video frame has the display effect shown by filter 155 .

Specifically, referring to steps (16)-(20) in FIG. 5, first, (16) the electronic device 100 may detect a user operation acting on a certain editing control. For example, in the user interface shown in FIG. 1F , the electronic device 100 may detect a user operation acting on the edit control filter 155 . The above-mentioned user operations acting on a certain edit processing control may be referred to as edit operations selected by the user.

(17) After detecting the editing operation selected by the user, the APP can send the editing operation to OpenGL. For example, the APP may send the editing operation of the filter 155 to OpenGL.

(18) After receiving the editing operation sent by the APP, OpenGL can determine the calculation logic for performing the editing operation on the SDR video frame to be edited. (19) Then, after the above-mentioned calculation logic is determined, OpenGL can send the above-mentioned calculation logic to the GPU to instruct the GPU to perform the calculation processing of the SDR video frame to be edited, so that the edited SDR video frame has the editing operation selected by the user display effect. At this time, the color gamut of the edited SDR video frame is still BT709, that is, the editing object selected by the user is the SDR video frame, and the edited video frame is still the SDR video frame.

(20) In response to the calculation logic delivered by OpenGL, the GPU can modify the video frame size and/or pixel color value of the SDR video frame to be edited according to the above calculation logic. The SDR video frame that has been processed according to the calculation logic may be called an edited SDR video frame.

When the editing operation selected by the user is an editing operation that modifies the color value of a pixel, such as the operation of adding a filter, the calculation logic received by the GPU will instruct the GPU to modify the color value of the pixel in the video frame according to the color conversion formula of the added filter. color value. When the editing operation selected by the user is to modify the size of the video frame, such as adding a cut operation, the calculation logic received by the GPU will instruct the GPU to modify the number of pixels in the video frame according to the video frame on which the cut operation is applied.

(21) After obtaining the edited SDR video frame, the GPU may send the address of the video memory C storing the edited SDR video frame to OpenGL. The above-mentioned video memory C may be the same as or different from video memory A or video memory B. (22) Subsequently, OpenGL can send the address of the above-mentioned video memory C to the APP.

(23) After receiving the video memory address for storing the edited SDR video frame returned by OpenGL, the APP can obtain the edited SDR video frame according to the above address, and further, the APP can display the above-mentioned edited SDR video frame. In combination with the user interface shown in FIG. 1F , the APP can display the above SDR video frame with the display effect shown by the filter 155 added in the preview window 141 .

It can be understood that the electronic device 100 can detect editing operations selected by multiple users. For example, in addition to the operation of adding a filter shown in FIG. 1F , the electronic device 100 may also detect the user's operation of adding a title/title as shown in FIG. 1I . When an editing operation selected by the user is not detected, the APP can send the above-mentioned editing operation to OpenGL, and then OpenGL can instruct the GPU to perform corresponding calculations, so as to add different video display effects to the SDR to be edited.

If the editing operation selected by the user also includes the processing of the audio data in the HDR video to be edited, then the electronic device 100 will also execute the editing processing of the audio data. Wherein, the processing of audio data includes audio clipping, adding audio, deleting audio, merging audio, etc., which will not be repeated here.

S105: The electronic device 100 generates an edited SDR video according to the edited SDR video frame.

The user interface for editing video also includes a save control, such as save control 146 in FIG. 1D . The electronic device 100 may detect a user operation on the save control, such as the user operation on the save control 146 shown in FIG. 1J . In response to the above user operation, the electronic device 100 can generate an SDR video according to the edited SDR video frame, and save it to a storage device such as a memory card or a hard disk for subsequent viewing by the user.

Specifically, FIG. 6 exemplarily shows a specific process for the electronic device 100 to generate an edited SDR video according to edited SDR video frames.

First, (1) APP can detect the user's save operation. For example, a user operation acting on the save control 146 as shown in FIG. 1J . (2) In response to this operation, the APP may send a request to OpenGL to call the C2D engine to output the edited SDR video. The C2D engine is a function provided by the GPU to output image data stored in video memory. At this point, the C2D engine can be used to output edited SDR video frames stored in the GPU.

(3) In response to the above request, the GPU may call the C2D engine to read the edited SDR video frame from the GPU. At this time, the color coding format of the edited SDR video frame stored in the GPU is RGB, the data type of the color value in the color channel is floating point type, and the color gamut is BT709 (RGB, FLOAT type, BT709).

In response to the above call, the GPU may output the above edited SDR video frame to OpenGL. During the output process, the C2D engine can perform format conversion on the above-mentioned edited SDR video frame, including changing the color coding format of the video frame and the data type of the color value, so that the color coding format and color value of the final output video The data type is the same as the video before editing.

Specifically, the C2D engine may set the color encoding format of the edited SDR video frame stored in the GPU to YUV, and set the data type of the color value to integer. In this way, the color encoding format of the edited SDR video frame obtained from the GPU by OpenGL calling the C2D engine is YUV, and the data type of the color value is an integer (YUV, INT type, BT709).

(4) Then, OpenGL can input the above-mentioned (YUV, INT type, BT709) edited SDR video frame into the surface that the encoder applies to cache the video frame. Specifically, with reference to step (11) in Figure 4, OpenGL can determine the ID and/or memory address of the surface used to cache the edited video frame in the encoder, and OpenGL can locate the above-mentioned cache editing according to the ID and/or memory address. The memory space of the surface of the next video frame. Then, OpenGL can write the above-mentioned edited SDR video frame (YUV, INT type, BT709) obtained from the GPU into the above-mentioned surface.

(5) The encoder can detect whether there is a video frame in the surface in real time, that is, the edited SDR video frame. When a video frame is detected in the surface, the encoder combines and encapsulates the video frames stored in the surface, so that the sequence of video frames split by the decoder and processed by OpenGL and GPU is repackaged into a video. In this way, after the encoding operation by the encoder, the electronic device 100 can obtain an SDR video composed of M edited SDR video frames, which is recorded as an edited SDR video. The above M may be the same as or different from N (the number of video frames obtained after decoding by the decoder) in step (5) in FIG. 5 .

Finally, (6) the encoder can return the edited SDR video obtained after encoding to the storage space specified by the APP. Further, the APP can obtain and display the edited video from the above storage space for the user to browse or use.

Combined with the user interface shown in Figure 1J and Figure 1K, after detecting the save operation acting on the save control 146 in Figure 1J, the electronic device 100 can execute the method shown in the above steps (1) to (6), and obtain the edited SDR video. Then, the APP can display the above-mentioned edited SDR video on the user interface shown in FIG. 1K . In this way, the user can obtain an edited and personalized video, so as to meet the user's demand for editing HDR video.

The edited SDR video is a video with a user-specified personalized display effect. The aforementioned personalized display effect refers to the display effect of the editing operation instruction selected by the aforementioned user. For example, the dark gray display effect indicated by the filter control 155, the opening effect indicated by the opening title "Title 5", and so on.

It can be understood that when S105 and S106 also include processing the decoded audio data, in the process of encoding the edited SDR video frame, the electronic device 100 will also encode the processed audio data at the same time. At this time, the edited output SDR video is also processed audio data. The above-mentioned processed audio data may or may not be consistent with the pre-processed audio data. When the editing operation selected by the user involves the processing of audio, such as the editing operation of adding audio shown in Figure 1G and Figure 1H, the above-mentioned processed audio data is inconsistent with the pre-processing audio data; otherwise, the above-mentioned processed audio The data is the same as the audio data before processing.

From the perspective of the electronic device 100 and its functional modules, the above steps S104 to S105 briefly introduce the method for the electronic device 100 to perform editing operations on the HDR video to be edited and save it as an SDR video. Wherein, OpenGL instructs the GPU to use LUT resources to convert the HDR video with a color gamut of BT2020 into an SDR video frame with a color gamut of BT709, as shown in FIG. 7 .

First, Table 1 exemplarily shows LUT resources. As shown in Table 1, the LUT resource is a data resource for converting a video frame with a color gamut of BT2020 into a video frame with a color gamut of BT709 established based on a color lookup table (LUT) algorithm, which can be expressed as a A color lookup table.

The LUT resource shown in Table 1 records the value of each color channel of 33*33*33 (35937) colors. The LUT resources are not limited to the 33*33*33 LUT resources shown in Table 1, and there are other forms of LUT resources, such as 64*64*64 specifications, which are not limited in this embodiment of the present application.

Table 1

颜色编号color number	R(33)R(33)	G(33)G(33)	B(33)B(33)
11	0.03331040.0333104	0.03064010.0306401	0.03079270.0307927
22	0.04776070.0477607	0.02955670.0295567	0.03064010.0306401
33	0.08081180.0808118	0.0270390.027039	0.03027390.0302739
44	11	22	33
……...	……...	……...	……...
34683468	11	0.982040.98204	00
……...	……...	……...	……...
3593535935	11	11	11
3593635936	11	11	11

35937

1

The color number 1 means that when R is at the first step, the step number is 0; when G is at the first step, the step number is 0; when B is at the first step, the step number is 0. That is, the color value processed by the step size is (0,0,0), the color number of the color value corresponding to the color value (0,0,0) in Table 1 is 1, and its color value is (0.0333104, 0.0306401, 0.0307927).

Color number 2 means that when R is at the first step, the step number is 0; when G is at the first step, the step number is 0; when B is at the first step, the step number is 1. Color number 3 means that when R is at the first step, the step number is 0; when G is at the first step, the step number is 0; when B is at the first step, the step number is 2. By analogy, the color number 33 means that when R is at the first step, the step number is 0; when G is at the first step, the step number is 0; when B is at the first step, the step The long number is 32. The color number 34 means that when R is at the first step, the step number is 0; when G is at the first step, the step number is 1; when B is at the first step, the step number is 0. No more examples here.

The following describes in detail how the GPU uses LUT resources to convert the HDR video with a color gamut of BT2020 into an SDR video frame with a color gamut of BT709.

As shown in FIG. 7 , the pixel point Q is any pixel point in the HDR video frame. Assume that the values of the color channels in the pixel point Q are respectively Q ₁ (0.1, 0.2, 0.1). The above Q ₁ is the color value of the pixel point Q at this time. Among them, 0.1, 0.2, and 0.1 are floating point types.

First, the GPU can multiply the value of each color channel in the color value Q ₁ of the pixel Q by 256 to determine the position of each color channel in Q ₁ from 0 to 256, and obtain the color value Q ₂ (25.6, 51.2, 25.6 ). According to the rounding method, at this time, the color value Q ₂ (26, 51, 26) actually obtained by the GPU.

With a step size of 8, the value of each color channel in Q ₂ (26, 51, 26) above is divided by 8 and rounded up to obtain Q ₃ (3, 6, 3). Here, the step size is 8 because the LUT resource format is 33*33*33. To map 256 color values to 32 color values, it is necessary to regard 8 color values as one, that is, 256/32=8.

Then, the GPU may determine the color value of Q ₃ (3, 6, 3) corresponding to the LUT resource according to the look-up formula and Q ₃ (3, 6, 3). Specifically, the GPU may determine the color number W of Q ₃ in the color lookup table (Table 1) according to the table lookup formula. Among them, the table lookup formula is as follows:

W＝R ₂ *33 ² +G ₂ *33 ¹ +B ₂ *33 ⁰

At this time, the color number W corresponding to Q ₃ (3, 6, 3) in Table 1:

W＝3*33 ² +6*33 ¹ +3*33 ⁰ ＝3468

At this time, by looking up Table 1, the GPU can determine that the color value of Q ₃ obtained through mapping in Table 1 is (1, 0.98204, 0).

Further, as shown in FIG. 7 , the GPU can determine another 7 color values that form a cube with Q ₃ in a three-dimensional space according to Q ₃ , which are denoted as S ₁ -S ₇ . Similarly, the GPU can determine the color numbers of S ₁ -S ₇ in the color look-up table (Table 1) according to the look-up formula, and then determine the converted color values of S ₁ -S ₇ in Table 1.

At this time, the GPU can determine the color values of each color channel corresponding to the above-mentioned Q ₃ , S ₁ -S ₇ in Table 1, as shown in Table 2:

Table 2

the	WW	the
Q ₃ Q ₃	34683468	(0，0.172259，0.0564126)(0, 0.172259, 0.0564126)
S ₁ S ₁	45574557	(0，0.171084，0.0907149)(0, 0.171084, 0.0907149)

S ₂ S ₂	35013501	(0，0.210559，0.0516823)(0, 0.210559, 0.0516823)
S ₃ S ₃	34663466	(0，0.1897，0.0623789)(0, 0.1897, 0.0623789)
S ₄ S ₄	45904590	(0，0.209201，0.0870527)(0, 0.209201, 0.0870527)
S ₅ S ₅	45584558	(0，0.163119，0.0860914)(0, 0.163119, 0.0860914)
S ₆ S ₆	45914591	(0，0.200137，0.0819867)(0, 0.200137, 0.0819867)
S ₇ S ₇	35023502	(0，0.20148，0.0486458)(0, 0.20148, 0.0486458)

Then, the GPU may use an interpolation method to determine Q ₄ (0, 0.174277, 0.035676) from the above Q ₃ and the above S ₁ -S ₇ . Q ₄ is the final color value of the pixel point Q. The above-mentioned interpolation methods include but are not limited to trilinear interpolation methods, tetrahedral interpolation methods, etc., which will not be repeated here.

Fig. 7 takes a pixel point Q in the HDR video frame to be edited as an example, and shows the specific process of converting a pixel point with a color gamut of BT2020 into a pixel point with a color gamut of BT709 using LUT resources by the GPU. By analogy, the GPU can perform the above processing on other pixels in the HDR video frame to be edited, so as to convert the color gamut of each pixel in the HDR video frame to be edited from BT2020 to BT709, and then convert the HDR video frame to be edited to SDR video frame to be edited.

FIG. 8 exemplarily shows a schematic diagram of a hardware structure of the electronic device 100 .

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory may hold instructions or data that processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present invention is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also provide power for electronic devices through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (wireless fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. applied on the electronic device 100. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

In this embodiment of the present application, the electronic device 100 may display the user interface shown in FIG. 1A-FIG. 1K through a GPU, an encoder, a decoder, OpenGL, and a display screen 194 .

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

In this embodiment of the application, the edited HDR video can be obtained by the electronic device 100 from other electronic devices through the wireless communication function, or it can be obtained by the electronic device 100 through the ISP, camera 193, video codec, GPU, display screen 194 shots were obtained.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The internal memory 121 may include one or more random access memories (random access memory, RAM) and one or more non-volatile memories (non-volatile memory, NVM).

Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory

(synchronous dynamic random access memory, SDRAM), double data rate synchronous dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM, for example, the fifth generation DDR SDRAM is generally called DDR5 SDRAM), etc. Non-volatile memory may include magnetic disk storage devices, flash memory (flash memory).

According to the operating principle, flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. According to the potential order of storage cells, it can include single-level storage cells (single-level cell, SLC), multi-level storage cells (multi-level cell, MLC), triple-level cell (TLC), quad-level cell (QLC), etc., can include universal flash storage (English: universal flash storage, UFS) according to storage specifications , Embedded multimedia card (embedded multimedia Card, eMMC), etc.

The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other running programs, and can also be used to store data of users and application programs.

The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be loaded into the random access memory in advance for the processor 110 to directly read and write.

In the embodiment of the present application, the internal memory 121 can support the electronic device 100 to apply for a surface and a conveyor bufferqueue from the memory.

The external memory interface 120 can be used to connect an external non-volatile memory, so as to expand the storage capacity of the electronic device 100 . The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, files such as music and video are stored in an external non-volatile memory. In the embodiment of the present application, when the electronic device 100 shoots the HDR video, the microphone 170C may collect sound. In the process of playing the video, the speaker 170A or the speaker connected to the earphone interface 170D can support playing the audio in the video.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the position of the display screen 194 .

In the embodiment of the present application, the electronic device 100 detects whether there is a user operation on the display screen 194 of the electronic device 100 through the touch sensor 180K. After the touch sensor 180K detects the above-mentioned user operation, the electronic device 100 may execute the image processing indicated by the above-mentioned user operation to realize video color gamut conversion.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

Implement the video editing method provided by the embodiment of the present application. When using the SDR video editor to edit the HDR video, when the SDR video editor cannot support the normal display of the HDR video, the electronic device 100 can convert the HDR video with a color gamut of BT2020 into a color gamut. The SDR video whose gamut is BT709, and the SDR video whose color gamut is BT709 is normally displayed by the above-mentioned SDR video editor. At this time, the electronic device 100 can display the above-mentioned converted SDR video instead of directly displaying the HDR video, thereby avoiding Use the SDR editor to directly display the display problems that affect the user experience such as the display of HDR video frames is not clear.

In the embodiment of this application:

1. The user's operation of clicking the edit control for triggering editing of the video service may be referred to as a first user operation, such as the operation of clicking the control 133 in FIG. 1C . When the first user operation is detected, the video currently displayed on the electronic device, that is, the video selected by the user to be edited, may be referred to as the first video, such as video A displayed in window 131 in FIG. 1C . A series of video frames obtained by decoding the first video by the decoder may be referred to as first video frames. The first video frame consists of N. The N number is determined by the duration of the first video. The video frame obtained after changing the color value of the pixels of the first video frame according to the color value and the color correspondence provided by the LUT resource may be called a second video frame, and the number of the second video frames is also N. The user interface shown in FIG. 1D may be referred to as a first interface.

2. The BT2020 color gamut can be called the first color gamut; the BT709 color gamut can be called the second color gamut.

3. The LUT resource in the format of 33*33*33 shown in Table 1 may be called the first color table.

4. Editing operations such as "editing", "filter", "music" and "text" shown in Figure 1D-Figure 1J can be referred to as editing operations selected by the user to change the display effect of the first video, that is, the first video Two user operations. The video frame obtained after the above editing operation may be referred to as the third video frame. The third video frame may include more or fewer video frames than the second video frame, and/or the third video frame may have the same number as the second video frame but have different display effects (filters, text stickers , image stickers).

5. The operation on the save control 146 in FIG. 1J may be referred to as a third user operation. The saved video shown in FIG. 1K may be referred to as a second video. The user interface shown in FIG. 1K may be referred to as a second interface.

6. The operation acting on the "split" control in Figure 1D can be called the operation of splitting the video; the operation acting on the "delete" control can be called the operation of deleting video frames; the operation acting on the "frame" control can be called cropping Manipulation of screen size. The operation of selecting the filter control 155 shown in FIGS. 1E-1F may be referred to as an operation of adding a filter. The operation of adding the "Title 5" title shown in FIGS. 1I-1J may be called adding a title or trailer.

7. The cache BufferQueue applied by the decoder can be called the first memory; the cache surface applied by the encoder can be called the second memory.

8. Pixel Q in Figure 7 can be called the first pixel, Q ₁ can be called the current color value Q1 of the first pixel, and the color values S ₁ to S ₇ can be called the 7 space cubes that form a space cube with Q1 The auxiliary color value; the color number W can be called the index value; the corresponding color values of Q1, S ₁ to S ₇ in Table 1 can be called the target color value; Q ₄ obtained after interpolation can be called the changed color value.

9. The address of video memory B in Figure 4 can be called the first video memory address; the address of video memory C in Figure 5 can be called the second video memory address.

The term "user interface (UI)" in the specification, claims and drawings of this application is a medium interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the internal form of information Conversion to and from a form acceptable to the user. The user interface of the application program is the source code written in specific computer languages such as java and extensible markup language (XML). Such as pictures, text, buttons and other controls. Control (control), also known as widget (widget), is the basic element of user interface. Typical controls include toolbar (toolbar), menu bar (menu bar), text box (text box), button (button), scroll bar (scrollbar), images and text. The properties and contents of the controls in the interface are defined through labels or nodes. For example, XML specifies the controls contained in the interface through nodes such as <Textview>, <ImgView>, and <VideoView>. A node corresponds to a control or property in the interface, and after the node is parsed and rendered, it is presented as the content visible to the user. In addition, the interfaces of many applications, such as hybrid applications, usually include web pages. A web page, also called a page, can be understood as a special control embedded in the application program interface. A web page is a source code written in a specific computer language, such as hyper text markup language (GTML), cascading style Tables (cascading style sheets, CSS), java scripts (JavaScript, JS), etc., the source code of the web page can be loaded and displayed as user-recognizable content by a browser or a web page display component similar in function to the browser. The specific content contained in the webpage is also defined by the tags or nodes in the source code of the webpage. For example, GTML defines the elements and attributes of the webpage through <p>, <img>, <video>, and <canvas>.

The commonly used form of user interface is the graphical user interface (graphic user interface, GUI), which refers to the user interface related to computer operation displayed in a graphical way. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. Visual interface elements.

As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to include Plural expressions, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in this application refers to and includes any and all possible combinations of one or more of the listed items. As used in the above embodiments, depending on the context, the term "when" may be interpreted to mean "if" or "after" or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrases "in determining" or "if detected (a stated condition or event)" may be interpreted to mean "if determining..." or "in response to determining..." or "on detecting (a stated condition or event)" or "in response to detecting (a stated condition or event)".

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state hard disk), etc.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.

Claims

A video editing method applied to electronic equipment, characterized in that the method comprises:

A first user operation is detected, the first user operation corresponds to an editing control, and is used to trigger a video editing service;

In response to the first user operation, decoding the first video into N first video frames, where the color gamut of the N first video frames is the first color gamut;

Perform color gamut conversion on the N first video frames to obtain N second video frames, the color gamut of the N second video frames is the second color gamut, and the first color gamut and the second Different color gamut;

Any one of the N second video frames is displayed on the first interface.
The method according to claim 1, characterized in that the range of colors that can be represented by the second color gamut is smaller than the range of colors that can be represented by the first color gamut.
The method according to claim 1 or 2, wherein the performing color gamut conversion on the N first video frames is specifically: performing color gamut conversion on the N first video frames using a first color table. Gamut conversion, the first color table includes a plurality of color values for changing the color gamut of the video frame.
The method according to any one of claims 1-3, wherein after the first interface displays the N second video frames, the method further comprises:

detecting a second user operation, where the second user operation is an editing operation selected by the user to change the display effect of the first video;

In response to the second user operation, increasing or decreasing the number of the second video frames, and/or changing the number of pixels of one or more of the N second video frames, and /or, change the color values of the pixels of one or more second video frames in the N second video frames to obtain M third video frames; the M is equal to or not equal to the N;

Any one of the M third video frames is displayed on the first interface.
The method according to claim 2, characterized in that the method further comprises;

detecting a third user operation, the third user operation corresponding to a save control;

In response to the third user operation, saving the M third video frames as a second video, where the second video is an edited video obtained by editing the first video;

The second video is displayed on the second interface.
The method according to claim 4, wherein the editing operation selected by the user to change the display effect of the first video includes: splitting the video, deleting video frames, adding a title or trailer, cropping the screen size, adding one or more of filters, operations that add text or graphics;

Wherein, the operations of splitting the video, deleting video frames, and adding credits or credits are used to increase or decrease the number of the second video frames, and the operation of cropping the picture size is used to change the number of the N second video frames. The number of pixels of one or more second video frames, the operation of adding a filter, adding text or graphics is used to change the number of pixels of one or more second video frames in the N second video frames color value.
The method according to claim 6, characterized in that,

When the second user operation includes an operation of increasing or decreasing the number of the second video frames, the M is not equal to the N; when the second user operation does not include increasing or decreasing the second video frame When operating on the number of video frames, the M is equal to the N.
The method according to any one of claims 1-7, wherein the first color gamut is BT2020, and the second color gamut is BT709.
The method according to any one of claims 1-8, wherein the first video is a high dynamic range (HDR) video, and the second video is a standard dynamic range (SDR) video.
The method according to any one of claims 3-8, wherein the electronic device includes a video editing application APP, a decoder, and a first memory, and the first memory is used in the encoder for caching The storage space of the video input by the APP, the decoding of the first video into N first video frames specifically includes:

The APP sends the first video to the first memory;

The decoder reads the first video from the first memory, and decomposes the first video into the N first video frames.
The method according to claim 10, wherein the electronic device further comprises an open graphics library OpenGL and a graphics processing unit (GPU), and the first color table is used to perform color gamut conversion on the N first video frames, Obtain N second video frames, specifically including:

The OpenGL receives the N first video frames sent by the decoder, the color coding format of the N first video frames is YUV format, and the data type of the data representing the color value is an integer;

The OpenGL changes the color coding format of the N first video frames into an RGB format, and changes the data type of the data representing the color value into a floating-point type;

The OpenGL calls the first color table from the GPU, uses the color value conversion relationship provided by the first color table to modify the color values of the pixels of the N first video frames, and obtains the N The second video frame; the color coding format of the N second video frames is RGB format, and the data type of the data representing the color value is a floating point type.
The method according to claim 11, wherein before the OpenGL calls the first color table from the GPU, the method further comprises: the OpenGL obtains the first color from the APP table, and load the first color table into the GPU.
The method according to claim 11 or 12, wherein the modifying the color values of the pixels of the N first video frames using the color value conversion relationship provided by the first color table specifically includes:

The OpenGL determines the current color value Q1 of the first pixel, the data type of the Q1 is an integer, and the first pixel is any pixel in any video frame of the N first video frames;

The OpenGL takes the position of the Q1 in the three-dimensional color space as the origin, and determines 7 auxiliary color values forming a space cube with the Q1;

The OpenGL determines respective index values of the Q1 and the 7 auxiliary color values in the first color table;

The OpenGL queries target color values corresponding to the Q1 and the 7 auxiliary color values in the first color table according to the index value;

The OpenGL interpolates the Q1 and the target color values of the seven auxiliary color values to obtain a changed color value, and sets the color value of the first pixel as the changed color value.
The method according to claim 13, wherein the formula for calculating the index value W of a color value (R, G, B) in the first color table is:

W=R*33 2 +G*33 1 +B*33 0 .
The method according to any one of claims 11-14, wherein the displaying any one of the N second video frames on the first interface specifically includes:

The GPU sends the first video memory address to the OpenGL, and the first video memory address is a video memory address storing the N second video frames; the OpenGL sends the first video memory address to the APP;

The APP acquires the N second video frames according to the first video memory address;

The APP displays any one of the N second video frames acquired through the first video memory address on the first interface.
The method according to claim 15, wherein in response to the second user operation, the number of the second video frames is increased or decreased, and/or, the number of the N second video frames is changed The number of pixels of one or more second video frames, and/or, change the color value of the pixels of one or more second video frames in the N second video frames to obtain M third video frames , including:

The APP sends the second user operation to the OpenGL;

The OpenGL determines according to the second user operation: realize the calculation logic of the video display effect indicated by the second user operation;

The OpenGL sends the calculation logic to the GPU;

The GPU increases or decreases the number of the second video frames according to the calculation logic, and/or changes the number of pixels of one or more second video frames in the N second video frames, and/or Or, change the color values of the pixels of one or more second video frames in the N second video frames to obtain M third video frames; the video composed of M third video frames has the second user's For the display effect indicated by the operation, the color coding format of the M third video frames is RGB format, and the data type of the data representing the color value is floating point.
The method according to claim 16, wherein displaying any one of the M third video frames on the first interface specifically includes:

The GPU sends a second video memory address to the OpenGL, and the second video memory address is a video memory address for storing the M third video frames; OpenGL sends the second video memory address to the APP;

The APP acquires the M third video frames according to the second video memory address;

The APP displays on the first interface that any video frame among the M third video frames is acquired through the second video memory address.
The method according to claim 17, wherein the electronic device further comprises an encoder and a second memory, and the storing the M third video frames as the second video specifically comprises:

The APP sends a request to call the C2D engine to the OpenGL;

In response to the request, the OpenGL invokes the C2D engine to obtain the M third video frames from the GPU; the color encoding format of the M third video frames obtained by the OpenGL is YUV format, representing color The data type of the value data is an integer;

The OpenGL sends the M third video frames to the second memory;

The encoder reads the M third video frames from the second memory, and packages the M third video frames into the second video.
The method according to claim 18, wherein before the APP sends the first video to the first memory, the method further comprises:

The APP calls mediacodec to create the encoder; the encoder applies to the memory for the first memory;

The OpenGL receives the memory identification ID and/or address of the first memory sent by the APP, and determines the first memory for receiving the video frame output by the OpenGL according to the ID and/or address;

The APP calls mediacodec to create the decoder; the decoder applies for the second memory from the memory.
An electronic device, characterized in that it includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories It is used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the method according to any one of claims 1-19 is executed.
A computer program product containing instructions, when the computer program product is run on the electronic device, the electronic device is made to execute the method according to any one of claims 1-19.
A computer-readable storage medium, comprising instructions, wherein, when the instructions are run on an electronic device, the method according to any one of claims 1-19 is executed.