WO2021120872A1 - 视频处理方法、装置与终端设备 - Google Patents

视频处理方法、装置与终端设备 Download PDF

Info

Publication number
WO2021120872A1
WO2021120872A1 PCT/CN2020/124494 CN2020124494W WO2021120872A1 WO 2021120872 A1 WO2021120872 A1 WO 2021120872A1 CN 2020124494 W CN2020124494 W CN 2020124494W WO 2021120872 A1 WO2021120872 A1 WO 2021120872A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
frame
video stream
edited
editing
Prior art date
Application number
PCT/CN2020/124494
Other languages
English (en)
French (fr)
Inventor
杜晓
周文
侯朋飞
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP20903996.5A priority Critical patent/EP4057633A4/en
Publication of WO2021120872A1 publication Critical patent/WO2021120872A1/zh
Priority to US17/843,690 priority patent/US20220328071A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/036—Insert-editing
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip

Definitions

  • This application relates to the field of video processing technology, and in particular to a video processing method, device, and terminal equipment.
  • live broadcast applications With the rapid improvement of Internet broadband and the popularity of smart phones, more and more live video applications have emerged. In these live broadcast applications, most of the content is directly provided by live broadcast users, and because live broadcast users are mostly non-professional photographers, and errors may also occur during recording, some problems are prone to occur during the playback process and cause the quality of the live broadcast to drop.
  • many video live broadcast applications provide some video editing functions for live broadcast users to use, so that live broadcast users can simply edit the live video.
  • the more common video editing function is to beautify the video, that is, the live video application provides some video beautification functions, such as adding filters and beautifying faces, etc.
  • the live video application adopts the corresponding beautification algorithm to beautify the live video.
  • the present application provides a video processing method, device, and terminal equipment to improve the flexibility of live video editing.
  • an embodiment of the present application provides a video processing method, including:
  • the original recorded video stream will be presented in the form of video frames
  • the recorded original video stream is presented in the form of video frames; and then the video frame selection operation performed by the user on the original video frame Determine the video frame to be edited; then perform video editing processing on the video frame to be edited according to the video frame editing operation input by the user to generate an edited video stream; finally, generate a target video stream based on the edited video stream and the original video stream, allowing the user to be in the video recording process Select the video frame that needs to be edited for editing, which can improve the flexibility of the video editing mode, and at the same time can ensure the real-time nature of video streaming.
  • the method further includes:
  • the application range of the editing effect is preset or determined by the user after selecting the video frame to be edited;
  • the associated frame is determined as the video frame to be edited.
  • the user can set the editing effect application range, so that the flexibility of video editing can be further improved.
  • the editing effect application range is preset; before the determining the video frame to be edited, the method further includes:
  • the editing effect application range corresponding to each editing effect is preset according to the user's editing effect application range setting operation.
  • the user can set different editing effect application ranges for different editing effects, so that the flexibility of video editing can be further improved.
  • the editing effect application range includes at least one of the following: the editing effect is applied to the corresponding to-be-edited video frame; the editing effect is applied to the corresponding to-be-edited video frame and the Part of the frames after the corresponding video frame to be edited; the editing effect is applied to the corresponding video frame to be edited and all frames after the corresponding video frame to be edited.
  • the generating a target video stream according to the edited video stream and the original video stream includes:
  • the video frames in the edited video stream are inserted into corresponding empty frame positions in the sequentially arranged transmission video stream to generate a target video stream.
  • the video frame By replacing the to-be-edited video frame in the original video stream with an empty frame, the video frame can be occupied while reducing the amount of data transmission and processing. In this way, when the target video stream is generated, the corresponding existing frame can be quickly replaced by the empty frame position.
  • the edited video frame is inserted into the transport video stream, which can improve the efficiency of generating the target video stream.
  • the method before the generating the target video stream, the method further includes:
  • the edited video stream is allocated to the second encoding channel for encoding processing.
  • the transmission efficiency of the video stream in the terminal device can be improved.
  • the number of the first encoding channel and the number of the second encoding channel is dynamically allocated according to the number of video frames in the edited video stream.
  • the transmission efficiency of the video stream in the terminal device can be further improved.
  • the inserting the video frames in the edited video stream into corresponding empty frame positions in the sequentially arranged transmission video stream according to the frame header information includes:
  • the found target video frame is inserted into the ordered transmission video stream. The position of the empty frame.
  • the method before inserting the found target video frame into the corresponding empty frame position in the sequentially arranged transmission video stream, the method further includes:
  • the method before the determining the video frame to be edited, the method further includes:
  • the inserting the video frames in the edited video stream into the corresponding empty frame positions in the sequentially arranged transmission video stream according to the frame header information includes:
  • the video frame in the edited video stream is inserted into the corresponding empty frame position in the sequentially arranged transmission video stream.
  • the generation efficiency of the target video stream can be improved.
  • the inserting the video frames in the edited video stream into corresponding empty frame positions in the sequentially arranged transport video stream according to the number of the video frame includes:
  • the video frame in the edited video stream is inserted into the corresponding empty frame position in the sequentially arranged transmission video stream.
  • the inserting the video frames in the edited video stream into corresponding empty frame positions in the sequentially arranged transmission video stream according to the frame header information to generate the target video stream includes :
  • the transmission video stream and the edited video stream are transmitted to the target device through different transmission channels to instruct the target device to insert the video frames in the edited video stream into the sequentially arranged transmission video stream according to the frame header information Generate the target video stream at the corresponding empty frame position in.
  • the transmission channel includes a first transmission channel for transmitting the transmission video stream and a second transmission channel for transmitting the edited video stream, and the first transmission channel
  • the number of transmission channels and the second transmission channel is dynamically allocated according to the number of video frames in the edited video stream.
  • the transmission efficiency of the video stream can be improved.
  • the method further includes:
  • the video frame before the target empty frame in the transmission video stream is marked as the processing completed state, and the target empty frame is all The empty frame with the earliest time in the transmission video stream.
  • the frame header information of the video frame in the edited video stream includes the editing time and/or frame size of the video frame.
  • the generating a target video stream according to the edited video stream and the original video stream includes:
  • the video frames in the video stream are all video frames to be edited, the video frames in the original video stream are replaced with corresponding video frames in the edited video stream to generate a target video stream.
  • an embodiment of the present application provides a video processing device, including:
  • the detection module is used to detect whether the user starts the video editing function during the video recording process
  • the display module is configured to present the recorded original video stream in the form of video frames when the detection module detects that the user starts the video editing function;
  • the determining module is configured to determine the video frame to be edited according to the video frame selection operation performed by the user on the video frame;
  • the first generating module is configured to perform video editing processing on the to-be-edited video frame according to the video frame editing operation input by the user to generate an edited video stream;
  • the second generating module is configured to generate a target video stream according to the edited video stream and the original video stream.
  • the determining module is further configured to:
  • the associated frame of the to-be-edited video frame After determining the to-be-edited video frame, determine the associated frame of the to-be-edited video frame according to the editing effect application range of the to-be-edited video frame, and the associated frame is the one indicated by the editing effect application range and the to-be-edited video
  • the frames use video frames with the same editing effect, and the editing effect application range is preset or determined by the user after selecting the video frame to be edited;
  • the associated frame is determined as the video frame to be edited.
  • the editing effect application range is preset; the device further includes:
  • the setting module is configured to preset the editing effect application range corresponding to each editing effect according to the user's editing effect application range setting operation before the determining module determines the video frame to be edited.
  • the editing effect application range includes at least one of the following: the editing effect is applied to the corresponding to-be-edited video frame; the editing effect is applied to the corresponding to-be-edited video frame and the Part of the frames after the corresponding video frame to be edited; the editing effect is applied to the corresponding video frame to be edited and all frames after the corresponding video frame to be edited.
  • the second generation module is specifically configured to:
  • the video frames in the edited video stream are inserted into corresponding empty frame positions in the sequentially arranged transmission video stream to generate a target video stream.
  • the second generation module is further configured to:
  • the edited video stream is allocated to the second encoding channel for encoding processing.
  • the numbers of the first encoding channel and the second encoding channel are dynamically allocated according to the number of video frames in the edited video stream.
  • the second generation module is specifically configured to:
  • the found target video frame is inserted into the ordered transmission video stream. The position of the empty frame.
  • the second generation module is further configured to:
  • the device further includes:
  • the numbering module is configured to number the video frames in the original video stream before the determining module determines the video frame to be edited, and add the number to the frame header information of the corresponding video frame;
  • the second generating module is specifically configured to insert the video frames in the edited video stream into corresponding empty frame positions in the sequentially arranged transmission video stream according to the number of the video frame.
  • the second generation module is specifically configured to:
  • the video frame in the edited video stream is inserted into the corresponding empty frame position in the sequentially arranged transmission video stream.
  • the second generation module is specifically configured to:
  • the transmission video stream and the edited video stream are transmitted to the target device through different transmission channels to instruct the target device to insert the video frames in the edited video stream into the sequentially arranged transmission video stream according to the frame header information Generate the target video stream at the corresponding empty frame position in.
  • the transmission channel includes a first transmission channel for transmitting the transmission video stream and a second transmission channel for transmitting the edited video stream.
  • the number of transmission channels and the second transmission channel is dynamically allocated according to the number of video frames in the edited video stream.
  • the second generation module is further configured to:
  • the video frame before the target empty frame in the transmission video stream is marked as the processing completed state, and the target empty frame is all The empty frame with the earliest time in the transmission video stream.
  • the frame header information of the video frame in the edited video stream includes the editing time and/or frame size of the video frame.
  • the second generation module is specifically configured to:
  • the video frames in the video stream are all video frames to be edited, the video frames in the original video stream are replaced with corresponding video frames in the edited video stream to generate a target video stream.
  • an embodiment of the present application provides a terminal device, including: a memory and a processor, the memory is used to store a computer program; the processor is used to execute the first aspect or any one of the implementation manners of the first aspect when the computer program is invoked The method described.
  • an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the method described in the first aspect or any one of the first aspects is implemented.
  • the embodiments of the present application provide a computer program product, which when the computer program product runs on a terminal device, causes the terminal device to execute the method described in the first aspect or any one of the implementation manners of the first aspect.
  • FIG. 1 is a schematic flowchart of a video processing method provided by an embodiment of the application
  • FIG. 2 is a schematic diagram of an interface of a video editing interface provided by an embodiment of the application.
  • FIG. 3 is a schematic flowchart of a method for generating a target video stream provided by an embodiment of the application
  • FIG. 4 is a schematic diagram of encoding channel allocation provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of another encoding channel allocation provided by an embodiment of this application.
  • FIG. 6 is a schematic diagram of a process of generating a target video stream provided by an embodiment of the application.
  • FIG. 7 is a schematic structural diagram of a video processing device provided by an embodiment of the application.
  • FIG. 8 is a schematic structural diagram of a terminal device provided by an embodiment of the application.
  • the current live video editing method can only edit all video frames, and the editing method is not flexible enough.
  • the embodiments of the present application provide a video processing method, device, and terminal equipment, which are mainly used in the video recording process. , In the case of detecting that the user starts the video editing function, present the recorded original video stream in the form of video frames; then determine the video frame to be edited according to the video frame selection operation performed by the user on the original video frame; and then according to the user input The video frame editing operation performs video editing processing on the video frame to be edited to generate the edited video stream; finally, the target video stream is generated according to the edited video stream and the original video stream, so that the user can select the video frame to be edited during the video recording process. Improve the flexibility of video editing methods.
  • the technical solutions provided in the embodiments of the present application may be applied to live broadcast scenarios, and may also be applied to other scenarios where recorded video streams are transmitted in real time.
  • the live broadcast scenario is used as an example for illustration in the embodiments of the present application.
  • the terminal device may be an electronic device capable of displaying and processing video, such as a smart phone, a computer, a tablet computer, a smart wearable device, or a video recorder.
  • FIG. 1 is a schematic flowchart of a video processing method provided by an embodiment of this application. As shown in FIG. 1, the method provided in this embodiment may include the following steps:
  • the terminal device can provide a video editing function startup logo, for example, a video editing function startup icon can be provided on the video recording interface, and the user can start the video editing function by selecting the logo; or the terminal device can also predefine the startup video editing function.
  • a video editing function startup icon can be provided on the video recording interface, and the user can start the video editing function by selecting the logo; or the terminal device can also predefine the startup video editing function.
  • the terminal device can also predefine a voice command for starting the video editing function, and the user can start the video editing function by inputting the voice command.
  • the terminal device can provide a preview function.
  • the user can turn on the preview function and start the video editing function on the preview interface. For example, the user can start the video editing function by clicking on the preview interface.
  • the terminal device After detecting that the user starts the video editing function, the terminal device can display the video editing interface for the user to perform video editing related operations.
  • Figure 2 is a schematic diagram of the video editing interface provided by an embodiment of the application. As shown in Figure 2, the terminal device can present the original video stream in the form of video frames in the video editing interface, that is, the terminal device can split the original video stream. The video frames are divided into one by one, and the video frames obtained after the splitting are displayed, so that the user can select the video frame to be edited.
  • the icons of the video frames can be displayed in sequence in the video frame display area, where the video frame display area can be video editing as shown in Figure 2 Part of the display area in the interface.
  • the video editing interface can also include an editing area for editing and displaying selected video frames and/or a functional area for providing function icons; the video frame display area can also be a video editing interface
  • the terminal device can display the editing area and/or the function area after the user selects the video frame.
  • S120 Determine the video frame to be edited according to the video frame selection operation performed by the user on the video frame.
  • the user can browse more video frames by performing a sliding operation on the video frame display area, and perform a video frame selection operation on the displayed video frame to determine the video frame to be edited.
  • the video frame selection operation may be a video frame click operation or other predefined video frame selection gestures.
  • the user can set the editing effect application range, that is, the video frame to which the editing effect of a certain video frame to be edited can be applied; the terminal device determines the video frame to be edited, and then Determine the associated frame of the to-be-edited video frame according to the application range of the editing effect of the to-be-edited video frame, and determine the associated frame as the to-be-edited video frame.
  • the associated frame is a video frame that uses the same editing effect as the to-be-edited video frame indicated by the editing effect application range.
  • the application scope of the editing effect may include at least one of the following: the editing effect is applied to the corresponding to-be-edited video frame; the editing effect is applied to the corresponding to-be-edited video frame and a part of the frame after the corresponding to-be-edited video frame; the editing effect It is applied to the corresponding video frame to be edited and all frames after the corresponding video frame to be edited.
  • the user can determine the editing effect application range after selecting the video frame to be edited. Specifically, it can be determined before or after editing a video frame to be edited. For example, the user adds an advertisement to a video frame to be edited. Information, and then choose to display once every 30s, that is, the editing effect is applied to the video frame (ie, the associated frame) that is an integer multiple of the target time interval (30s) between the editing effect and the video frame to be edited, and the terminal device will use these associated frames It is also determined to be the video frames to be edited, and the above-mentioned advertisement information is automatically added to these associated frames.
  • the user can also set the editing effect application range in the application setting function before determining the video frame to be edited.
  • the terminal device presets the editing effect application range according to the user’s editing effect application range setting operation, and applies the editing effect application range to All the user's video frame editing operations are in progress.
  • the editing effect application range preset by the user is that the editing effect is applied to all the video frames to be edited selected by the user in a video frame selection process, that is, the editing effect is applied to the corresponding to-be-edited video frame and the corresponding to-be-edited video frame
  • the part of the frame is the video frame to be edited selected by the user; then every time the user edits the video, the terminal device will apply the editing effect of the first video frame to be edited selected by the user to the user this time All selected video frames to be edited.
  • the associated frame is the selected video frame to be edited except the first video frame to be edited. The user only needs to edit the selected first video frame to be edited each time That's it.
  • the user can set different editing effect application ranges for different editing effects, that is, the terminal device can preset the editing effect corresponding to each editing effect according to the user's editing effect application range setting operation.
  • the scope of effect application is not limited to the editing effect application.
  • the user can set the added advertisement to be displayed every 30, that is, the time interval between the editing effect and the video frame to be edited is an integer multiple of the target time interval (30s). );
  • the user can set to apply only to the currently edited video frame to be edited, that is, to apply the editing effect to the corresponding video frame to be edited;
  • the background replacement function the user can set to all video frames Perform background replacement, that is, the editing effect is applied to the corresponding video frame to be edited and all frames after the corresponding video frame to be edited.
  • the terminal device will determine the video frame whose time interval with the video frame to be edited is an integer multiple of 30s as the video frame of the video frame to be edited Associated frame; if the user erases a video frame to be edited, the associated frame of the video frame to be edited determined by the terminal device is 0; if the user replaces the background of a video frame to be edited, the terminal device will All video frames after the edited video frame are determined to be associated frames of the to-be-edited video frame.
  • S130 Perform video editing processing on the to-be-edited video frame according to the video frame editing operation input by the user to generate an edited video stream.
  • the user can input a video frame editing operation to perform video editing on the video frame to be edited.
  • the terminal device can display the editing area and the functional area on the video editing interface, and the functional area can provide various predefined editing function logos (for example: text logo or as shown in Figure 2 Function icon), the user can select the function icon in the function area to edit the to-be-edited video frame displayed in the editing area.
  • the terminal device can automatically display the first video frame to be edited selected by the user in the editing area, and the user can also display the first video frame to be edited in the editing area by inputting related operations (for example, a drag operation).
  • the predefined editing functions can be some simpler editing functions, such as: erasing function, the user can use this function to erase or delete a certain area or object in the video frame to be edited; the advertisement adding function, the user can use This function adds specified advertisements in certain positions in the video frame to be edited; the image addition function allows users to add preset pictures or animations in certain positions in the video frame to be edited; the background replacement function allows users to use This function replaces the background of the video frame to be edited; the beautification function allows the user to change the color and/or hair style of the character in the video frame to be edited.
  • the predefined editing function in the terminal device may include at least one of the above-listed editing functions, and may also include other editing functions, which is not particularly limited in this embodiment.
  • the user’s video frame editing operations include a series of operations performed by the user in the process of editing video frames, such as: editing the video frame to be edited by selecting the editing function of the functional area and the aforementioned operations after selecting the video frame to be edited
  • the operation of determining the application scope of the editing effect the terminal device performs the corresponding video editing processing on the video frame to be edited based on the user's video frame editing operation, so that the edited video frame can present the corresponding editing effect during playback; then it can be edited according to the sequence
  • the obtained video frames (hereinafter referred to as edited video frames) generate an edited video stream, where the order of the edited video frames in the edited video stream can be sorted according to the editing completion time of the video frames.
  • the frame header information of the video frame in the edited video stream includes the frame header information of the corresponding video frame to be edited.
  • it may also include editing information such as the editing time and/or frame size of the video frame.
  • the editing time may be the editing start time or the editing completion time; the edited video frame may also include other editing information such as editing effects, which can be specifically selected according to needs, which is not particularly limited in this embodiment.
  • S140 Generate a target video stream according to the edited video stream and the original video stream.
  • the terminal device generates the edited video stream in addition to the original video stream, the original video stream does not need to be stopped, and the subsequent processing is continued; when the target video stream is generated, the video frames in the edited video stream are inserted into the original video stream In the corresponding position, it can ensure the real-time transmission of video streaming.
  • the original video stream and the edited video stream need to be transmitted and processed between the functional modules inside the terminal device.
  • the original video stream can be The video data of the video frame to be edited is removed to reduce the amount of data transmission and processing; in addition, when the target video stream is generated, the video frame in the edited video stream needs to be inserted into the corresponding position in the original video stream, in order to reduce data transmission and While processing volume, the efficiency of video stream generation is improved.
  • the to-be-edited video frame in the original video stream can be replaced with an empty frame to occupy the space, so that when the target video stream is synthesized in the later stage, the empty frame position can be quickly replaced.
  • the corresponding edited video frame is inserted into the transmission video stream.
  • FIG. 3 is a schematic flowchart of a method for generating a target video stream provided by an embodiment of the application. As shown in FIG. 3, in step S140, the actual internal implementation may include the following steps:
  • the empty frame may only include the header information of the corresponding video frame to be edited, where the frame header information contains some basic information of the video frame to be edited, for example: frame size And frame position, etc., where the frame position is the position of the video frame to be edited in the original video stream, which may specifically be a timestamp.
  • S142 Allocate the transmission video stream to the first encoding channel for encoding processing, and allocate the edited video stream to the second encoding channel for encoding processing.
  • the two can be combined to generate the target video stream.
  • the transmission video stream and the edited video stream may be encoded and compressed.
  • multiple encoding channels can be predefined in the terminal device.
  • one part of the encoding channel ie, the first encoding channel
  • the other part of the encoding channel ie, the second Encoding channel
  • the number of the first encoding channel and the second encoding channel can be dynamically allocated according to the number of video frames in the edited video stream to improve the transmission and encoding efficiency of the video stream.
  • the encoding channel can be implemented based on the internal video stream transmission channel of the terminal, that is, the video stream is encoded and compressed in the internal video stream transmission channel to form an encoding channel.
  • the following uses three predefined encoding channels in the terminal device as an example to illustrate the allocation method of encoding channels.
  • Figure 4 is a schematic diagram of encoding channel allocation provided by an embodiment of the application.
  • frame 2, frame 3, and frame 4 in the original video stream are edited, and there are fewer edited video frames.
  • two encoding channels ie, the first encoding channel
  • frame 1, frame 3 (empty frame) and frame 5 is allocated to one of the first encoding channels
  • frame 4 (empty frame) and frame 6 are allocated to another first encoding channel
  • the third encoding channel ie, the second encoding channel Used to transmit and encode video frames in the edited video stream: frame 2 (after editing), frame 3 (after editing) and frame 4 (after editing).
  • Figure 5 is a schematic diagram of another encoding channel allocation provided by an embodiment of the application.
  • the user applies the editing result to all subsequent frames.
  • one encoding channel ie, the first One encoding channel
  • the video frames in the transmission video stream are all empty frames, as shown in Figure 5, frame 1 (empty frame) and frame 2 (empty frame).
  • Frame) and frame 3 empty frame
  • the other two encoding channels ie, the second encoding channel
  • frame 1 after editing
  • Frame 3 after editing
  • frame 5 after editing
  • frame 2 after editing
  • frame 4 after editing
  • frame 6 after editing
  • Figure 5 only shows 3 empty frames.
  • the empty frames corresponding to frame 4, frame 5, and frame 6 can be processed by the first encoding channel for frame 1, frame 2, and frame 3. The processing is performed after the empty frame.
  • the number of frames processed in the foregoing encoding channel is only an example, and the specific number of frames that can be processed can be determined according to the allocated memory, which is not particularly limited in this embodiment.
  • the specific allocation rules can also be determined according to needs. For example, in the above figure 4, it can also be frame 1, frame 2 (empty frame) and frame 3 ( The empty frame) is allocated to one of the first encoding channels, and the frame 4 (empty frame), frame 5, and frame 6 are allocated to the other first encoding channel.
  • the specific allocation rules are not particularly limited in this embodiment.
  • the edited video stream can be merged into the transmission video stream to generate the target video stream.
  • the video frames in the transmission video stream can be sorted according to the frame header information, so that the video frames in the transmission video stream are arranged in order; then for each empty frame in the transmission video stream, edit the video After the target video frame corresponding to the empty frame is found in the stream, the found target video frame is inserted into the corresponding empty frame position in the sequentially arranged transmission video stream.
  • the order of each video frame in the transmission video stream after encoding and compression processing can be determined according to the frame position in the frame header information, and each video frame is sorted according to the sequence.
  • the empty frames in the transmission video stream can be processed in sequence according to the sequence of the video frames. For each empty frame, you can also find the video with the same frame position in the edited video stream according to the frame position in the frame header information. Frame (ie the target video frame), after finding it, insert it into the corresponding empty frame position.
  • the target video frame found can be checked first according to the frame header information, and the If the target video frame is successfully checked, insert it into the corresponding empty frame position in the transport video stream.
  • the frame header information contains more information about the video frame. In the specific verification, you can compare the frame header information of the empty frame and the corresponding target video frame to determine the similarity between the two, and determine the correction when the target similarity is satisfied. The test was successful.
  • the video frames in the original video stream may be numbered first, and the numbers are added to the frame header information of the corresponding video frames. Then the terminal device can sort the video frames in the transmission video stream according to the number of the video frame, and insert the video frames in the edited video stream into the corresponding empty frame position in the ordered transmission video stream according to the number of the video frame .
  • a terminal device edits a video stream, it generally puts the video stream into a video stream buffer (buffer) for processing in sequence, and when numbering, the original video stream in the video stream buffer can be numbered.
  • the video frame to be edited in the video stream buffer is replaced with an empty frame, it is sent to the encoding channel for encoding processing.
  • the transmission video stream After the transmission video stream is encoded, it can be put into the transmission buffer. At this time, it can be transmitted according to the number of the video frame.
  • the video frames in the buffer are sorted.
  • the video frames to be edited in the video stream buffer are extracted for video editing and encoding, they can be put into the editing buffer; after the video frames in the transmission buffer are sorted, the empty frames can be added to the editing queue; After the incoming video frames in the editing buffer, the empty frames in the editing queue can be processed in turn, the corresponding target video frame is searched for the empty frame in the editing buffer according to the number of the video frame, and then the corresponding empty frame position in the transmission buffer is inserted. Among them, after the target video frame of a certain empty frame is inserted, the corresponding empty frame in the editing queue can be deleted, and then the next empty frame in the editing queue can be processed.
  • Fig. 6 is a schematic diagram of the process of generating a target video stream provided by an embodiment of the application.
  • the transmission video stream in the transmission buffer includes: frame 1, frame 2 (empty frame), frame 3 (empty frame), frame 4 (empty frame), frame 5 and frame 6.
  • the editing video stream in the editing buffer includes: frame 2 (after editing), frame 3 (after editing) and frame 4 (after editing); when merging, frame 2 (After editing) insert the corresponding empty frame position: frame 2 (empty frame), frame 3 (after editing) insert the corresponding empty frame position: frame 3 (empty frame), frame 4 (after editing) insert the corresponding empty frame position : Frame 4 (empty frame), generate the target video stream, including: frame 1, frame 2 (after editing), frame 3 (after editing), frame 4 (after editing), frame 5 and frame 6.
  • the number and frame position of the video frame can be combined to sort and merge the video stream at the same time, that is, the video in the transmission video stream is transferred according to the number and frame position of the video frame.
  • the frames are sorted, and the video frames in the edited video stream are inserted into the corresponding empty frame positions in the sequentially arranged transmission video stream according to the number and frame position of the video frame.
  • the video frames in the transmission video stream can be sorted according to the number of the video frame, and then the order of the video frames in the transmission video stream is checked according to the position of the video frame. If it is not correct, then adjust; similarly, when searching for the target video frame, you can first find the target video frame corresponding to the empty frame in the edited video stream according to the number of the video frame, and then check and find it according to the position of the video frame If the target video frame is correct, when the check passes, the target video frame is inserted into the corresponding empty frame position in the transmission video stream.
  • the video frame before the target empty frame in the transmission video stream may be marked as a processing completed state, where the target empty frame It is the earliest empty frame in the transmission video stream.
  • the video frame is marked as being processed, indicating that the video frame can be subjected to subsequent processing, such as transmission, storage, or display.
  • the target empty frame can be determined according to the above-mentioned editing queue, and the target empty frame is the empty frame at the head of the queue in the editing queue.
  • the terminal device may also transmit the transmission video stream and the editing video stream to the target device through different transmission channels, and instruct the target device to edit the video stream according to the frame header information. Insert the corresponding empty frame positions in the sequentially arranged transmission video stream to generate the target video stream.
  • the target device may be other terminal devices, and the process of generating the target video stream by the target device is similar to the process of generating the target video stream by the terminal device described above, and will not be repeated here.
  • the transmission channel includes the first transmission channel used to transmit the transmission video stream and the second transmission channel used to transmit the edited video stream. Similarly, in order to improve the transmission efficiency, the number of the first transmission channel and the second transmission channel can be based on the edited video
  • the number of video frames in the stream is dynamically allocated, and the allocation principle is similar to the allocation principle of the encoding channel, and will not be repeated here.
  • the editing effect is applied to all subsequent video frames
  • empty frames can be omitted, and the video frames in the original video stream can be directly replaced with
  • the edited video frame generates the target video stream, that is, when the video frames in the video stream are all the video frames to be edited, replace the video frames in the original video stream with the corresponding video frames in the edited video stream to generate The target video stream.
  • the edited video frame after editing the video frame to be edited, can be directly inserted into the video stream buffer corresponding to the original video stream to realize the video frame replacement of the original video stream; then, the replaced video stream can be performed Processing, the processing process is similar to the current video stream processing process, that is, all the encoding channels are used to process the edited video frames. After the encoding process, the encoded edited video frames are put into the transmission buffer, and then sorted. Generate the target video stream.
  • the video processing method provided in this embodiment presents the recorded original video stream in the form of video frames when it is detected that the user has started the video editing function during the video recording process; and then according to the user’s processing of the original video frame
  • the video frame selection operation determines the video frame to be edited; then the video frame to be edited is processed according to the video frame editing operation input by the user to generate the edited video stream; finally, the target video stream is generated according to the edited video stream and the original video stream, so that the user can In the video recording process, select the video frame that needs to be edited for editing, which can improve the flexibility of the video editing mode, and at the same time can ensure the real-time nature of video streaming.
  • an embodiment of the present application provides a video processing device.
  • the device embodiment corresponds to the foregoing method embodiment.
  • this device embodiment no longer compares the foregoing method embodiment.
  • the detailed content in will be repeated one by one, but it should be clear that the device in this embodiment can correspondingly implement all the content in the foregoing method embodiment.
  • FIG. 7 is a schematic structural diagram of a video processing device provided in an embodiment of the application. As shown in FIG. 7, the device provided in this embodiment includes:
  • the detection module 110 is used to detect whether the user starts the video editing function during the video recording process
  • the display module 120 is configured to present the recorded original video stream in the form of video frames when the detection module 110 detects that the user starts the video editing function;
  • the determining module 130 is configured to determine the video frame to be edited according to the video frame selection operation performed by the user on the video frame;
  • the first generating module 140 is configured to perform video editing processing on the to-be-edited video frame according to the video frame editing operation input by the user to generate an edited video stream;
  • the second generating module 150 is configured to generate a target video stream according to the edited video stream and the original video stream.
  • the determining module 130 is further configured to:
  • the associated frame is the video frame that uses the same editing effect as the to-be-edited video frame indicated by the editing effect application range.
  • the effect application range is preset or determined by the user after selecting the video frame to be edited;
  • the editing effect application range is preset; the device further includes:
  • the setting module 160 is configured to preset the editing effect application range corresponding to each editing effect according to the user's editing effect application range setting operation before the determining module 130 determines the video frame to be edited.
  • the editing effect application scope includes at least one of the following: the editing effect is applied to the corresponding to-be-edited video frame; the editing effect is applied to the corresponding to-be-edited video frame and the corresponding to-be-edited video frame; Edit the part of the frame after the video frame; the editing effect is applied to the corresponding video frame to be edited and all frames after the corresponding video frame to be edited.
  • the second generation module 150 is specifically configured to:
  • the empty frame includes the frame header information of the corresponding video frame to be edited
  • the video frames in the edited video stream are inserted into the corresponding empty frame positions in the sequentially arranged transmission video stream to generate the target video stream.
  • the second generation module 150 is further configured to:
  • the edited video stream is allocated to the second encoding channel for encoding processing.
  • the numbers of the first encoding channel and the second encoding channel are dynamically allocated according to the number of video frames in the edited video stream.
  • the second generation module 150 is specifically configured to:
  • the found target video frame is inserted into the corresponding empty frame position in the ordered transmission video stream.
  • the second generation module 150 is further configured to:
  • the device further includes:
  • the numbering module 170 is configured to number the video frames in the original video stream before the determining module 130 determines the video frame to be edited, and add the number to the frame header information of the corresponding video frame;
  • the second generating module 150 is specifically configured to insert the video frames in the edited video stream into corresponding empty frame positions in the sequentially arranged transmission video stream according to the number of the video frame.
  • the second generation module 150 is specifically configured to:
  • the video frame in the edited video stream is inserted into the corresponding empty frame position in the transmission video stream arranged in sequence.
  • the second generation module 150 is specifically configured to:
  • the transmission video stream and the edited video stream are transmitted to the target device through different transmission channels to instruct the target device to insert the video frames in the edited video stream into the corresponding empty frame positions in the sequentially arranged transmission video stream according to the frame header information to generate The target video stream.
  • the transmission channel includes a first transmission channel for transmitting a transmission video stream and a second transmission channel for transmitting an edited video stream.
  • the number is dynamically allocated according to the number of video frames in the edited video stream.
  • the second generation module 150 is further configured to:
  • the frame header information of the video frame in the edited video stream includes the editing time and/or frame size of the video frame.
  • the second generation module 150 is specifically configured to:
  • the video frames in the video stream are all video frames to be edited, the video frames in the original video stream are replaced with corresponding video frames in the edited video stream to generate the target video stream.
  • the device provided in this embodiment can execute the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.
  • FIG. 8 is a schematic structural diagram of a terminal device provided by an embodiment of the application.
  • the terminal device provided in this embodiment may include: a memory 210, a processor 220, an input unit 230, a display unit 240 and other components
  • the memory 210 may be used to store computer programs and modules, and the processor 220 may execute the methods described in the foregoing method embodiments by running the computer programs and modules stored in the memory 210.
  • the memory 210 may mainly include a program storage area and a data storage area.
  • the program storage area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.;
  • the data (such as audio data, phone book, etc.) created by the use of the terminal device, etc.
  • the memory 210 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.
  • the input unit 230 may be used to receive inputted numeric or character information, and generate key signal input related to user settings and function control of the terminal device.
  • the input unit 230 may include a touch panel 231 and other input devices 232.
  • the touch panel 231 also called a touch screen, can collect the user's touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on the touch panel 231 or near the touch panel 231. Operation), and drive the corresponding connection device according to the preset program.
  • the touch panel 231 may include two parts: a touch detection device and a touch controller.
  • the touch detection device detects the user's touch position, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 220, and can receive and execute the commands sent by the processor 220.
  • the touch panel 231 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave.
  • the input unit 230 may also include other input devices 232.
  • the other input device 232 may include, but is not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackball, mouse, and joystick.
  • the display unit 240 may be used to display information input by the user or information provided to the user and various menus of the terminal device.
  • the display unit 240 may include a display panel 241.
  • the display panel 241 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.
  • the touch panel 231 can cover the display panel 241. When the touch panel 231 detects a touch operation on or near it, it transmits it to the processor 220 to determine the type of the touch event, and then the processor 220 responds to the touch event. The type provides corresponding visual output on the display panel 241.
  • the touch panel 231 and the display panel 241 are used as two independent components to implement the input and input functions of the terminal device, but in some embodiments, the touch panel 231 and the display panel 241 may be integrated And realize the input and output functions of terminal equipment.
  • the processor 220 is the control center of the terminal device. It uses various interfaces and lines to connect the various parts of the entire terminal device, runs or executes software programs and/or modules stored in the memory 210, and calls data stored in the memory 210 , Perform various functions of the terminal equipment and process data, so as to monitor the terminal equipment as a whole.
  • the processor 220 may include one or more processing units.
  • the terminal device also includes a power source (such as a battery) (not shown) for supplying power to various components.
  • a power source such as a battery
  • the power source can be logically connected to the processor 220 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. Management and other functions.
  • the terminal device may also include a camera (not shown).
  • the position of the camera on the terminal device may be front-mounted or rear-mounted, which is not limited in the embodiment of the present application.
  • the terminal device may include a single camera, a dual camera, or a triple camera, etc., which is not limited in the embodiment of the present application.
  • the terminal device may include three cameras, of which one is a main camera, one is a wide-angle camera, and one is a telephoto camera.
  • the multiple cameras may be all front-mounted, or all rear-mounted, or part of the front-mounted and the other part of the rear-mounted, which is not limited in the embodiment of the present application.
  • the terminal device provided in this embodiment can execute the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.
  • FIG. 8 is only an example of a terminal device, and does not constitute a limitation on the terminal device. It may include more or less components than shown in the figure, or combine certain components, or different component arrangements.
  • the embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.
  • the embodiments of the present application also provide a computer program product, which when the computer program product runs on a terminal device, enables the terminal device to execute the method described in the above method embodiment.
  • the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the computer program can be stored in a computer-readable storage medium.
  • the computer program can be stored in a computer-readable storage medium.
  • the steps of the foregoing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms.
  • the computer-readable storage medium may at least include: any entity or device capable of carrying the computer program code to the photographing device/terminal device, recording medium, computer memory, read-only memory (ROM), random access Memory (Random Access Memory, RAM), electrical carrier signal, telecommunications signal, and software distribution medium.
  • ROM read-only memory
  • RAM random access Memory
  • electrical carrier signal telecommunications signal
  • software distribution medium for example, U disk, mobile hard disk, floppy disk or CD-ROM, etc.
  • computer-readable media cannot be electrical carrier signals and telecommunication signals.
  • the processor can be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), ready-made programmable Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM).
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • the disclosed apparatus/equipment and method may be implemented in other ways.
  • the device/equipment embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the term “if” can be construed as “when” or “once” or “in response to determination” or “in response to detecting “.
  • the phrase “if determined” or “if detected [described condition or event]” can be interpreted as meaning “once determined” or “in response to determination” or “once detected [described condition or event]” depending on the context ]” or “in response to detection of [condition or event described]”.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

本申请提供一种视频处理方法、装置与终端设备，涉及视频处理技术领域，其中，该方法包括：在视频录制过程中，若检测到用户启动视频编辑功能，则将录制的原始视频流以视频帧形式进行呈现；根据用户对视频帧进行的视频帧选择操作，确定待编辑视频帧；根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流；根据编辑视频流和原始视频流生成目标视频流。本申请提供的技术方案可以提高直播视频编辑的灵活性。

Description

视频处理方法、装置与终端设备
本申请要求于2019年12月18日提交国家知识产权局、申请号为201911309949.9、申请名称为“视频处理方法、装置与终端设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及视频处理技术领域，尤其涉及一种视频处理方法、装置与终端设备。
背景技术
随着网络宽带的迅速提升和智能手机的普及，越来越多的视频直播应用涌现出来。在这些直播应用中，大部分内容由直播用户来直接提供，而由于直播用户大多是非专业拍摄人员，而且录制过程中也可能出现失误，因而播放过程中容易出现一些问题而导致直播质量下降。
为了提高直播质量，许多视频直播应用都提供了一些视频编辑功能供直播用户使用，使直播用户可以对直播的视频进行简单的编辑。目前，比较常见的视频编辑功能是对视频进行美化，即视频直播应用中提供一些视频美化功能，例如：添加滤镜和美化人脸等，当用户在视频直播过程中选择某视频美化功能时，视频直播应用采用对应的美化算法对直播的视频进行美化。
但是，目前这些常见的视频编辑功能，视频编辑方式不够灵活。
发明内容
有鉴于此，本申请提供一种视频处理方法、装置与终端设备，用于提高直播视频编辑的灵活性。
为了实现上述目的，第一方面，本申请实施例提供一种视频处理方法，包括：
在视频录制过程中，若检测到用户启动视频编辑功能，则将录制的原始视频流以视频帧形式进行呈现；
根据用户对所述视频帧进行的视频帧选择操作，确定待编辑视频帧；
根据用户输入的视频帧编辑操作对所述待编辑视频帧进行视频编辑处理，生成编辑视频流；
根据所述编辑视频流和所述原始视频流生成目标视频流。
本实施例中，通过在视频录制过程中，在检测到用户启动视频编辑功能的情况下，将录制的原始视频流以视频帧形式进行呈现；然后根据用户对原始视频帧进行的视频帧选择操作确定待编辑视频帧；再根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流；最后根据编辑视频流和原始视频流生成目标视频流，可以使用户 在视频录制过程中选择需要编辑的视频帧进行编辑，从而可以提高视频编辑方式的灵活性，同时还可以保证视频流传输的实时性。
在第一方面的一种可能的实施方式中，在所述确定待编辑视频帧之后，所述方法还包括：
根据所述待编辑视频帧的编辑效果应用范围确定所述待编辑视频帧的关联帧，所述关联帧为所述编辑效果应用范围指示的与所述待编辑视频帧采用相同编辑效果的视频帧，所述编辑效果应用范围是预先设置的或者用户在选择完待编辑视频帧后确定的；
将所述关联帧确定为待编辑视频帧。
通过上述实施方式，可以使用户设置编辑效果应用范围，从而可以进一步提高视频编辑的灵活性。
在第一方面的一种可能的实施方式中，所述编辑效果应用范围是预先设置的；在所述确定待编辑视频帧之前，所述方法还包括：
根据用户的编辑效果应用范围设置操作预先设置各编辑效果对应的编辑效果应用范围。
通过上述实施方式，用户可以为不同的编辑效果设置不同的编辑效果应用范围，从而可以进一步提高视频编辑的灵活性。
在第一方面的一种可能的实施方式中，所述编辑效果应用范围包括下列中的至少一种：编辑效果应用于对应的待编辑视频帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的部分帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的所有帧。
在第一方面的一种可能的实施方式中，所述根据所述编辑视频流和所述原始视频流生成目标视频流，包括：
将所述原始视频流中的待编辑视频帧替换为空帧，生成传输视频流，所述空帧中包括对应的待编辑视频帧的帧头信息；
根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
通过将原始视频流中的待编辑视频帧替换为空帧，可以在减少数据传输和处理量的同时进行视频帧占位，这样在生成目标视频流时，可以通过空帧位置快速将对应的已编辑视频帧插入传输视频流，从而可以提高目标视频流生成效率。
在第一方面的一种可能的实施方式中，所述生成目标视频流之前，所述方法还包括：
将所述传输视频流分配至第一编码通道进行编码处理；
将所述编辑视频流分配至第二编码通道进行编码处理。
通过上述实施方式，可以提高视频流在终端设备中的传输效率。
在第一方面的一种可能的实施方式中，所述第一编码通道和所述第二编码通道的数量根据所述编辑视频流中视频帧的数量动态分配。
通过上述实施方式，可以进一步提高视频流在终端设备中的传输效率。
在第一方面的一种可能的实施方式中，所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
根据帧头信息对所述传输视频流中的视频帧进行排序处理，使所述传输视频流中的视 频帧按序排列；
对于所述传输视频流中的每个空帧，在所述编辑视频流中查找到所述空帧对应的目标视频帧后，将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置。
在第一方面的一种可能的实施方式中，在所述将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置之前，所述方法还包括：
根据帧头信息对查找到的目标视频帧进行校验；
确定所述查找到的目标视频帧校验成功。
通过上述实施方式，可以提高生成的目标视频流的数据准确性。
在第一方面的一种可能的实施方式中，在所述确定待编辑视频帧之前，所述方法还包括：
对所述原始视频流中的视频帧进行编号，并将编号加入对应的视频帧的帧头信息中；
所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
根据视频帧的编号将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
通过上述实施方式，可以提高目标视频流的生成效率。
在第一方面的一种可能的实施方式中，所述根据视频帧的编号将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
根据视频帧的编号和帧位置将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
通过上述实施方式，可以在提高目标视频流的生成效率的同时，提高生成的目标视频流的数据准确性。
在第一方面的一种可能的实施方式中，所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流，包括：
将所述传输视频流和所述编辑视频流通过不同的传输通道传输给目标设备，以指示所述目标设备根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
在第一方面的一种可能的实施方式中，所述传输通道包括用于传输所述传输视频流的第一传输通道和用于传输所述编辑视频流的第二传输通道，所述第一传输通道和所述第二传输通道的数量根据所述编辑视频流中视频帧的数量动态分配。
通过上述实施方式，可以提高视频流的传输效率。
在第一方面的一种可能的实施方式中，所述方法还包括：
在将所述编辑视频流中的视频帧插入按序排列的传输视频流的过程中，将所述传输视频流中目标空帧之前的视频帧标记为处理完成状态，所述目标空帧为所述传输视频流中时间最早的空帧。
在第一方面的一种可能的实施方式中，所述编辑视频流中视频帧的帧头信息中包括所述视频帧的编辑时间和/或帧大小。
在第一方面的一种可能的实施方式中，所述根据所述编辑视频流和所述原始视频流生成目标视频流，包括：
若所述视频流中的视频帧均为待编辑视频帧，则将所述原始视频流中的视频帧替换为所述编辑视频流中对应的视频帧，生成目标视频流。
第二方面，本申请实施例提供一种视频处理装置，包括：
检测模块，用于在视频录制过程中，检测用户是否启动视频编辑功能；
显示模块，用于在所述检测模块检测到用户启动视频编辑功能的情况下，将将录制的原始视频流以视频帧形式进行呈现；
确定模块，用于根据用户对所述视频帧进行的视频帧选择操作，确定待编辑视频帧；
第一生成模块，用于根据用户输入的视频帧编辑操作对所述待编辑视频帧进行视频编辑处理，生成编辑视频流；
第二生成模块，用于根据所述编辑视频流和所述原始视频流生成目标视频流。
在第二方面的一种可能的实施方式中，所述确定模块还用于：
在确定待编辑视频帧之后，根据所述待编辑视频帧的编辑效果应用范围确定所述待编辑视频帧的关联帧，所述关联帧为所述编辑效果应用范围指示的与所述待编辑视频帧采用相同编辑效果的视频帧，所述编辑效果应用范围是预先设置的或者用户在选择完待编辑视频帧后确定的；
将所述关联帧确定为待编辑视频帧。
在第二方面的一种可能的实施方式中，所述编辑效果应用范围是预先设置的；所述装置还包括：
设置模块，用于在所述确定模块确定待编辑视频帧之前，根据用户的编辑效果应用范围设置操作预先设置各编辑效果对应的编辑效果应用范围。
在第二方面的一种可能的实施方式中，所述编辑效果应用范围包括下列中的至少一种：编辑效果应用于对应的待编辑视频帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的部分帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的所有帧。
在第二方面的一种可能的实施方式中，所述第二生成模块具体用于：
将所述原始视频流中的待编辑视频帧替换为空帧，生成传输视频流，所述空帧中包括对应的待编辑视频帧的帧头信息；
根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
在第二方面的一种可能的实施方式中，所述第二生成模块还用于：
在生成目标视频流之前，将所述传输视频流分配至第一编码通道进行编码处理；
将所述编辑视频流分配至第二编码通道进行编码处理。
在第二方面的一种可能的实施方式中，所述第一编码通道和所述第二编码通道的数量根据所述编辑视频流中视频帧的数量动态分配。
在第二方面的一种可能的实施方式中，所述第二生成模块具体用于：
根据帧头信息对所述传输视频流中的视频帧进行排序处理，使所述传输视频流中的视频帧按序排列；
对于所述传输视频流中的每个空帧，在所述编辑视频流中查找到所述空帧对应的目标视频帧后，将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置。
在第二方面的一种可能的实施方式中，所述第二生成模块还用于：
在所述将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置之前，根据帧头信息对查找到的目标视频帧进行校验；
确定所述查找到的目标视频帧校验成功。
在第二方面的一种可能的实施方式中，所述装置还包括：
编号模块，用于在所述确定模块确定待编辑视频帧之前，对所述原始视频流中的视频帧进行编号，并将编号加入对应的视频帧的帧头信息中；
所述第二生成模块具体用于：根据视频帧的编号将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
在第二方面的一种可能的实施方式中，所述第二生成模块具体用于：
根据视频帧的编号和帧位置将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
在第二方面的一种可能的实施方式中，所述第二生成模块具体用于：
将所述传输视频流和所述编辑视频流通过不同的传输通道传输给目标设备，以指示所述目标设备根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
在第二方面的一种可能的实施方式中，所述传输通道包括用于传输所述传输视频流的第一传输通道和用于传输所述编辑视频流的第二传输通道，所述第一传输通道和所述第二传输通道的数量根据所述编辑视频流中视频帧的数量动态分配。
在第二方面的一种可能的实施方式中，所述第二生成模块还用于：
在将所述编辑视频流中的视频帧插入按序排列的传输视频流的过程中，将所述传输视频流中目标空帧之前的视频帧标记为处理完成状态，所述目标空帧为所述传输视频流中时间最早的空帧。
在第二方面的一种可能的实施方式中，所述编辑视频流中视频帧的帧头信息中包括所述视频帧的编辑时间和/或帧大小。
在第二方面的一种可能的实施方式中，所述第二生成模块具体用于：
若所述视频流中的视频帧均为待编辑视频帧，则将所述原始视频流中的视频帧替换为所述编辑视频流中对应的视频帧，生成目标视频流。
第三方面，本申请实施例提供一种终端设备，包括：存储器和处理器，存储器用于存储计算机程序；处理器用于在调用计算机程序时执行上述第一方面或第一方面的任一实施方式所述的方法。
第四方面，本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述第一方面或第一方面的任一实施方式所述的方法。
第五方面，本申请实施例提供一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备执行上述第一方面或第一方面的任一实施方式所述的方法。
可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的视频处理方法的流程示意图；
图2为本申请实施例提供的视频编辑界面的界面示意图；
图3为本申请实施例提供的生成目标视频流的方法流程示意图；
图4为本申请实施例提供的一种编码通道分配示意图；
图5为本申请实施例提供的另一种编码通道分配示意图；
图6为本申请实施例提供的生成目标视频流的过程示意图；
图7为本申请实施例提供的视频处理装置的结构示意图；
图8为本申请实施例提供的终端设备的结构示意图。
具体实施方式
目前的直播视频编辑方式，只能对所有的视频帧进行编辑，编辑方式不够灵活，针对此技术问题，本申请实施例提供一种视频处理方法、装置与终端设备，主要通过在视频录制过程中，在检测到用户启动视频编辑功能的情况下，将录制的原始视频流以视频帧形式进行呈现；然后根据用户对原始视频帧进行的视频帧选择操作确定待编辑视频帧；再根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流；最后根据编辑视频流和原始视频流生成目标视频流，使用户可以在视频录制过程中选择需要编辑的视频帧进行编辑，以提高视频编辑方式的灵活性。
本申请实施例提供的技术方案可以应用于直播场景，也可以应用于其他实时传输录制的视频流的场景，为了便于理解，本申请实施例中以直播场景为例进行示例性说明。
本申请实施例中，终端设备可以是智能手机、计算机、平板电脑、智能穿戴设备或录像机等具有显示和处理视频功能的电子设备。
下面以具体地实施例对本申请的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。
图1为本申请实施例提供的视频处理方法的流程示意图，如图1所示，该本实施例提供的方法可以包括如下步骤：
S110、在视频录制过程中，若检测到用户启动视频编辑功能，则将录制的原始视频流以视频帧形式进行呈现。
具体的，终端设备上可以提供视频编辑功能启动标识，例如可以在视频录制界面上提供视频编辑功能启动图标，用户可以通过选择该标识来启动视频编辑功能；或者终端设备也可以预定义启动视频编辑功能的手势，例如双击操作，用户可以通过输入该预定义的手势启动视频编辑功能；终端设备还可以预定义启动视频编辑功能的语音指令，用户可以通过输入该语音指令启动视频编辑功能。
为了便于用户查看录制和编辑效果，终端设备可以提供预览功能，用户可以开启预览功能，在预览界面上启动视频编辑功能，例如：用户可以在预览界面上通过单击操作启动视频编辑功能。
终端设备检测到用户启动视频编辑功能后，可以显示视频编辑界面供用户进行视频编辑的相关操作。图2为本申请实施例提供的视频编辑界面的界面示意图，如图2所示，终端设备可以在视频编辑界面中将原始视频流以视频帧形式进行呈现，即终端设备可以将原始视频流拆分为一个个的视频帧，将拆分后得到的视频帧进行显示，以便用户选择待编辑视频帧。
在具体实现时，可以在视频帧显示区中按序显示视频帧的图标(例如图2中所示的视频帧缩略图)，其中，视频帧显示区可以如图2中所示的为视频编辑界面中的部分显示区，此时，视频编辑界面中还可以包括用于编辑显示选择的视频帧的编辑区和/或用于提供功能图标的功能区；视频帧显示区也可以为视频编辑界面中的全部显示区，此时，终端设备可以在用户选择完视频帧后显示编辑区和/或功能区。
S120、根据用户对视频帧进行的视频帧选择操作，确定待编辑视频帧。
具体的，用户可以通过在视频帧显示区上进行滑动操作来浏览更多的视频帧，在显示的视频帧上进行视频帧选择操作，确定待编辑视频帧。其中，视频帧选择操作可以是视频帧点击操作或其他预定义的视频帧选择手势。
为了进一步提高视频编辑的灵活性，本实施例中，用户可以设置编辑效果应用范围，即某待编辑视频帧的编辑效果可应用的视频帧；终端设备则在确定了待编辑视频帧后，再根据待编辑视频帧的编辑效果应用范围确定待编辑视频帧的关联帧，将关联帧也确定为待编辑视频帧。其中，关联帧为编辑效果应用范围指示的与待编辑视频帧采用相同编辑效果的视频帧。
其中，编辑效果应用范围可以包括下列中的至少一种：编辑效果应用于对应的待编辑视频帧；编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的部分帧；编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的所有帧。
本实施例中，用户可以在选择完待编辑视频帧后确定编辑效果应用范围，具体可以在某待编辑视频帧编辑前或编辑后进行确定，例如：用户在某待编辑视频帧上添加一个广告信息，然后选择每30s显示一次，即将编辑效果应用于与该待编辑视频帧之间的时间间隔是目标时间间隔(30s)整数倍的视频帧(即关联帧)，终端设备则将这些关联帧也确定为待编辑视频帧，对这些关联帧自动添加上述广告信息。
用户也可以在确定待编辑视频帧之前，在应用的设置功能中设置编辑效果应用范围，终端设备根据用户的编辑效果应用范围设置操作预先设置好编辑效果应用范围，将该编辑效果应用范围应用到用户的所有视频帧编辑操作中。例如：用户预先设置的编辑效果应用范围为编辑效果应用于用户在一次视频帧选择过程中选择的所有待编辑视频帧，即编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的部分帧，其中，该部分帧为用户选择的待编辑视频帧；那么在用户每次编辑视频时，终端设备都是将用户选择的第一个待编辑视频帧的编辑效果应用于用户本次选择的所有待编辑视频帧，此时，关联帧即为本次选择的除第一个待编辑视频帧之外的待编辑视频帧，用户每次只需编辑选择的第一个待编辑视频帧即可。
为了进一步提高视频编辑的灵活性，本实施例中，用户可以针对不同的编辑效果设置不同的编辑效果应用范围，即终端设备可以根据用户的编辑效果应用范围设置操作预先设置各编辑效果对应的编辑效果应用范围。
例如：对于添加广告功能，用户可以设置每30显示一次添加的广告，即将编辑效果应用于与该待编辑视频帧之间的时间间隔是目标时间间隔(30s)整数倍的视频帧(即关联帧)；对于擦除功能，用户可以设置仅应用于当前编辑的待编辑视频帧，即将编辑效果应用于编辑效果应用于对应的待编辑视频帧；对于背景替换功能，用户可以设置对所有的视频帧进行背景替换，即将编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的所有帧。那么，用户在视频编辑过程中，若对某待编辑视频帧添加广告，则终端设备将与该待编辑视频帧之间的时间间隔是为30s整数倍的视频帧确定为该待编辑视频帧的关联帧；若用户对某待编辑视频帧进行擦除操作，则终端设备确定的该待编辑视频帧的关联帧为0；若用户对某待编辑视频帧进行背景替换，则终端设备将该待编辑视频帧之后的所有视频帧确定为该待编辑视频帧的关联帧。
S130、根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流。
具体的，用户在选择完待编辑视频帧后，就可以输入视频帧编辑操作对待编辑视频帧进行视频编辑。
具体实现时，如图2所示，终端设备可以在视频编辑界面上显示编辑区和功能区，功能区中可以提供各种预定义的编辑功能的标识(例如：文字标识或图2中所示的功能图标)，用户可以在功能区中选择功能图标对编辑区显示的待编辑视频帧进行编辑。其中，终端设备可以在编辑区中自动显示用户选择的第一个待编辑视频帧，用户也可以通过输入相关操作(例如拖动操作)将第一个待编辑视频帧显示在编辑区。
其中，预定义的编辑功能可以是一些较为简单的编辑功能，例如：擦除功能，用户可以通过该功能对待编辑视频帧中某一区域或对象进行擦除或删除；广告添加功能，用户可以通过该功能在待编辑视频帧中某些位置添加指定的广告；图片添加功能，用户可以通过该功能在待编辑视频帧中某些位置添加预设的图片或动图；背景替换功能，用户可以通过该功能对待编辑视频帧的背景进行替换；美化功能，用户可以通过该功能对待编辑视频帧中人物的衣服颜色和/或头发样式进行更换。终端设备中预定义的编辑功能可以包括上述列举的至少一种编辑功能，还可以包括其他编辑功能，本实施例对此不做特别限定。
用户的视频帧编辑操作包括用户在编辑视频帧的过程中进行的一系列操作，例如：选用功能区的编辑功能对待编辑视频帧进行的编辑操作和前述的在选择完待编辑视频帧后进行的确定编辑效果应用范围的操作；终端设备基于用户的视频帧编辑操作对待编辑视频帧进行对应的视频编辑处理，使编辑后得到的视频帧在播放时可以呈现对应的编辑效果；然后可以根据先后编辑得到的视频帧(后续称为已编辑视频帧)生成编辑视频流，其中，编辑视频流中的已编辑视频帧的顺序可以按照视频帧的编辑完成时间排序。
本实施例中，编辑视频流中视频帧的帧头信息中包含对应的待编辑视频帧的帧头信息，除此之外，还可以包括：视频帧的编辑时间和/或帧大小等编辑信息，以方便后端处理模块识别经过编辑的视频帧。其中，编辑时间可以是编辑开始时间，也可以是编辑完成时间；编辑视频帧中也可以包括编辑效果等其他编辑信息，具体可以根据需要选择，本实施例对此不做特别限定。
需要说明的是，本申请实施例所述的用户输入的各种操作既可以是手动操作，也可以语音指令输入操作，为了便于说明，本申请实施例中以手动操作为例进行示例性说明。
S140、根据编辑视频流和原始视频流生成目标视频流。
本实施例中，终端设备在原始视频流之外生成编辑视频流，则原始视频流可以不用停止，继续进行后续处理；在生成目标视频流时，将编辑视频流中的视频帧插入原始视频流中对应位置即可，这样可以保证视频流传输的实时性。
在生成目标视频流的过程中，原始视频流和编辑视频流需在终端设备内部的功能模块之间进行传输和处理，为了提升终端设备的处理效率，本实施例中，可以将原始视频流中的待编辑视频帧的视频数据去除，以减少数据传输和处理量；另外，在生成目标视频流时，需要将编辑视频流中的视频帧插入原始视频流中对应位置，为了在减少数据传输和处理量的同时，提高视频流生成效率，本实施例中，可以将原始视频流中的待编辑视频帧替换为空帧进行占位，以在后期合成目标视频流时，通过空帧位置快速将对应的已编辑视频帧插入传输视频流，该方法的实际内部实现过程可以参见图3。
图3为本申请实施例提供的生成目标视频流的方法流程示意图，如图3所示，步骤S140中，实际的内部实现可以包括如下步骤：
S141、将原始视频流中的待编辑视频帧替换为空帧，生成传输视频流。
具体的，为了减小传输视频流的数据量，空帧中可以只包括对应的待编辑视频帧的帧头信息，其中，帧头信息中包含待编辑视频帧的一些基本信息，例如：帧大小和帧位置等，其中，帧位置即为待编辑视频帧在原始视频流中的位置，其具体可以是时间戳。
S142、将传输视频流分配至第一编码通道进行编码处理，将编辑视频流分配至第二编码通道进行编码处理。
本实施例中，在生成传输视频流和编辑视频流之后就可以将两者进行合并以生成目标视频流。为了提高视频流在终端设备中的传输效率，本实施例中，在生成目标视频流之前，可以先对传输视频流和编辑视频流进行编码压缩。
在具体实现时，终端设备中可以预定义多个编码通道，在进行编码时，可以将其中一部分编码通道(即第一编码通道)用于编码压缩传输视频流，另一部分编码通道(即第二编码通道)用于编码压缩编辑视频流，具体分配时，第一编码通道和第二编码通道的数量可以根据编辑视频流中视频帧的数量动态分配，以提高视频流的传输和编码效率。其中，编码通道可以基于终端内部视频流传输通道实现，即在内部视频流传输通道中对视频流进行编码压缩形成编码通道。下面以终端设备中预定义3个编码通道为例示例性说明编码通道的分配方法。
图4为本申请实施例提供的一种编码通道分配示意图，如图4所示，在编辑时，对原始视频流中的帧2、帧3和帧4进行了编辑，编辑视频帧较少，此种情况下，可以将两个编码通道(即第一编码通道)用于传输和编码传输视频流中的视频帧，如图4所示，其中，帧1、帧3(空帧)和帧5被分配到其中一个第一编码通道，帧2(空帧)、帧4(空帧)和帧6被分配到另一个第一编码通道；将第三个编码通道(即第二编码通道)用于传输和编码编辑视频流中的视频帧：帧2(编辑后)、帧3(编辑后)和帧4(编辑后)。
图5为本申请实施例提供的另一种编码通道分配示意图，如图5所示，在编辑时，用户将编辑结果应用于后续所有帧，此种情况下，可以将一个编码通道(即第一编码通道)用于传输和编码传输视频流中的视频帧，此时，传输视频流中的视频帧均为空帧，如图5中所示的帧1(空帧)、帧2(空帧)和帧3(空帧)；将另外两个编码通道(即第二编码 通道)用于传输和编码编辑视频流中的视频帧，如图5所示，其中，帧1(编辑后)、帧3(编辑后)和帧5(编辑后)被分配到其中一个第二编码通道，帧2(编辑后)、帧4(编辑后)和帧6(编辑后)被分配到另一个第二编码通道。由于编码通道占用的内存有限，图5中只是示出了3个空帧，帧4、帧5和帧6对应的空帧可以待第一编码通道处理完帧1、帧2和帧3对应的空帧后再进行处理。
需要说明的是，上述编码通道中处理的帧数量只是一种示例，其具体可处理的帧数量可以根据其分配的内存确定，本实施例对此不做特别限定。另外，传输视频流或编辑视频流被分配到多个编码通道时，具体的分配规则也可以根据需要确定，例如上述图4中，也可以是帧1、帧2(空帧)和帧3(空帧)被分配到其中一个第一编码通道，帧4(空帧)、帧5和帧6被分配到另一个第一编码通道，具体的分配规则本实施例不做特别限定。
S143、根据帧头信息将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
具体的，传输视频流和编辑视频流经过编码压缩等处理后，可以将编辑视频流合并入传输视频流中生成目标视频流。
在具体实现时，可以先根据帧头信息对传输视频流中的视频帧进行排序处理，使传输视频流中的视频帧按序排列；然后对于传输视频流中的每个空帧，在编辑视频流中查找到空帧对应的目标视频帧后，将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置。
其中，在进行排序时，可以根据帧头信息中的帧位置确定经过编码压缩处理后的传输视频流中各视频帧的顺序，按照先后顺序对各视频帧进行排序。
在排序完成后，可以按照视频帧先后顺序依次处理传输视频流中的空帧，对于每个空帧，具体也可以根据帧头信息中的帧位置，在编辑视频流中查找帧位置相同的视频帧(即目标视频帧)，在查找到后将其插入对应的空帧位置。
为了提高目标视频流的数据准确性，本实施例中，在查找到某个空帧对应的目标视频帧后，可以先根据帧头信息对查找到的目标视频帧进行校验，在确定查找到的目标视频帧校验成功的情况下，再将其插入传输视频流中对应的空帧位置。
帧头信息中包含的视频帧的信息较多，在具体校验时，可以比对空帧和对应的目标视频帧的帧头信息，确定两者的相似度，在满足目标相似度时确定校验成功。
为了提高排序和视频流合并速度，本实施例中，在确定编辑视频帧之前，可以先对原始视频流中的视频帧进行编号，并将编号加入对应的视频帧的帧头信息中。则终端设备就可以根据视频帧的编号对传输视频流中的视频帧进行排序处理，以及根据视频帧的编号将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
具体的，终端设备在编辑视频流时，一般是将视频流依次放入视频流缓冲区(buffer)中进行处理，则在编号时，可以针对视频流buffer中的原始视频流进行编号。视频流buffer中待编辑视频帧被替换为空帧后，则被传入编码通道进行编码处理，传输视频流经过编码处理后，可以放入传输buffer中，此时可以根据视频帧的编号对传输buffer中的视频帧进行排序。另外，视频流buffer中的待编辑视频帧被提取出来进行视频编辑处理和编码处理后，可以放入编辑buffer；传输buffer中的视频帧排序完成后，可以将其中的空帧加入编辑队列；在编辑buffer中传入视频帧后，可以依次处理编辑队列中的空帧，根据视频帧的 编号在编辑buffer中为空帧查找对应的目标视频帧，然后插入传输buffer中对应的空帧位置。其中，在将某个空帧的目标视频帧插入完成后，可以将编辑队列中对应的空帧删除，然后继续处理编辑队列中的下一个空帧。
图6为本申请实施例提供的生成目标视频流的过程示意图，如图6所示，传输buffer中的传输视频流包括：帧1、帧2(空帧)、帧3(空帧)、帧4(空帧)、帧5和帧6，编辑buffer中的编辑视频流包括：帧2(编辑后)、帧3(编辑后)和帧4(编辑后)；在进行合并时，将帧2(编辑后)插入对应的空帧位置：帧2(空帧)，帧3(编辑后)插入对应的空帧位置：帧3(空帧)，帧4(编辑后)插入对应的空帧位置：帧4(空帧)，生成目标视频流，包括：帧1、帧2(编辑后)、帧3(编辑后)、帧4(编辑后)、帧5和帧6。
为了在提高处理速度的同时提高数据准确性，本实施例中，可以同时结合视频帧的编号和帧位置进行排序和视频流合并，即根据视频帧的编号和帧位置对传输视频流中的视频帧进行排序处理，以及根据视频帧的编号和帧位置将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
具体的，在进行传输视频流的排序时，可以先根据视频帧的编号对传输视频流中的视频帧进行排序处理，然后再根据视频帧的位置检查传输视频流中视频帧的顺序是否正确，若不正确再进行调整；同样的，在进行目标视频帧的查找时，可以先根据视频帧的编号在编辑视频流中查找空帧对应的目标视频帧，然后再根据视频帧的位置检查查找到的目标视频帧是否正确，在检查通过时，再将目标视频帧插入传输视频流中对应的空帧位置。
本实施例中，在将编辑视频流中的视频帧插入按序排列的传输视频流的过程中，可以将传输视频流中目标空帧之前的视频帧标记为处理完成状态，其中，目标空帧为传输视频流中时间最早的空帧。视频帧被标记为处理完成状态，说明该视频帧可以进行后续的处理，例如传输、存储或显示等。
在具体实现时，可以根据上述的编辑队列来确定目标空帧，该目标空帧即为编辑队列中位于队首位置的空帧。
本实施例中，在生成传输视频流和编辑视频流后，终端设备也可以将传输视频流和编辑视频流通过不同的传输通道传输给目标设备，指示目标设备根据帧头信息将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
其中，目标设备可以是其他的终端设备，目标设备生成目标视频流的过程与上述终端设备生成目标视频流的过程类似，此处不再赘述。传输通道包括用于传输传输视频流的第一传输通道和用于传输编辑视频流的第二传输通道，同样的，为了提高传输效率，第一传输通道和第二传输通道的数量可以根据编辑视频流中视频帧的数量动态分配，其分配原则与编码通道的分配原则类似，此处不再赘述。
作为一种可选的实施方式，本实施例中，对于编辑效果应用于后续所有视频帧的情况，若实时性要求不高，则可以省略空帧，直接将原始视频流中的视频帧替换为编辑后的视频帧，生成目标视频流，即可以在视频流中的视频帧均为待编辑视频帧的情况下，将原始视频流中的视频帧替换为编辑视频流中对应的视频帧，生成目标视频流。
在具体实现时，可以在编辑完待编辑视频帧后，将已编辑视频帧直接插入原始视频流对应的视频流buffer中，实现原始视频流的视频帧替换；然后可以对替换后的视频流进行 处理，该处理过程与目前的视频流处理过程类似，即将全部的编码通道用来处理已编辑视频帧，在编码处理后，将经过编码处理的已编辑视频帧放入传输buffer，然后进行排序，生成目标视频流。
本实施例提供的视频处理方法，通过在视频录制过程中，在检测到用户启动视频编辑功能的情况下，将录制的原始视频流以视频帧形式进行呈现；然后根据用户对原始视频帧进行的视频帧选择操作确定待编辑视频帧；再根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流；最后根据编辑视频流和原始视频流生成目标视频流，可以使用户在视频录制过程中选择需要编辑的视频帧进行编辑，从而可以提高视频编辑方式的灵活性，同时还可以保证视频流传输的实时性。
基于同一发明构思，作为对上述方法的实现，本申请实施例提供了一种视频处理装置，该装置实施例与前述方法实施例对应，为便于阅读，本装置实施例不再对前述方法实施例中的细节内容进行逐一赘述，但应当明确，本实施例中的装置能够对应实现前述方法实施例中的全部内容。
图7为本申请实施例提供的视频处理装置的结构示意图，如图7所示，本实施例提供的装置包括：
检测模块110，用于在视频录制过程中，检测用户是否启动视频编辑功能；
显示模块120，用于在检测模块110检测到用户启动视频编辑功能的情况下，将将录制的原始视频流以视频帧形式进行呈现；
确定模块130，用于根据用户对视频帧进行的视频帧选择操作，确定待编辑视频帧；
第一生成模块140，用于根据用户输入的视频帧编辑操作对待编辑视频帧进行视频编辑处理，生成编辑视频流；
第二生成模块150，用于根据编辑视频流和原始视频流生成目标视频流。
作为本申请实施例一种可选的实施方式，确定模块130还用于：
在确定待编辑视频帧之后，根据待编辑视频帧的编辑效果应用范围确定待编辑视频帧的关联帧，关联帧为编辑效果应用范围指示的与待编辑视频帧采用相同编辑效果的视频帧，编辑效果应用范围是预先设置的或者用户在选择完待编辑视频帧后确定的；
将关联帧确定为待编辑视频帧。
作为本申请实施例一种可选的实施方式，编辑效果应用范围是预先设置的；该装置还包括：
设置模块160，用于在确定模块130确定待编辑视频帧之前，根据用户的编辑效果应用范围设置操作预先设置各编辑效果对应的编辑效果应用范围。
作为本申请实施例一种可选的实施方式，编辑效果应用范围包括下列中的至少一种：编辑效果应用于对应的待编辑视频帧；编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的部分帧；编辑效果应用于对应的待编辑视频帧和对应的待编辑视频帧之后的所有帧。
作为本申请实施例一种可选的实施方式，第二生成模块150具体用于：
将原始视频流中的待编辑视频帧替换为空帧，生成传输视频流，空帧中包括对应的待编辑视频帧的帧头信息；
根据帧头信息将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
作为本申请实施例一种可选的实施方式，第二生成模块150还用于：
在生成目标视频流之前，将传输视频流分配至第一编码通道进行编码处理；
将编辑视频流分配至第二编码通道进行编码处理。
作为本申请实施例一种可选的实施方式，第一编码通道和第二编码通道的数量根据编辑视频流中视频帧的数量动态分配。
作为本申请实施例一种可选的实施方式，第二生成模块150具体用于：
根据帧头信息对传输视频流中的视频帧进行排序处理，使传输视频流中的视频帧按序排列；
对于传输视频流中的每个空帧，在编辑视频流中查找到空帧对应的目标视频帧后，将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置。
作为本申请实施例一种可选的实施方式，第二生成模块150还用于：
在将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置之前，根据帧头信息对查找到的目标视频帧进行校验；
确定查找到的目标视频帧校验成功。
作为本申请实施例一种可选的实施方式，该装置还包括：
编号模块170，用于在确定模块130确定待编辑视频帧之前，对原始视频流中的视频帧进行编号，并将编号加入对应的视频帧的帧头信息中；
第二生成模块150具体用于：根据视频帧的编号将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
作为本申请实施例一种可选的实施方式，第二生成模块150具体用于：
根据视频帧的编号和帧位置将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
作为本申请实施例一种可选的实施方式，第二生成模块150具体用于：
将传输视频流和编辑视频流通过不同的传输通道传输给目标设备，以指示目标设备根据帧头信息将编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
作为本申请实施例一种可选的实施方式，传输通道包括用于传输传输视频流的第一传输通道和用于传输编辑视频流的第二传输通道，第一传输通道和第二传输通道的数量根据编辑视频流中视频帧的数量动态分配。
作为本申请实施例一种可选的实施方式，第二生成模块150还用于：
在将编辑视频流中的视频帧插入按序排列的传输视频流的过程中，将传输视频流中目标空帧之前的视频帧标记为处理完成状态，目标空帧为传输视频流中时间最早的空帧。
作为本申请实施例一种可选的实施方式，编辑视频流中视频帧的帧头信息中包括视频帧的编辑时间和/或帧大小。
作为本申请实施例一种可选的实施方式，第二生成模块150具体用于：
若视频流中的视频帧均为待编辑视频帧，则将原始视频流中的视频帧替换为编辑视频流中对应的视频帧，生成目标视频流。
本实施例提供的装置可以执行上述方法实施例，其实现原理与技术效果类似，此处不再赘述。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
基于同一发明构思，本申请实施例还提供了一种终端设备。图8为本申请实施例提供的终端设备的结构示意图，如图8所示，本实施例提供的终端设备可以包括：存储器210、处理器220、输入单元230和显示单元240等部件
下面结合图8对终端设备的各个构成部件进行具体的介绍：
存储器210可用于存储计算机程序以及模块，处理器220通过运行存储在存储器210的计算机程序以及模块可以执行上述方法实施例所述的方法。存储器210可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据终端设备的使用所创建的数据(比如音频数据、电话本等)等。此外，存储器210可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
输入单元230可用于接收输入的数字或字符信息，以及产生与终端设备的用户设置以及功能控制有关的键信号输入。具体地，输入单元230可包括触控面板231以及其他输入设备232。触控面板231，也称为触摸屏，可收集用户在其上或附近的触摸操作(比如用户使用手指、触笔等任何适合的物体或附件在触控面板231上或在触控面板231附近的操作)，并根据预先设定的程式驱动相应的连接装置。可选的，触控面板231可包括触摸检测装置和触摸控制器两个部分。其中，触摸检测装置检测用户的触摸方位，并检测触摸操作带来的信号，将信号传送给触摸控制器；触摸控制器从触摸检测装置上接收触摸信息，并将它转换成触点坐标，再送给处理器220，并能接收处理器220发来的命令并加以执行。此外，可以采用电阻式、电容式、红外线以及表面声波等多种类型实现触控面板231。除了触控面板231，输入单元230还可以包括其他输入设备232。具体地，其他输入设备232可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆等中的一种或多种。
显示单元240可用于显示由用户输入的信息或提供给用户的信息以及终端设备的各种菜单。显示单元240可包括显示面板241，可选的，可以采用液晶显示器(Liquid Crystal Display，LCD)、有机发光二极管(Organic Light-Emitting Diode,OLED)等形式来配置显示面板241。进一步的，触控面板231可覆盖显示面板241，当触控面板231检测到在其 上或附近的触摸操作后，传送给处理器220以确定触摸事件的类型，随后处理器220根据触摸事件的类型在显示面板241上提供相应的视觉输出。虽然在图8中，触控面板231与显示面板241是作为两个独立的部件来实现终端设备的输入和输入功能，但是在某些实施例中，可以将触控面板231与显示面板241集成而实现终端设备的输入和输出功能。
处理器220是终端设备的控制中心，利用各种接口和线路连接整个终端设备的各个部分，通过运行或执行存储在存储器210内的软件程序和/或模块，以及调用存储在存储器210内的数据，执行终端设备的各种功能和处理数据，从而对终端设备进行整体监控。可选的，处理器220可包括一个或多个处理单元。
终端设备还包括给各个部件供电的电源(比如电池)(未示出)，优选的，电源可以通过电源管理系统与处理器220逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。
终端设备还可以包括摄像头(未示出)。可选地，摄像头在终端设备的上的位置可以为前置的，也可以为后置的，本申请实施例对此不作限定。
可选地，终端设备可以包括单摄像头、双摄像头或三摄像头等，本申请实施例对此不作限定。例如，终端设备可以包括三摄像头，其中，一个为主摄像头、一个为广角摄像头、一个为长焦摄像头。
可选地，当终端设备包括多个摄像头时，这多个摄像头可以全部前置，或者全部后置，或者一部分前置、另一部分后置，本申请实施例对此不作限定。
本实施例提供的终端设备可以执行上述方法实施例，其实现原理与技术效果类似，此处不再赘述。
本领域技术人员可以理解，图8仅仅是终端设备的举例，并不构成对终端设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
本申请实施例还提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述方法实施例所述的方法。
本申请实施例还提供一种计算机程序产品，当计算机程序产品在终端设备上运行时，使得终端设备执行时实现上述方法实施例所述的方法。
上述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读存储介质至少可以包括：能够将计算机程序代码携带到拍照装置/终端设备的任何实体或装置、记录介质、计算机存储器、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、电载波信号、电信信号以及软件分发介质。例如U盘、移动硬盘、磁碟或者光盘等。在某些司法管辖区，根据立法和专利实践，计算机可读介质不可以是电载波信号和电信信号。
处理器可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific  Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。存储器是计算机可读介质的示例。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/设备和方法，可以通过其它的方式实现。例如，以上所描述的装置/设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
应当理解，当在本申请说明书和所附权利要求书中使用时，术语“包括”指示所描述特征、整体、步骤、操作、元素和/或组件的存在，但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。
还应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。
如在本申请说明书和所附权利要求书中所使用的那样，术语“如果”可以依据上下文被解释为“当...时”或“一旦”或“响应于确定”或“响应于检测到”。类似地，短语“如果确定”或“如果检测到[所描述条件或事件]”可以依据上下文被解释为意指“一旦确定”或“响应于确定”或“一旦检测到[所描述条件或事件]”或“响应于检测到[所描述条件或事件]”。
另外，在本申请说明书和所附权利要求书的描述中，术语“第一”、“第二”、“第三”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
在本申请说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解： 其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (19)

  1. 一种视频处理方法，其特征在于，包括：
    在视频录制过程中，若检测到用户启动视频编辑功能，则将录制的原始视频流以视频帧形式进行呈现；
    根据用户对所述视频帧进行的视频帧选择操作，确定待编辑视频帧；
    根据用户输入的视频帧编辑操作对所述待编辑视频帧进行视频编辑处理，生成编辑视频流；
    根据所述编辑视频流和所述原始视频流生成目标视频流。
  2. 根据权利要求1所述的方法，其特征在于，在所述确定待编辑视频帧之后，所述方法还包括：
    根据所述待编辑视频帧的编辑效果应用范围确定所述待编辑视频帧的关联帧，所述关联帧为所述编辑效果应用范围指示的与所述待编辑视频帧采用相同编辑效果的视频帧，所述编辑效果应用范围是预先设置的或者用户在选择完待编辑视频帧后确定的；
    将所述关联帧确定为待编辑视频帧。
  3. 根据权利要求2所述的方法，其特征在于，所述编辑效果应用范围是预先设置的；在所述确定待编辑视频帧之前，所述方法还包括：
    根据用户的编辑效果应用范围设置操作预先设置各编辑效果对应的编辑效果应用范围。
  4. 根据权利要求2所述的方法，其特征在于，所述编辑效果应用范围包括下列中的至少一种：编辑效果应用于对应的待编辑视频帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的部分帧；编辑效果应用于对应的待编辑视频帧和所述对应的待编辑视频帧之后的所有帧。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述根据所述编辑视频流和所述原始视频流生成目标视频流，包括：
    将所述原始视频流中的待编辑视频帧替换为空帧，生成传输视频流，所述空帧中包括对应的待编辑视频帧的帧头信息；
    根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
  6. 根据权利要求5所述的方法，其特征在于，所述生成目标视频流之前，所述方法还包括：
    将所述传输视频流分配至第一编码通道进行编码处理；
    将所述编辑视频流分配至第二编码通道进行编码处理。
  7. 根据权利要求6所述的方法，其特征在于，所述第一编码通道和所述第二编码通道的数量根据所述编辑视频流中视频帧的数量动态分配。
  8. 根据权利要求5所述的方法，其特征在于，所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
    根据帧头信息对所述传输视频流中的视频帧进行排序处理，使所述传输视频流中的视频帧按序排列；
    对于所述传输视频流中的每个空帧，在所述编辑视频流中查找到所述空帧对应的目标视频帧后，将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置。
  9. 根据权利要求8所述的方法，其特征在于，在所述将查找到的目标视频帧插入按序排列的传输视频流中对应的空帧位置之前，所述方法还包括：
    根据帧头信息对查找到的目标视频帧进行校验；
    确定所述查找到的目标视频帧校验成功。
  10. 根据权利要求5所述的方法，其特征在于，在所述确定待编辑视频帧之前，所述方法还包括：
    对所述原始视频流中的视频帧进行编号，并将编号加入对应的视频帧的帧头信息中；
    所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
    根据视频帧的编号将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
  11. 根据权利要求10所述的方法，其特征在于，所述根据视频帧的编号将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，包括：
    根据视频帧的编号和帧位置将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置。
  12. 根据权利要求5所述的方法，其特征在于，所述根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流，包括：
    将所述传输视频流和所述编辑视频流通过不同的传输通道传输给目标设备，以指示所述目标设备根据帧头信息将所述编辑视频流中的视频帧插入按序排列的传输视频流中对应的空帧位置，生成目标视频流。
  13. 根据权利要求12所述的方法，其特征在于，所述传输通道包括用于传输所述传输视频流的第一传输通道和用于传输所述编辑视频流的第二传输通道，所述第一传输通道和所述第二传输通道的数量根据所述编辑视频流中视频帧的数量动态分配。
  14. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    在将所述编辑视频流中的视频帧插入按序排列的传输视频流的过程中，将所述传输视频流中目标空帧之前的视频帧标记为处理完成状态，所述目标空帧为所述传输视频流中时间最早的空帧。
  15. 根据权利要求5所述的方法，其特征在于，所述编辑视频流中视频帧的帧头信息中包括所述视频帧的编辑时间和/或帧大小。
  16. 根据权利要求1-4任一项所述的方法，其特征在于，所述根据所述编辑视频流和所述原始视频流生成目标视频流，包括：
    若所述视频流中的视频帧均为待编辑视频帧，则将所述原始视频流中的视频帧替换为所述编辑视频流中对应的视频帧，生成目标视频流。
  17. 一种视频处理装置，其特征在于，包括：
    检测模块，用于在视频录制过程中，检测用户是否启动视频编辑功能；
    显示模块，用于在所述检测模块检测到用户启动视频编辑功能的情况下，将将录制的原始视频流以视频帧形式进行呈现；
    确定模块，用于根据用户对所述视频帧进行的视频帧选择操作，确定待编辑视频帧；
    第一生成模块，用于根据用户输入的视频帧编辑操作对所述待编辑视频帧进行视频编辑处理，生成编辑视频流；
    第二生成模块，用于根据所述编辑视频流和所述原始视频流生成目标视频流。
  18. 一种终端设备，其特征在于，包括：存储器和处理器，所述存储器用于存储计算机程序；所述处理器用于在调用所述计算机程序时执行如权利要求1-16任一项所述的方法。
  19. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-16任一项所述的方法。
PCT/CN2020/124494 2019-12-18 2020-10-28 视频处理方法、装置与终端设备 WO2021120872A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20903996.5A EP4057633A4 (en) 2019-12-18 2020-10-28 VIDEO PROCESSING METHOD AND EQUIPMENT AND TERMINAL DEVICE
US17/843,690 US20220328071A1 (en) 2019-12-18 2022-06-17 Video processing method and apparatus and terminal device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911309949.9A CN112995746B (zh) 2019-12-18 2019-12-18 视频处理方法、装置与终端设备
CN201911309949.9 2019-12-18

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/843,690 Continuation US20220328071A1 (en) 2019-12-18 2022-06-17 Video processing method and apparatus and terminal device

Publications (1)

Publication Number Publication Date
WO2021120872A1 true WO2021120872A1 (zh) 2021-06-24

Family

ID=76343941

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/124494 WO2021120872A1 (zh) 2019-12-18 2020-10-28 视频处理方法、装置与终端设备

Country Status (4)

Country Link
US (1) US20220328071A1 (zh)
EP (1) EP4057633A4 (zh)
CN (1) CN112995746B (zh)
WO (1) WO2021120872A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873295A (zh) * 2021-10-26 2021-12-31 北京金山云网络技术有限公司 多媒体信息处理方法、装置、设备及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113518187B (zh) * 2021-07-13 2024-01-09 北京达佳互联信息技术有限公司 视频编辑方法及设备
CN116132719A (zh) * 2021-11-15 2023-05-16 北京字跳网络技术有限公司 视频处理方法、装置、电子设备及可读存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2238743A1 (en) * 2007-12-17 2010-10-13 Stein Gausereide Real time video inclusion system
CN101909161A (zh) * 2009-12-17 2010-12-08 新奥特（北京）视频技术有限公司 一种视频剪辑方法及装置
US20120195573A1 (en) * 2011-01-28 2012-08-02 Apple Inc. Video Defect Replacement
CN102740162A (zh) * 2012-06-19 2012-10-17 深圳Tcl新技术有限公司 电视机及其视频编辑方法、装置
CN103916535A (zh) * 2013-01-04 2014-07-09 Lg电子株式会社 移动终端及其控制方法
CN106331869A (zh) * 2015-06-29 2017-01-11 北京金山安全软件有限公司 一种基于视频的图片重编辑方法及装置
CN108259788A (zh) * 2018-01-29 2018-07-06 努比亚技术有限公司 视频编辑方法、终端和计算机可读存储介质
CN108635851A (zh) * 2018-05-16 2018-10-12 网易（杭州）网络有限公司 一种游戏画面的处理方法和装置

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6993081B1 (en) * 1999-11-23 2006-01-31 International Business Machines Corporation Seamless splicing/spot-insertion for MPEG-2 digital video/audio stream
US7145912B1 (en) * 2000-03-23 2006-12-05 Tut Systems, Inc. Demultiplexing a statistically multiplexed MPEG transport stream into CBR single program transport streams
US8990214B2 (en) * 2001-06-27 2015-03-24 Verizon Patent And Licensing Inc. Method and system for providing distributed editing and storage of digital media over a network
US7739599B2 (en) * 2005-09-23 2010-06-15 Microsoft Corporation Automatic capturing and editing of a video
CN102243615B (zh) * 2011-03-21 2015-05-13 广东志成冠军集团有限公司 一种支持音视频帧存储的内存缓冲管理方法
WO2013020709A1 (en) * 2011-08-10 2013-02-14 Telefonaktiebolaget L M Ericsson (Publ) Media stream handling
US9357248B2 (en) * 2013-03-13 2016-05-31 Arris Enterprises, Inc. Method and apparatus for adaptive bit rate content delivery
US9578333B2 (en) * 2013-03-15 2017-02-21 Qualcomm Incorporated Method for decreasing the bit rate needed to transmit videos over a network by dropping video frames
CN105280205B (zh) * 2014-05-30 2018-03-16 深圳锐取信息技术股份有限公司 非线性编辑软件音视频同步处理方法及装置
CN104052935B (zh) * 2014-06-18 2017-10-20 广东欧珀移动通信有限公司 一种视频编辑方法及装置
CN104113777B (zh) * 2014-08-01 2018-06-05 广州猎豹网络科技有限公司 一种音频流解码方法及装置
KR20160091468A (ko) * 2015-01-23 2016-08-03 한국전자통신연구원 분산 파일 시스템 상의 비디오 구간 편집 방법 및 장치
KR20170052364A (ko) * 2015-11-04 2017-05-12 삼성전자주식회사 디스플레이장치 및 그 제어방법
CN106851385B (zh) * 2017-02-20 2019-12-27 北京乐我无限科技有限责任公司 视频录制方法、装置和电子设备
CN107454416B (zh) * 2017-09-12 2020-06-30 广州酷狗计算机科技有限公司 视频流发送方法和装置
CN107992246A (zh) * 2017-12-22 2018-05-04 珠海格力电器股份有限公司 一种视频编辑方法与其装置及智能终端
CN109195007B (zh) * 2018-10-19 2021-09-07 深圳市轱辘车联数据技术有限公司 视频生成方法、装置、服务器及计算机可读存储介质
CN110049266A (zh) * 2019-04-10 2019-07-23 北京字节跳动网络技术有限公司 视频数据发布方法、装置、电子设备和存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2238743A1 (en) * 2007-12-17 2010-10-13 Stein Gausereide Real time video inclusion system
CN101909161A (zh) * 2009-12-17 2010-12-08 新奥特（北京）视频技术有限公司 一种视频剪辑方法及装置
US20120195573A1 (en) * 2011-01-28 2012-08-02 Apple Inc. Video Defect Replacement
CN102740162A (zh) * 2012-06-19 2012-10-17 深圳Tcl新技术有限公司 电视机及其视频编辑方法、装置
CN103916535A (zh) * 2013-01-04 2014-07-09 Lg电子株式会社 移动终端及其控制方法
CN106331869A (zh) * 2015-06-29 2017-01-11 北京金山安全软件有限公司 一种基于视频的图片重编辑方法及装置
CN108259788A (zh) * 2018-01-29 2018-07-06 努比亚技术有限公司 视频编辑方法、终端和计算机可读存储介质
CN108635851A (zh) * 2018-05-16 2018-10-12 网易（杭州）网络有限公司 一种游戏画面的处理方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873295A (zh) * 2021-10-26 2021-12-31 北京金山云网络技术有限公司 多媒体信息处理方法、装置、设备及存储介质
CN113873295B (zh) * 2021-10-26 2024-05-28 北京金山云网络技术有限公司 多媒体信息处理方法、装置、设备及存储介质

Also Published As

Publication number Publication date
EP4057633A1 (en) 2022-09-14
US20220328071A1 (en) 2022-10-13
CN112995746A (zh) 2021-06-18
EP4057633A4 (en) 2023-01-11
CN112995746B (zh) 2022-09-09

Similar Documents

Publication Publication Date Title
WO2021120872A1 (zh) 视频处理方法、装置与终端设备
CN107426403B (zh) 移动终端
CN111050203B (zh) 一种视频处理方法、装置、视频处理设备及存储介质
JP4752897B2 (ja) 画像処理装置、画像表示方法および画像表示プログラム
JP5552767B2 (ja) 表示処理装置、表示処理方法および表示処理プログラム
US20070223878A1 (en) Image displaying method and video playback apparatus
CN111010610B (zh) 一种视频截图方法及电子设备
JP2017062797A (ja) メディア編集アプリケーション用の扇形編出ユーザインタフェースコントロール
US20220417417A1 (en) Content Operation Method and Device, Terminal, and Storage Medium
US20130279811A1 (en) Method and system for automatically selecting representative thumbnail of photo folder
WO2021093623A1 (zh) 一种拼图方法、装置及终端设备
US8717286B2 (en) Information processing apparatus, processing method thereof, and computer-readable storage medium
US20020085022A1 (en) Presentation system, method for generating presentation data, and program for generating presentation data
CN104333688A (zh) 基于拍摄的图像生成表情符的设备和方法
US11770603B2 (en) Image display method having visual effect of increasing size of target image, mobile terminal, and computer-readable storage medium
CN108256071A (zh) 录屏文件的生成方法、装置、终端及存储介质
WO2022068721A1 (zh) 截屏方法、装置及电子设备
US20170046350A1 (en) Media organization
WO2020057241A1 (zh) 应用程序显示的方法、装置及终端设备
WO2020103002A1 (zh) 一种文件收藏方法、文件收藏装置及终端设备
WO2024041468A1 (zh) 文件的处理方法、装置、电子设备和可读存储介质
WO2024041514A1 (zh) 视频播放方法、装置和电子设备
WO2023179539A1 (zh) 视频编辑方法、装置及电子设备
WO2022174826A1 (zh) 图片处理方法、装置、设备及存储介质
US20130308836A1 (en) Photo image managing method and photo image managing system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20903996

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020903996

Country of ref document: EP

Effective date: 20220610

NENP Non-entry into the national phase

Ref country code: DE