WO2023030269A1

WO2023030269A1 - Audio and video processing method and apparatus, electronic device and readable storage medium

Info

Publication number: WO2023030269A1
Application number: PCT/CN2022/115581
Authority: WO
Inventors: 蒋作函
Original assignee: 维沃移动通信（杭州）有限公司
Priority date: 2021-08-30
Filing date: 2022-08-29
Publication date: 2023-03-09
Also published as: CN113747230A

Abstract

The present application discloses an audio and video processing method and apparatus, belonging to the technical field of audio and video processing. The method comprises: receiving a first input for target audio and video, the first input comprising a segmentation parameter; in response to the first input, displaying a preset progress bar of the target audio and video, and displaying a node on the preset progress bar, the node being used to segment the target audio and video according to the segmentation parameter, for editing a segmentation result of the target audio and video, and for dividing sub-progress bars of different audio and video clips in at least two audio and video clips in the preset progress bar; receiving a second input for the sub-progress bars, the second input comprising a weighting parameter; in response to the second input, configuring the weighting parameter for the audio and video clips corresponding to the sub-progress bar; and according to the ordering of the at least two audio and video clips, sequentially playing the at least two audio and video clips, the ordering being related to the weighting parameter.

Description

Audio and video processing method, device, electronic device and readable storage medium

Cross References to Related Applications

This application claims the priority of the Chinese patent application submitted to the China Patent Office on August 30, 2021, with the application number 202111008454.X and the title "Audio and Video Processing Method, Device, Electronic Equipment, and Readable Storage Medium", all of which The contents are incorporated by reference in this application.

technical field

The present application belongs to the technical field of audio and video processing, and in particular relates to an audio and video processing method, device, electronic equipment and readable storage medium.

Background technique

With the development of the mobile Internet, the channels for people to obtain information have also become more abundant. Among them, audio and video learning has become an effective way for users to absorb information daily. Compared with the text information method of simply reading books, information browsing in the form of audio and video is more convenient for users.

At present, when browsing audio and video information, in order to browse the key information in the audio and video information, the main processing methods of the user may include triggering fast-forward playback, playback pause, dragging the progress bar, screenshots, text notes, etc. by the user. However, these processing methods need to spend a long time on the screening of key audio and video information, and cannot quickly clarify the key information in the audio and video information, which affects the browsing efficiency of key information in the audio and video information.

Contents of the invention

The purpose of the embodiments of the present application is to provide an audio and video processing method, device, electronic equipment, and readable storage medium, which can solve the problem that the audio and video processing methods in the related art need to spend a long time in the screening of key audio and video information In fact, it is impossible to quickly clarify the key information in the audio and video information, which affects the browsing efficiency of the key information in the audio and video information.

In the first aspect, the embodiment of the present application provides an audio and video processing method, the method comprising:

receiving a first input to the target audio and video, the first input including segmentation parameters;

In response to the first input, displaying a preset progress bar of the target audio and video, and displaying nodes on the preset progress bar, wherein the nodes are used to adjust the target audio and video according to the segmentation parameters Segmenting the video, and editing the segmentation result of the target audio and video, and sub-progress bars for dividing different audio and video segments in at least two audio and video segments in the preset progress bar;

receiving a second input to the sub-progress bar, the second input including a weight parameter;

In response to the second input, setting the weight parameter for the audio and video segment corresponding to the sub-progress bar;

Playing the at least two audio and video clips sequentially according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.

In the second aspect, the embodiment of the present application provides an audio and video processing device, which includes:

The first receiving module is used to receive the first input to the target audio and video, and the first input includes segmentation parameters;

A first response module, configured to display a preset progress bar of the target audio and video in response to the first input, and display nodes on the preset progress bar, wherein the nodes are used to The segment parameter segments the target audio and video, and is used to edit the segmentation result of the target audio and video, and is used to divide different audio and video in at least two audio and video segments in the preset progress bar Fragment's child progress bar;

A second receiving module, configured to receive a second input to the sub-progress bar, where the second input includes a weight parameter;

A second response module, configured to set the weight parameter for the audio and video segment corresponding to the sub-progress bar in response to the second input;

A playing module, configured to sequentially play the at least two audio and video clips according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is The processor implements the steps of the method described in the first aspect when executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.

In a sixth aspect, an embodiment of the present application provides an apparatus configured to perform the steps of the method described in the first aspect.

In the seventh aspect, the embodiment of the present application provides a computer program product, the computer program product is stored in a non-volatile storage medium, and the program product is executed by at least one processor to implement the method described in the first aspect step.

In the embodiment of the present application, the preset progress bar of the target audio and video can be displayed by means of segmentation parameters of the target audio and video input by the user, and nodes can be displayed on the preset progress bar, wherein the The node is used to segment the target audio and video according to the segmentation parameters, and is used to divide the sub-progress bars of different audio and video segments in the preset progress bar, and respond Based on the second input to the sub-progress bar, weight parameters are set for the audio and video clips corresponding to the sub-progress bar, which can be used for the target audio and video in the form of nodes on the preset progress bar based on the segmentation parameters input by the user segment division, then when the user sets the weight parameters of the audio and video, it does not need to spend a long time searching and filtering the key audio and video information that needs to be browsed from the complete target audio and video, and only needs to correspond to each sub-progress after dividing by nodes Screen whether there is key audio and video information and its importance in the audio and video clips associated with the progress bar, so as to set weight parameters for the audio and video clips corresponding to the sub progress bar, and filter from the key audio and video information of long videos to short video clips The screening of key audio and video information in the middle can greatly improve the screening efficiency of key audio and video information, and quickly clarify the key information in the audio and video information; moreover, when playing the audio and video clips after setting the weight parameters, It can be played sequentially according to the sorting method related to the weight parameter, so that more and more important audio and video information can be browsed within a limited time, effectively shortening the browsing time of key information in the target audio and video and improving browsing efficiency. In addition, the browsing of target audio and video in the form of long videos will take up more time costs for users, and it is impossible to quickly identify key audio and video information to facilitate users' targeted information browsing. However, this application uses short video audio and video Playing audio and video content in the form of fragments can also reduce the boringness of browsing the information in the audio and video for users; in addition, by dividing the target audio and video into segments in the preset progress bar, users can Based on the segmentation of the sub-progress bar in the preset progress bar, the segmentation result of the target audio and video after being divided can be intuitively understood, wherein the nodes in the preset progress bar can be used to segment the target audio and video Therefore, it is convenient for the user to fine-tune the segmentation effect of the target audio and video through this node; by setting the weight of the audio and video segment corresponding to the sub-progress bar in the preset progress bar, it is possible to browse the audio and video Efficiently mark audio and video clips that contain important information during the process.

Description of drawings

Fig. 1 is the flowchart of the audio-video processing method of an embodiment of the present application;

Fig. 2A is one of the interface schematic diagrams of the audio and video processing method of an embodiment of the present application;

Fig. 2B is the second schematic interface diagram of the audio and video processing method of an embodiment of the present application;

Fig. 2C is the third schematic interface diagram of the audio and video processing method of an embodiment of the present application;

FIG. 2D is a fourth schematic interface diagram of an audio and video processing method according to an embodiment of the present application;

FIG. 2E is a fifth schematic interface diagram of an audio and video processing method according to an embodiment of the present application;

FIG. 2F is a sixth schematic interface diagram of an audio and video processing method according to an embodiment of the present application;

FIG. 2G is a seventh schematic interface diagram of an audio and video processing method according to an embodiment of the present application;

Fig. 3 is the block diagram of the audio-video processing device of an embodiment of the present application;

FIG. 4 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a hardware structure of an electronic device according to another embodiment of the present application.

specific embodiment

The technical solutions in the embodiments of the present application will be clearly described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

The audio and video processing method provided by the embodiment of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

Referring to Fig. 1, the flow chart of the audio-video processing method of an embodiment of the present application is shown, and described method specifically can comprise the following steps:

Step 101, receiving the first input to the target audio and video, the first input includes segmentation parameters;

Wherein, the processing object of each embodiment of the present application can be video or audio, and the following description will be made by taking the target video as the processing object as an example, but the processing method described by taking video as the processing object is also applicable to the processing of audio, and the principle is similar, so I won't repeat them one by one.

Wherein, the first input may be triggered on a video playback interface, or may be triggered on a video thumbnail interface in the mobile phone.

Wherein, the segmentation parameter is used to equally divide the target video.

The parameter type of the segmentation parameter can include the value of the segment number parameter and the value of the duration parameter, wherein the value of the segment number parameter is used to represent that the target video is divided into the value (for example, 5) video segments, and the duration parameter The value is used to indicate that the target video is equally divided into video clips with a duration of this value (for example, 15 minutes). Of course, considering that the entire video duration of the target video is not necessarily the value of the duration parameter set Integer multiples, therefore, there may be at least one video segment in the segmented video segment whose duration is shorter than the value of the set duration parameter.

In addition, in an embodiment, the segmentation parameter may include the value of the segment number parameter;

In another embodiment, the segmentation parameter may include the value of the duration parameter;

In yet another embodiment, the segment parameter may also include multiple values of the segment number parameter, which are used to divide the target video into multiple equal parts.

For example, if the segment number parameter includes 2 and 3 in turn, the target video can be divided into 2 video segments, and then these 2 video segments can be subdivided into 3 video segments to generate 6 video segments with the same duration. video clip.

Exemplarily, as shown in FIG. 2A , a video playback interface 10 is shown, and the user can click on any position of the playback interface 10 to trigger the display of the playback progress bar 11 and the preset icon of the video in the playback interface 10 12. Among them, the duration of the video is 1.5 hours, and the playback progress is the first 30 minutes. Wherein, the preset icon 12 may be an entry for segment editing of the video. By clicking the preset icon 12 with the user's finger, a text box 13 can be displayed on the playback interface, prompting the user to input the amount of segmentation used to segment the video, which is the segmentation parameter described above. For example, The user can input 6 segments in the text box 13, which means that the target video is divided into 6 video segments according to the playing sequence.

Step 102, in response to the first input, displaying a preset progress bar of the target audio and video, and displaying nodes on the preset progress bar;

Wherein, the node is used to segment the target audio and video according to the segmentation parameters, and to edit the segmentation result of the target audio and video, and to display in the preset progress bar A sub-progress bar that divides different audio-video segments in at least two audio-video segments.

Wherein, in this step, nodes can be laid out on the preset progress bar according to the segmentation parameters, so as to divide the preset progress bar in the form of different sub-progress bars formed by the nodes; since the preset progress bar and the target audio and video The playback time of each is completely corresponding. Therefore, the division of different audio and video segments of the target audio and video can be realized by dividing the preset progress bar into different sub progress bars.

Of course, in this step, the time position of the node displayed in the preset progress bar is mainly used to represent the division method of the first input to the audio and video segments of the target audio and video, but this step does not necessarily divide the target audio and video according to the division method. The video has already been divided into audio and video segments, and the audio and video segments can be divided flexibly at an appropriate time according to needs, so that the target audio and video is divided into at least two audio and video segments.

For example, in this step, the target audio and video can be divided into audio and video segments. Taking video as an example, when this step is segmented, the video is divided into equal parts, so that at least two video segments are generated. The duration of most of the video clips in is the same; and, when the video is segmented, it is sequentially segmented according to the order of playing time, so at least two generated video clips are played continuously In other words, at least two video clips are sequentially connected according to the playing time, and the original target video can still be restored.

For another example, after the following step 105, that is, after the weight parameter is set for the audio and video segment corresponding to the sub-progress bar, the audio and video segment is divided into the target audio and video, because, after the weight parameter is set, then you can It is clear that the user has determined the segmentation results of the target audio and video corresponding to each node displayed in the preset progress bar, and before setting the weight parameters, the user can segment the target audio and video according to the segmentation parameters as needed The subsequent segmentation results can be flexibly adjusted.

Taking video as an example, in order to visualize the segmented result of the target audio and video, the segmented progress bar can be used to reflect the segmented result of the video, then it is used to display the segmented result of the video, or used to The progress bar for segment editing the video can be called a preset progress bar. The preset progress bar can be the playback progress bar 11 in FIG. 2B, and this purpose can be achieved by adding some additional functions of video editing provided by the application to the playback progress bar 11; or, the preset progress bar can also be It is a separately set progress bar that can be used to segment the video, such as the segment progress bar 20 shown in FIG. 2B .

It should be noted that the length of the paragraph progress bar 20 and the indicated video duration, and the playing time range are the same as the playback progress bar 11. In FIG. 2B, the paragraph progress bar 20 is displayed on the playback progress bar 11 in parallel. above, and the start position and end position of the two progress bars are aligned, that is to say, the paragraph progress bar 20 also represents the playback order of the video frames from the beginning of the progress bar to the end of the progress bar, except that the paragraph The progress bar 20 does not display the playback progress. In other embodiments, the application does not limit the position of the paragraph progress bar 20 relative to the playback progress bar 11. For example, on the basis of FIG. 2B, the paragraph progress bar 20 can be translated to the left toward the camera of the mobile phone.

In the following, the default progress bar is the paragraph progress bar 20 as an example for illustration, but this is not intended to limit the present application.

Optionally, based on the nodes in the preset progress bar, after segmenting the target audio and video, the time point of one audio and video segment in the generated at least two audio and video segments may include each playback of the audio and video segment Time point, or, the playing time range of the audio and video clip, or, the start playing time point and the ending playing time point of the audio and video clip.

Illustratively, after inputting and confirming the segmentation amount in FIG. 2A, it can turn to display FIG. 2B. In FIG. In other embodiments, the sub-progress bars corresponding to different video segments can be marked with different colors in the segment progress bar.

In addition, node A, node B, node C, node D, node E, and node F may be displayed in the paragraph progress bar 20, and these nodes are used to divide the sub-progress bars of each video segment. Here, different letters are used to identify different nodes, and in other embodiments, other symbols can also be used to indicate different nodes.

In addition, the six nodes here are generated at corresponding positions on the paragraph progress bar according to the end time points of each video segment. In other embodiments, each node can also be generated on the paragraph progress bar 20 according to the start time point of each video segment, and can also be generated on the paragraph progress bar 20 according to the start time point and the end time point of each video segment. Each node is generated, and the nodes can also be divided according to the manner of sub-progress bars capable of distinguishing different video clips (for example, node F in FIG. 2B can be omitted).

In Fig. 2B, the target video can be divided into 6 sections, the video segments of the 0th to 15th minutes of the target video correspond to the sub-progress bar 21, the video segments of the 15th to 30th minutes correspond to the sub-progress bar AB, and the 30th to 30th minute video segments correspond to the sub-progress bar AB, The 45-minute video segment corresponds to the sub-progress bar BC, the 45th-60th-minute video segment corresponds to the sub-progress bar CD, the 60th-75th minute video segment corresponds to the sub-progress bar DE, and the 75th-90th minute video segment corresponds to the sub-progress bar DE. Corresponds to the child progress bar EF. Among them, the different video clips exemplified here do not include the same frame of video image, for example, there is only one video between the 15th to 30th minute video clip and the 30th to 15th minute video clip to determine that there is a 30th minute frame image.

In this way, the user can indicate that the video is divided into multiple equal parts by inputting the segment amount to form multiple video segments, and mark the end node of the segment, wherein the end node of the upper segment is the start node of the lower segment.

Step 104, receiving a second input to the sub-progress bar, where the second input includes a weight parameter;

Wherein, as described above, considering that there are multiple ways to generate nodes in the sub-progress bar, the sub-progress bar may include at least one node among two nodes: a start node and an end node. Then the second input can be an input to a paragraph of the sub-progress bar, for example, a finger clicks on the paragraph of the sub-progress bar CD in FIG. 2D; the second input can also be an input to any node of the sub-progress bar, For example, the finger clicks node A of the sub-progress bar 21 in FIG. 2D .

As shown in Figure 2D, after clicking node A of the sub-progress bar 21, a text box 31 with transparency can be displayed on the playback interface, prompting the user to set the weight of the 0-15 minute video segment corresponding to the sub-progress bar 21 Value, optionally, title can also be set to the video segment of the 0th～15 minutes; Similarly, after sub-progress bar 22 (being sub-progress bar CD) is clicked, text box 32 can also be displayed to prompt the user Set the weight value of the 60th-75th minute video segment corresponding to the sub-progress bar 22, and optionally, set the title.

In addition, step 104 may trigger a second input on the sub-progress bar corresponding to the video clip whose weight needs to be set to set a weight value for the video clip corresponding to the sub-progress bar. For example, the weight value ranges from 0 to 10. The higher the value, the higher the importance of key information included in the video segment.

Among them, when the weight value of a video clip is set to 0, it means that the information in the video clip is not important. The sorting of the clips may also not participate in the playback of the video clips.

Therefore, when the user thinks that certain video clips are not important, the weight of the video clip can be set to 0, or the weight of the video clip can not be set directly, so as to achieve the effect of subsequent filtering of the video clip.

Step 105, in response to the second input, setting the weight parameter for the audio and video segment corresponding to the sub-progress bar;

Step 106: Play the at least two audio and video clips sequentially according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.

Wherein, in this step, each video segment may be automatically played according to the order, or the step of playing may be triggered after receiving the input from the user.

Optionally, when step 106 is executed, the at least two audio and video clips may be sorted based on preset parameters, wherein the preset parameters include the weight parameters, or the weight parameters of the audio and video clips and the duration; then, play the at least two audio and video clips sequentially according to the sequence.

Wherein, taking video as an example, each video clip may be played sequentially according to the order of weight value, or may be played sequentially according to the order of ratio of weight value to duration value.

In the embodiment of the present application, at least two audio and video clips can be sorted according to the weight parameters, and output and played according to the sorting, which can facilitate the user to quickly browse to the audio and video information with higher importance; At least two audio and video clips are output and played by sorting the ratio of the weight parameter to the duration, so that the user can browse more and more important audio and video information in a short period of time.

Optionally, after step 102 and before step 104, the method according to the embodiment of the present application may further include: receiving a third input of a target node in the nodes corresponding to the sub-progress bar; responding to the third Input, adjust the time point and duration corresponding to the first audio and video segment in the target audio and video, wherein the first audio and video segment is the sub-progress bar of the at least two audio and video segments including the target node Audio and video clips.

Wherein, when the user is not satisfied with the division effect of the audio and video segments directly according to the segmentation parameters, the fine-tuning of the division method of the audio and video segments can be realized by triggering the third input to the node of the sub-progress bar.

The target node may be the start node and/or the end node of the sub-progress bar. Referring to Figure 2B and Figure 2C, the user drags node B along the direction of the arrow in Figure 2C to make the time point and duration of the video segment corresponding to the sub-progress bar AB, and the time point and duration of the video segment corresponding to the sub-progress bar BC. The duration is adjusted. Among them, the video playback range of the video segment corresponding to the sub-progress bar AB is elongated, and the duration is also elongated, specifically the end position is extended; while the video playback range of the video segment corresponding to the sub-progress bar BC is elongated. shortened, the duration is also shortened, specifically from the starting position.

For example, if node B drags the video duration of 5 minutes along the direction of the arrow, the video segment corresponding to the sub-progress bar AB is adjusted from the video segment of the 15th to 30th minute to the video segment of the 15th to 35th minute; the sub-progress bar BC The corresponding video clips are adjusted from the 30th to 45th minute video clips to the 35th to 45th minute video clips.

It should be noted that triggering the third input on the target node of the sub-progress bar in the preset progress bar will not change the playback progress of the target audio and video, even if the playback progress bar 11 in Figure 2A is used as the preset progress bar, for The movement of the node will not change the current playback progress of the video.

In the embodiment of the present application, since the segmented parameters are used to divide the audio and video segments of the target audio and video, the duration of the generated audio and video segments is generally the same, so in order to enable the user to The actual layout of information points is used to divide the audio and video segments. Users can flexibly adjust the time points of each audio and video segment generated by segment parameters and duration, so that the adjusted audio and video clips meet the user's browsing needs for key information (that is, information of interest).

Optionally, after step 105 and before step 106, the method of the embodiment of the present application can also display the weight parameters set for each audio and video segment on the sub-progress bar corresponding to each audio and video segment, and the specific display position can be Is the middle position of the child progress bar.

Optionally, if the user does not set a title for each audio/video segment, the method in this embodiment of the present application may use the end node of each sub-progress bar as the title of the audio/video segment of the sub-progress bar.

For example, in the paragraph progress bar 20 in Fig. 2E, the middle position in each sub-progress bar has all shown the value of the weight parameter, and the sub-progress bar 21 has shown the weight value of the video clip (titled A) of the 0th to 15 minutes is 3, the sub-progress bar AB shows the weight value of the 15th to 35th minute video clip (titled as B) is 8, and the sub-progress bar BC shows the weight value of the 35th to 45th minute video clip (titled as C) is 2, the sub-progress bar CD shows the weight value of the 45th to 60th minute video segment (titled as D), and the sub-progress bar DE shows the weight value of the 60th to 75th minute video segment (titled as E) is 2, and the sub-progress bar EF shows that the weight value of the video segment (titled F) from the 75th to the 90th minute is 5.

Optionally, when the at least two audio and video clips are sorted based on preset parameters, among the at least two audio and video clips, the audio and video clips whose weight parameter is a target value and whose weight is not set parameters of the audio and video clips; then, based on the preset parameters, the filtered at least two audio and video clips are sorted.

For example, referring to the example of the weight value above, a weight value of 0 indicates that the video segment is not important, so the target value here is 0. In other embodiments, considering the difference in the value range of the weight value, the target The value can also be other values except 0, which is not limited in this application. Then in FIG. 2E , the video clips corresponding to the sub-progress bar CD will be filtered out and will not participate in sorting and sequential playback.

In the embodiment of the present application, when the target audio and video is segmented according to the segmentation parameters (such as the duration, or the amount of segmentation), the generated audio and video segments can be time-continuous between each audio and video segments, thus forming a complete The target audio and video, then considering that some audio and video clips in the target audio and video do not require the user to pay special attention to browse, therefore, the user can set the weight value of the audio and video clip to the target value, or not set the weight value way, to achieve the purpose of discarding some audio and video segments after segmentation, these discarded audio and video segments do not need to participate in the sorting between audio and video segments and do not need to participate in video playback according to audio and video segments, thus saving The user's browsing time of the video improves the browsing efficiency and the browsing efficiency of the points of interest in the audio and video.

Optionally, in the case where the preset parameters include the weight parameter, when sorting the at least two audio and video clips based on the preset parameters, the values of the weight parameters may be from high to low The order of the at least two audio and video clips is sorted.

Among them, when sorting, if there are two or more audio and video clips with the same weight value, these audio and video clips with the same weight value can be ordered according to the playing time in the target audio and video, or their The sub-progress bars are sorted in the order in which they are arranged in the preset progress bar.

Therefore, as shown in FIG. 2G , the video segment of the sub-progress bar BC, that is, the video segment titled C has a weight value of 2, but it is arranged in front of the video segment titled E with the same weight value of 2.

In the embodiment of the present application, when sorting at least two audio and video clips after segmentation, they can be sorted and played according to the value of the weight parameter of each audio and video clip from large to small, so that priority can be given to Playing audio and video clips with higher weight, in the case of limited audio and video browsing time, can ensure that users browse to the more important audio and video content first, so that users can browse the most important or the most important ones by browsing audio and video clips. Interested information, which can help users quickly summarize audio and video information, and facilitate users to effectively browse and review key information from their own perspective.

Optionally, when the preset parameters include the ratio of the weight parameter of the audio and video clips to the duration, when sorting the at least two audio and video clips based on the preset parameters, the audio and video clips may be sorted according to the audio and video clips. The ratio of the value of the weight parameter of the video segment to the duration is used to sort the at least two audio and video segments in descending order.

Among them, when sorting, if there are two or more audio and video clips with the same weight value and duration ratio, then these audio and video clips with the same ratio can be ordered according to the playing time in the target audio and video, or Said that their sub-progress bars are sorted in the order in which they are arranged in the preset progress bar.

Wherein, after each audio and video clip is set with a weight value, then each audio and video clip can be automatically sorted, that is, each audio and video clip is automatically added to the user's video path map (that is, the sorting result box in Figure 2F 41. In the sorting result box 42) in FIG. 2G.

In the embodiment of the present application, when sorting at least two audio and video clips after segmentation, they can be sorted and played according to the ratio of the value of the weight parameter of each audio and video clip to the duration from large to small. , so as to ensure that users can browse more important or interesting audio and video content through the playback of audio and video clips in the shortest time, improve the browsing efficiency of interesting and important audio and video content and shorten the browsing time time.

Optionally, when step 106 is executed, it may be played automatically, or after an input operation is received, or, as described in the following embodiments, be triggered by a user to play after displaying the sorting results.

Optionally, when step 106 is executed, a fourth input to the preset progress bar may be received; in response to the fourth input, a sorting result of the at least two audio and video clips is displayed.

Wherein, the fourth input can be triggered by inputting any sub-progress bar in the preset progress bar, or can be triggered by inputting the start node or end node of any sub-progress bar. The input can be double-clicking Input, or long-press input, etc., this application does not limit this.

As shown in Figure 2F, the finger double-clicks the sub-progress bar AB to display the sorting result box 41, and the sorting result box 41 shows the results of sorting according to the ratio of the weight value to the duration; as shown in Figure 2G, the finger double-clicks the sub-progress bar AB to display the sorting result The result box 42, the sorted result box 42, shows the sorted results according to the weight value.

Optionally, the sorting result box 41 and the sorting result box 42 may be displayed on the playback interface 10 with a certain degree of transparency, or the sorting result may be displayed on the playback interface 10 in the form of subtitles, which is not limited in this application.

In this embodiment of the application, the fourth input can be triggered by the user on the preset progress bar to display the sorting results of at least two audio and video clips, so that the user can browse the sorting results to find the audio and video clips to be browsed.

Optionally, after displaying the sorting results of the at least two audio and video clips in response to the fourth input, the method according to the embodiment of the present application may further include: receiving a list of the second audio and video clips in the sorting results The fifth input of the segment, the at least two audio and video segments include the second audio and video segment; in response to the fifth input, each audio and video segment is played sequentially from the second audio and video segment according to the ordering .

As shown in Figure 2F, if the user needs to select a video segment to be browsed and studied at first from the sorting result box, the title of the corresponding video segment in the sorting result box 41 or the sorting result box 42 can be clicked to trigger the selection from the sorting result box. The video clip of starts playing each video clip in sorted order.

For example, if the user selects title F in the sorting result box 41, the video segment corresponding to title F will be played, and the video segments corresponding to title C, title A, and title E will be played sequentially.

The method for selecting the video segment to be played first in the sorting result box 42 is the same, and will not be repeated one by one.

In the embodiment of the present application, after displaying the sorting results of each audio and video segment, the user can select an option of an audio and video segment of interest from the sorting results to trigger starting from the selected audio and video segment, and follow the The above sequence plays each video segment in sequence, so that the user can browse to the audio and video segment of interest more quickly.

It should be noted that the audio and video processing method provided in the embodiment of the present application may be executed by an audio and video processing device, or a control module in the audio and video processing device for executing the audio and video processing method. In the embodiment of the present application, the audio and video processing device provided in the embodiment of the present application is described by taking the audio and video processing device executing the audio and video processing method as an example.

Referring to FIG. 3 , it shows a block diagram of an audio and video processing device according to an embodiment of the present application. The audio and video processing device includes:

The first receiving module 201 is used to receive the first input to the target audio and video, and the first input includes segmentation parameters;

The first response module 202 is configured to display a preset progress bar of the target audio and video in response to the first input, and display a node on the preset progress bar, wherein the node is used to follow the The segmentation parameter segments the target audio and video, and is used to edit the segmentation result of the target audio and video, and is used to divide different audio and video in at least two audio and video segments in the preset progress bar. Sub-progress bar for video clips;

The second receiving module 203 is configured to receive a second input to the sub-progress bar, where the second input includes a weight parameter;

The second response module 204 is configured to set the weight parameter for the audio and video segment corresponding to the sub-progress bar in response to the second input;

The playing module 205 is configured to sequentially play the at least two audio and video clips according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.

Optionally, the device also includes:

A third receiving module, configured to receive a third input to a target node in the nodes corresponding to the sub-progress bar;

The third response module is configured to adjust the corresponding time point and duration of the first audio and video segment in the target audio and video in response to the third input, wherein the first audio and video segment is the at least two The sub-progress bar in the audio and video segment includes the audio and video segment of the target node.

Optionally, the playback module 205 includes:

A sorting submodule, configured to sort the at least two audio and video clips based on preset parameters, wherein the preset parameters include the weight parameter, or the ratio of the weight parameter to the duration of the audio and video clip;

The playing sub-module is used to play the at least two audio and video clips sequentially according to the sequence.

Optionally, the sorting submodule includes:

A filtering unit, configured to filter the audio and video segments whose weight parameters are target values and the audio and video segments whose weight parameters are not set among the at least two audio and video segments;

A sorting unit, configured to sort the at least two filtered audio and video clips based on the preset parameters.

Optionally, the sorting submodule is further configured to sort the at least two audio and video files according to the order of values of the weight parameters from high to low when the preset parameters include the weight parameter. Fragments are sorted.

Optionally, the sorting submodule is further configured to, in the case that the preset parameters include the ratio of the weight parameter of the audio-video segment to the duration, according to the value of the weight parameter of the audio-video segment The at least two audio and video clips are sorted in descending order according to the ratio of the duration to the duration.

Optionally, the playback module 205 includes:

A receiving submodule, configured to receive a fourth input to the preset progress bar;

The display submodule is used to display the sorting results of the at least two audio and video clips in response to the fourth input.

Optionally, the device also includes:

A fourth receiving module, configured to receive a fifth input to the second audio-video segment in the sorting result, the at least two audio-video segments include the second audio-video segment;

The fourth response module is used to respond to the fifth input and play each audio and video segment sequentially from the second audio and video segment according to the ordering.

The audio and video processing device in this embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant). assistant, PDA), etc., and the non-mobile electronic device may be a personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., which are not specifically limited in this embodiment of the present application.

The audio and video processing device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The audio and video processing device provided in the embodiment of the present application can realize each process realized in the above method embodiment, and to avoid repetition, details are not repeated here.

Optionally, as shown in FIG. 4 , the embodiment of the present application further provides an electronic device 2000, including a processor 2002, a memory 2001, and programs or instructions stored in the memory 2001 and operable on the processor 2002, When the program or instruction is executed by the processor 2002, each process of the above-mentioned audio and video processing method embodiment can be achieved, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 5 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1010 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 5 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .

Wherein, the user input unit 1007 is configured to receive a first input to the target audio and video, the first input includes segmentation parameters; receive a second input to the sub-progress bar, the second input includes a weight parameter;

Processor 1010, configured to display a preset progress bar of the target audio and video in response to the first input, and display nodes on the preset progress bar, wherein the nodes are used to follow the segmentation The parameters segment the target audio and video, and are used to edit the segmentation result of the target audio and video, and to divide different audio and video segments in at least two audio and video segments in the preset progress bar The sub-progress bar; the weight parameter is set for the audio and video segment corresponding to the sub-progress bar;

The audio output unit 1003 is configured to sequentially play the at least two audio and video clips according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.

Optionally, the user input unit 1007 is configured to receive a third input to the target node in the nodes corresponding to the sub-progress bar;

Processor 1010, configured to adjust the time point and duration corresponding to the first audio-video segment in the target audio-video segment in response to the third input, wherein the first audio-video segment is the at least two The sub-progress bar in each audio-video segment includes the audio-video segment of the target node.

Optionally, the processor 1010 is configured to sort the at least two audio and video clips based on preset parameters, wherein the preset parameters include the weight parameter, or, the weight parameter of the audio and video clip is the same as ratio of duration;

The display unit 1006 is configured to sequentially play the at least two audio and video clips according to the sequence.

Optionally, the processor 1010 is configured to filter the at least two audio and video clips, the audio and video clips whose weight parameters are target values and the audio and video clips whose weight parameters are not set; The filtered at least two audio and video clips are sorted.

Optionally, the processor 1010 is configured to sort the at least two audio and video clips according to the descending order of the value of the weight parameter when the preset parameter includes the weight parameter .

Optionally, the processor 1010 is configured to, if the preset parameter includes the ratio of the weight parameter of the audio and video clip to the duration, according to the value of the weight parameter of the audio and video clip and the According to the ratio of duration, the at least two audio and video clips are sorted in descending order.

Optionally, the user input unit 1007 is configured to receive a fourth input to the preset progress bar;

The processor 1010 is configured to display the sorting results of the at least two audio and video clips in response to the fourth input.

In the embodiment of the present application, the fourth input can be triggered by the user on the preset progress bar to display the sorting results of at least two audio and video clips, so that the user can browse the sorting results to find the audio and video clips to be browsed.

Optionally, the user input unit 1007 is configured to receive a fifth input to the second audio-video segment in the sorting result, the at least two audio-video segments include the second audio-video segment;

The processor 1010 is configured to, in response to the fifth input, play each audio and video segment sequentially starting from the second audio and video segment according to the sequence.

It should be understood that, in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 is used for the image capture device ( Such as the image data of the still picture or video obtained by the camera) for processing. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 can include two parts: a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here. The memory 1009 can be used to store software programs as well as various data, including but not limited to application programs and operating systems. Processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communications. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by the processor, the various processes of the above-mentioned audio and video processing method embodiment are realized, and can achieve The same technical effects are not repeated here to avoid repetition.

Wherein, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer readable storage medium, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the above audio and video processing method embodiment Each process, and can achieve the same technical effect, in order to avoid repetition, will not repeat them here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims

An audio and video processing method, the method comprising:

receiving a first input to the target audio and video, the first input including segmentation parameters;

In response to the first input, displaying a preset progress bar of the target audio and video, and displaying nodes on the preset progress bar, wherein the nodes are used to adjust the target audio and video according to the segmentation parameters Segmenting the video, and editing the segmentation result of the target audio and video, and sub-progress bars for dividing different audio and video segments in at least two audio and video segments in the preset progress bar;

receiving a second input to the sub-progress bar, the second input including a weight parameter;

In response to the second input, setting the weight parameter for the audio and video segment corresponding to the sub-progress bar;

Playing the at least two audio and video clips sequentially according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.
The method according to claim 1, wherein, before receiving the second input to the sub-progress bar, the method further comprises:

receiving a third input to a target node in the nodes corresponding to the sub-progress bar;

In response to the third input, adjust the time point and duration corresponding to the first audio-video segment in the target audio-video segment, wherein the first audio-video segment is a sub-progress bar in the at least two audio-video segments An audio and video segment including the target node.
The method according to claim 1, wherein said playing said at least two audio and video clips sequentially according to the ordering of said at least two audio and video clips comprises:

Sorting the at least two audio and video clips based on preset parameters, wherein the preset parameters include the weight parameter, or the ratio of the weight parameter to the duration of the audio and video clip;

Playing the at least two audio and video clips sequentially according to the sequence.
The method according to claim 3, wherein said sorting said at least two audio and video clips based on preset parameters comprises:

Among the at least two audio and video segments, the weight parameter is the audio and video segment of the target value and the audio and video segment without the weight parameter is filtered;

Based on the preset parameters, the filtered at least two audio and video clips are sorted.
The method according to claim 3, wherein, in the case where the preset parameters include the weight parameter, the sorting the at least two audio and video clips based on the preset parameters includes:

The at least two audio and video clips are sorted in descending order of values of the weight parameters.
The method according to claim 3, wherein, in the case where the preset parameters include the ratio of the weight parameter of the audio and video clips to the duration, the at least two audio and video clips are performed based on the preset parameters Sort, including:

The at least two audio and video segments are sorted in descending order according to the ratio of the value of the weight parameter of the audio and video segments to the duration.
The method according to claim 1, wherein said playing said at least two audio and video clips sequentially according to the ordering of said at least two audio and video clips comprises:

receiving a fourth input to the preset progress bar;

In response to the fourth input, display the sorting results of the at least two audio and video clips.
The method according to claim 7, wherein, after displaying the sorting results of the at least two audio and video clips in response to the fourth input, the method further comprises:

receiving a fifth input to a second audiovisual segment in the sorting result, the at least two audiovisual segments including the second audiovisual segment;

In response to the fifth input, the audio and video clips are played sequentially from the second audio and video clip according to the sequence.
An audio and video processing device, the device comprising:

The first receiving module is used to receive the first input to the target audio and video, and the first input includes segmentation parameters;

A first response module, configured to display a preset progress bar of the target audio and video in response to the first input, and display nodes on the preset progress bar, wherein the nodes are used to The segment parameter segments the target audio and video, and is used to edit the segmentation result of the target audio and video, and is used to divide different audio and video in at least two audio and video segments in the preset progress bar Fragment's child progress bar;

A second receiving module, configured to receive a second input to the sub-progress bar, where the second input includes a weight parameter;

A second response module, configured to set the weight parameter for the audio and video segment corresponding to the sub-progress bar in response to the second input;

A playing module, configured to sequentially play the at least two audio and video clips according to the order of the at least two audio and video clips, wherein the order is related to the weight parameter.
The device according to claim 9, wherein the device further comprises:

A third receiving module, configured to receive a third input to a target node in the nodes corresponding to the sub-progress bar;

The third response module is configured to adjust the corresponding time point and duration of the first audio and video segment in the target audio and video in response to the third input, wherein the first audio and video segment is the at least two The sub-progress bar in the audio and video segment includes the audio and video segment of the target node.
The device according to claim 9, wherein the playing module comprises:

A sorting submodule, configured to sort the at least two audio and video clips based on preset parameters, wherein the preset parameters include the weight parameter, or the ratio of the weight parameter to the duration of the audio and video clip;

The playing sub-module is used to play the at least two audio and video clips sequentially according to the sequence.
The device according to claim 11, wherein the sorting submodule comprises:

A filtering unit, configured to filter the audio and video segments whose weight parameters are target values and the audio and video segments whose weight parameters are not set among the at least two audio and video segments;

A sorting unit, configured to sort the at least two filtered audio and video clips based on the preset parameters.
The device according to claim 11, wherein the sorting submodule is further configured to, in the case that the preset parameters include the weight parameter, sort the values of the weight parameter in descending order The at least two audio and video segments are sorted.
The device according to claim 11, wherein the sorting submodule is further configured to, when the preset parameters include the ratio of the weight parameter of the audio-video segment to the duration, sort the audio-video segment according to the The ratio of the value of the weight parameter to the duration is used to sort the at least two audio and video clips in descending order.
The device according to claim 9, wherein the playing module comprises:

A receiving submodule, configured to receive a fourth input to the preset progress bar;

The display submodule is used to display the sorting results of the at least two audio and video clips in response to the fourth input.
The device according to claim 15, wherein the device further comprises:

A fourth receiving module, configured to receive a fifth input to the second audio-video segment in the sorting result, the at least two audio-video segments include the second audio-video segment;

The fourth response module is configured to, in response to the fifth input, play each audio and video clip sequentially starting from the second audio and video clip according to the sequence.
An electronic device, comprising a processor, a memory, and a program or instruction stored on the memory and operable on the processor, when the program or instruction is executed by the processor, claims 1 to 8 are realized The step of the audio-video processing method described in any one.
A readable storage medium, storing programs or instructions on the readable storage medium, and implementing the steps of the audio and video processing method according to any one of claims 1 to 8 when the programs or instructions are executed by a processor.
A device configured to execute the steps of the audio and video processing method according to any one of claims 1 to 8.
A chip, comprising a processor and a communication interface, the communication interface is coupled to the processor, the processor is used to run programs or instructions to implement the audio and video processing method according to any one of claims 1 to 8 step.
A computer program product, the computer program product is stored in a non-volatile storage medium, and the program product is executed by at least one processor to realize the steps of the audio and video processing method according to any one of claims 1 to 8 .