WO2021052058A1

WO2021052058A1 - Video processing method and apparatus, electronic device and storage medium

Info

Publication number: WO2021052058A1
Application number: PCT/CN2020/108268
Authority: WO
Inventors: 耿玉峰; 周超
Original assignee: 北京达佳互联信息技术有限公司
Priority date: 2019-09-20
Filing date: 2020-08-10
Publication date: 2021-03-25
Also published as: CN110602122A

Abstract

The present application relates to a video processing method and apparatus, an electronic device and a storage medium, and relates to the technical field of multimedia information processing. The video processing method comprises: encoding a video to be processed, and generating a plurality of video fragments one by one according to an encoding sequence; when each video fragment is obtained, and after a fragment identifier is allocated to the video fragment, uploading the video fragment and the corresponding fragment identifier to a serving end, so as to enable the serving end to decode the video fragment according to the fragment identifier of the video fragment after receiving the video fragment, wherein the fragment identifier is used for marking the sequence of generating a plurality of video fragments; and in the process of uploading each video fragment, if the encoding for the video to be processed is not finished, continuously encoding the video to be processed to obtain the next video fragment. Through the uploading of the method, most of the time for uploading and decoding by the serving end is saved, and therefore, the overall time consumption is markedly reduced.

Description

Video processing method, device, electronic equipment and storage medium

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 20, 2019, the application number is 201910894816.6, and the invention title is "video processing methods, devices, electronic equipment and storage media", the entire contents of which are incorporated by reference In this application.

Technical field

This application relates to the technical field of multimedia information processing, and in particular to a video processing method, device, electronic equipment, and storage medium.

Background technique

Video upload is the core function of short video sharing software. It shortens the time-consuming video upload and the delay in distributing to users after uploading, which will greatly enhance the author's content production experience. The time consumption of video uploading and sharing is limited by multiple processes such as terminal video transcoding speed, transcoding file size, uplink network bandwidth, and server transcoding time consumption, and there is a large room for optimization.

Short video sharing generally includes five steps: user shooting and editing works, video file terminal transcoding, video file network upload, server transcoding, and video file distribution. In addition to the user’s initiative to shoot and edit works, terminal transcoding and uploading require the user to wait for completion. Otherwise, the software may be restricted from running on the server or killed by the system, causing the upload to fail. The server transcoding can be asynchronous. Proceed, and then distribute through CDN (Content Delivery Network) after completion. Generally speaking, after the user finishes editing through the terminal, the original file first needs to be transcoded and compressed by the editing module to generate a small video file that is easy to upload, and then call the network interface to upload the file to the server. The server receives the file, calls the transcoding service to further process the video (such as further compressing the size, transferring multiple files with different bitrates, etc.) before distributing it.

The inventor found that related technologies have been optimized in terms of improving transcoding and uploading speed, but the inventor found that the industry's transcoding and transmission speed optimization has reached a relative bottleneck, and the optimization space is limited.

Summary of the invention

This application provides a video processing method, device, electronic equipment, and storage medium to at least solve the problem that the speed optimization of transcoding and transmission has reached a relative bottleneck in related technologies. The form of video fragmentation can realize a highly parallel uploading video processing method in the three stages of client-side transcoding, uploading, and server-side transcoding. This method greatly saves the time required for the successful publication of users' video works.

According to a first aspect of the embodiments of the present application, a video processing method is provided, including:

Encode the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

Each time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server receives the video segment Then decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

In the process of uploading each video segment, if the encoding of the to-be-processed video is not completed, the encoding of the to-be-processed video is continued at the same time to obtain the next video segment.

According to a second aspect of the embodiments of the present application, a video processing method is provided, including:

Receiving each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

According to the segment identifier, decode each video segment;

In the process of decoding each video segment, if there is an upload of the next video segment, the next video segment is received at the same time.

According to a third aspect of the embodiments of the present application, a video processing device is provided, including:

The first encoding module is configured to perform encoding of the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

The uploading module is configured to execute each time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server is After receiving the video segment, decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

The second encoding module is configured to perform the process of uploading each video segment, if the encoding for the to-be-processed video is not finished, at the same time continue to encode the to-be-processed video to obtain the next video segment.

According to a fourth aspect of the embodiments of the present application, a video processing device is provided, including:

The first receiving module is configured to receive each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

The decoding module is configured to perform decoding of each video segment according to the segment identifier;

The second receiving module is configured to perform, in the process of decoding each video segment, if there is an upload of the next video segment, simultaneously receive the next video segment.

According to a fifth aspect of the embodiments of the present application, an electronic device is provided, including:

A memory for storing computer programs, candidate intermediate data and result data generated by executing the computer programs;

Processor for:

According to a sixth aspect of the embodiments of the present application, a computer storage medium is provided, the computer storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute any video provided in the embodiments of the present application. Approach.

Description of the drawings

The drawings here are incorporated into the specification and constitute a part of the specification, show embodiments conforming to the application, and together with the specification are used to explain the principle of the application, and do not constitute an improper limitation of the application.

FIG. 1 is a structural diagram of a terminal device provided by an embodiment of this application;

2 is a schematic flowchart of a video serial upload method provided by an embodiment of the application;

FIG. 3 is a sequence diagram of a video serial upload method provided by an embodiment of the application;

4 is a schematic flowchart of a terminal side of a video processing method provided by an embodiment of this application;

FIG. 5 is a schematic flowchart of a server side of a video processing method provided by an embodiment of this application;

FIG. 6 is a sequence diagram of a video processing method provided by an embodiment of this application;

FIG. 7 is a schematic structural diagram of a terminal side of a video processing device provided by an embodiment of the application;

FIG. 8 is a schematic structural diagram of a server side of a video processing device provided by an embodiment of this application;

FIG. 9 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

detailed description

In order to enable those of ordinary skill in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings.

The terminal equipment involved in the embodiments of the present application may also be referred to as user equipment (UE). Terminal devices can be smart phones, tablet computers, various wearable devices, vehicle-mounted devices, and so on. Various applications, such as social software, maps, etc., can be installed in the terminal device.

The multiple mentioned in the embodiments of the present application refer to greater than or equal to two.

The embodiments of the present application provide a video processing method and terminal equipment, and the method is suitable for terminal equipment. Figure 1 shows a structural diagram of a possible terminal device. 1, the terminal device 100 includes: a radio frequency (RF) circuit 110, a power supply 120, a processor 130, a memory 140, an input unit 150, a display unit 160, a camera 170, a communication interface 180, and a wireless Components such as a wireless fidelity (Wireless Fidelity, WiFi) module 190. Those skilled in the art can understand that the structure of the terminal device shown in FIG. 1 does not constitute a limitation on the terminal device. The terminal device provided by the embodiment of the present application may include more or less components than those shown in the figure, or a combination of certain components. Some components, or different component arrangements.

Hereinafter, each component of the terminal device 100 will be specifically introduced with reference to FIG. 1:

The RF circuit 110 can be used to receive and send data during a communication or call. In particular, after the RF circuit 110 receives the downlink data from the base station, it sends it to the processor 130 for processing; in addition, it sends the uplink data to be sent to the base station. Generally, the RF circuit 110 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like.

In addition, the RF circuit 110 may also communicate with the network and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division Multiple Access). Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), Email, Short Messaging Service (SMS), etc.

The WiFi technology is a short-range wireless transmission technology. The terminal device 100 can be connected to an access point (AP) through the WiFi module 190, so as to achieve data network access. The WiFi module 190 can be used to receive and send data in the communication process.

The terminal device 100 may be physically connected with other devices through the communication interface 180. In one embodiment, the communication interface 180 and the communication interface of the other device are connected by a cable to realize data transmission between the terminal device 100 and the other device.

Since in the embodiment of the present application, the terminal device 100 can implement communication services and send information to other contacts, the terminal device 100 needs to have a data transmission function, that is, the terminal device 100 needs to include a communication module inside. Although FIG. 1 shows communication modules such as the RF circuit 110, the WiFi module 190, and the communication interface 180, it is understandable that there are at least one of the above-mentioned components or other functions in the terminal device 100. A communication module (such as a Bluetooth module) for realizing communication for data transmission.

For example, when the terminal device 100 is a mobile phone, the terminal device 100 may include the RF circuit 110 and may also include the WiFi module 190; when the terminal device 100 is a computer, the terminal device 100 may The communication interface 180 may be included, and the WiFi module 190 may also be included; when the terminal device 100 is a tablet computer, the terminal device 100 may include the WiFi module.

The memory 140 can be used to store software programs and modules. The processor 130 executes various functional applications and data processing of the terminal device 100 by running software programs and modules stored in the memory 140.

In an embodiment, the memory 140 may mainly include a program storage area and a data storage area. Among them, the storage program area can store operating systems, various application programs (such as communication applications), and face recognition modules, etc.; the storage data area can store data created based on the use of the terminal device (such as various pictures and video files). And other multimedia files, as well as face information templates) and so on.

In addition, the memory 140 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

The input unit 150 may be used to receive numeric or character information input by the user, and generate key signal input related to user settings and function control of the terminal device 100.

In one embodiment, the input unit 150 may include a touch panel 151 and other input devices 152.

Wherein, the touch panel 151, also called a touch screen, can collect user touch operations on or near it (for example, the user uses any suitable objects or accessories such as fingers, stylus, etc.) on or on the touch panel 151. The operation near the touch panel 151), and drive the corresponding connection device according to the preset program. In one embodiment, the touch panel 151 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch position, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it To the processor 130, and can receive and execute the commands sent by the processor 130. In addition, the touch panel 151 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave.

In an embodiment, the other input device 152 may include, but is not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackball, mouse, joystick, and the like.

The display unit 160 may be used to display information input by the user or information provided to the user and various menus of the terminal device 100. The display unit 160 is the display system of the terminal device 100, and is used to present an interface and realize human-computer interaction.

The display unit 160 may include a display panel 161. In one embodiment, the display panel 161 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light emitting diode (Organic Light-Emitting Diode, OLED), etc.

Further, the touch panel 151 can cover the display panel 161, and when the touch panel 151 detects a touch operation on or near it, it is transmitted to the processor 130 to determine the type of touch event, Then the processor 130 provides corresponding visual output on the display panel 161 according to the type of the touch event.

Although in FIG. 1, the touch panel 151 and the display panel 161 are used as two independent components to implement the input and input functions of the terminal device 100, but in some embodiments, the The touch panel 151 is integrated with the display panel 161 to realize the input and output functions of the terminal device 100.

The processor 130 is the control center of the terminal device 100, uses various interfaces and lines to connect various components, runs or executes software programs and/or modules stored in the memory 140, and calls The data in the memory 140 executes various functions of the terminal device 100 and processes data, thereby realizing various services based on the terminal device.

In an embodiment, the processor 130 may include one or more processing units. In an embodiment, the processor 130 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 130.

The camera 170 is used for realizing the shooting function of the terminal device 100 and shooting pictures or videos. The camera 170 can also be used to realize the scanning function of the terminal device 100 and scan the scanned object (two-dimensional code/barcode).

The terminal device 100 further includes a power source 120 (such as a battery) for supplying power to various components. In one embodiment, the power supply 120 may be logically connected to the processor 130 through a power management system, so that functions such as charging, discharging, and power consumption can be managed through the power management system.

Although not shown, the terminal device 100 may also include at least one sensor, an audio circuit, etc., which will not be repeated here.

In one embodiment, referring to Figure 2, short video sharing generally includes user shooting and editing works (step 201), terminal transcoding of video files (step 202), network uploading of video files (step 203), and server transcoding (step 204) and the five steps of video file distribution (step 205). Among them, the three steps of terminal transcoding, video file uploading, and server transcoding are background processing processes that can be optimized.

At present, in related technologies, the performance optimization of terminal transcoding, video file upload, and server transcoding generally has reached a relative bottleneck, and the optimization space is limited. Refer to FIG. 3, which is a sequence diagram of a short video upload method. The video processing flow of FIG. 3 may include the following steps 301 to 303.

Step 301: Terminal transcoding.

After the user finishes shooting and editing the work, the first thing to do is transcoding at the terminal. For example, the terminal sequential transcoding of the video file to be processed in FIG. 3 generates video fragments CF1, CF2, CF3, CF4, and CF5. When all the video segments are obtained, step 302 is executed.

Step 302: File upload.

After all the video segments that have been transcoded are obtained, the video segments that have been transcoded for the video file to be processed are uploaded as a whole, such as the video segments of UF1, UF2, UF3, UF4, and UF5 in FIG. 3. When all the video segments of the to-be-processed video file are uploaded, step 303 is continued.

Step 303: Transcoding on the server side.

When the server receives all the video segments of the video file to be processed, it starts to further transcode all the video segments of the video file to be processed on the server, for example, SF1, SF2, SF3, SF4, and SF5 in Figure 3 Fragmentation. After all the video segments of the video file to be processed are transcoded on the server side, the complete video file can be output and subsequent distribution processing can continue.

In view of this, the present application provides a video processing method, which can realize the parallel processing of video file video segments in the above three steps, and save the time for the user's work to be successfully published. This application explains the method separately from the terminal side and the server side. First of all, the implementation on the terminal side is shown in Figure 4. The method includes:

Step 401: Encode the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

Among them, in the field of streaming media transmission, such as the commonly used DASH (Dynamic Adaptive Streaming over HTTP, HTTP-based dynamic adaptive streaming) and HLS (HTTP Live Streaming, HTTP-based streaming media network transmission) protocols commonly used in video on demand, short videos are commonly used Video storage formats include fMP4 (Fragmented MP4, a file format). Unlike ordinary MP4 files, fMP4 allows video files to be organized in video fragments, and can be played in units of video fragments without waiting for the completion of the entire file processing, which solves the problem that MP4 cannot be streamed and played. At the same time, fMP4 also provides the possibility for parallel upload of short videos.

Step 402: After each video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server receives the After the video is segmented, the video segment is decoded according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

In one embodiment, the encoding performed on the terminal side (ie, terminal transcoding) and the upload of the video can be performed by two independent modules of the terminal. The modules can communicate with each other. After the user completes the video editing, the terminal first encodes the video to be processed, and generates multiple video segments one by one in the order of encoding; after each video segment is obtained, the message of the obtained video segment is output to the upload module, and the upload module detects this If a message is received, the upload of the video segment is immediately started. For example, when the encoding in the video file to be processed generates the first video segment, the information of the first video segment is output, and the upload module detects this information and immediately starts uploading the first video segment.

In addition, when the video segments are obtained, in order to enable the server to decode the video segments in the order of the video segments, each video segment is assigned a segment identifier, where the segment identifier is used to mark the generation of multiple video segments. Before and after the sequence; when the terminal receives the video segment and the corresponding segment identifier of the video segment, the terminal decodes each video segment according to the sequence identified in the segment identifier, thereby outputting the video file.

Step 403: In the process of uploading each video segment, if the encoding for the to-be-processed video is not finished, continue to encode the to-be-processed video at the same time to obtain the next video segment.

In one embodiment, when starting the process of uploading video segments, the terminal does not pause the encoding and uploading of the video segments to be processed one by one. Therefore, for the same video to be processed as a whole, the encoding and uploading of the video to be processed It can be performed in parallel; for example, during the uploading process of the first video segment, the second video segment can be generated by encoding in parallel. Therefore, the uploading of the first video segment and the encoding of the second video segment are performed in parallel Processed, that is, after each encoded video segment is obtained, the upload of the video segment can be started. The encoding of the video segment is executed one by one in serial, and does not affect the parallel processing of the video segments generated by each encoding Upload. In this process, the terminal's encoding of the to-be-processed video file and uploading of video segments are executed in parallel. It can be understood that each video segment is an independent pipeline processing line, and after each video segment is encoded, the upload of the video segment can be started. For example, the first video segment is in the process of uploading, and at the same time, the terminal encodes the video to be processed to generate the second video segment. In this implementation manner, it is possible to save the time for uploading video fragments until the terminal obtains all the video fragments by encoding the entire to-be-processed video.

In one embodiment, when multiple video segments are generated one by one in the encoding sequence, the video segments generated one by one are output to a file, and the offset of each video segment in the one file is recorded . Therefore, according to the offset of the video segment in the one file, the data corresponding to the video segment can be read from the one file according to the offset of the video segment; and the video can be uploaded The data of the fragments and the fragment identification are sent to the server.

Or, in another embodiment, after each video segment is obtained, a corresponding video segment file is created for each video segment, and the storage address of each video segment file is recorded. Therefore, it is also possible to read the data of each video segment according to the storage address of the video segment file of each video segment; upload each video segment and the corresponding segment identifier to the server.

In an embodiment, uploading the data corresponding to the video segment to the server needs to be implemented through an upload task. Therefore, every time a video segment is obtained, an upload task for the video segment is created. Wherein, when there are multiple upload tasks, the fragments corresponding to each upload task can be uploaded to the server at the same time. Through this method, the parallel execution of upload tasks of multiple video segments can be realized, thus saving the waiting time for uploading of each video segment.

In one embodiment, when the encoding results of each video segment uploaded by the user are all uploaded, a notification of the completion of the video upload is output. At this time, the upload process for the video segment has ended.

On the other hand, the implementation from the server side is shown in Figure 5. The method includes:

Step 501: Receive each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

Step 502: Decode each video segment according to the segment identifier;

Step 503: In the process of decoding each video segment, if there is an upload of the next video segment, simultaneously receive the next video segment.

In one embodiment, each time the server receives a video segment, it can start to decode the video segment, but you need to pay attention to the transcoding output as a file, so the server's decoding needs to be performed serially, so it needs to be maintained A receiving queue for receiving video segments, sorted according to the segment identifier assigned to each video segment when outputting the encoding result, wherein the segment identifier is used to mark the sequence of generating multiple video segments; Therefore, the video segments can be decoded in sequence according to the segment identifiers. Alternatively, in one embodiment, the video frame information of the video segments can be obtained after decoding, and the decoded video segments can also be sorted according to the video frame information to obtain the entire complete video file.

Through the parallel upload of video segments and the processing of server-side decoding on the terminal side and the server side, the waiting time of each segment when performing serial processing in the related technology can be saved, and thus the time for the user to wait for the upload of the work can be saved.

The following uses another embodiment to further illustrate a video processing method provided by the present application. Refer to FIG. 6, which is a sequence diagram of a video processing method provided by this embodiment of the present application.

Figure 6 includes three main processes: terminal transcoding, uploading, and server transcoding. It can be seen that when the terminal transcodes the first video segment CF1 in the figure, the upload module of the first video segment UF1 starts immediately. During the upload of the first video segment UF1, the terminal transfers The second video segment CF2 obtained by transcoding the to-be-processed video one by one in the code is also being executed in parallel. Subsequently, when the transcoding of the second video segment CF2 in the transcoding stage of the terminal ends, the uploading of the second video segment UF2 is immediately started in the upload module. Among them, the transcoding of the terminal is a process of generating multiple video segments one by one according to the encoding sequence of the video to be processed.

In one embodiment, in the terminal transcoding stage, the video to be processed is encoded to generate video segments one by one, but in the upload stage, each video segment has a different upload duration due to network, file size, etc. If the upload of the previous video segment has not been completed, and the next video segment has been transcoded at the terminal at this time, multiple video segments obtained after transcoding can be uploaded in parallel. For example, in Figure 6 for the upload of video segments UF3, UF4, and UF5, the upload of the video segment UF3 has not been completed, but the terminal has been transcoded to obtain the video segment CF4. At this time, for the video segment UF3 and The upload of the video segment UF4 can be executed in parallel in the upload module. In the same way, the upload of the video segment UF4 has not been completed, but the terminal has been transcoded to obtain the video segment CF5. At this time, the upload of the video segments UF4 and UF5 can be performed in parallel. This method can be implemented through multiple uploads Mission accomplished.

In one embodiment, because a complete video file is to be output, the transcoding on the server side is performed serially. Among them, the server-side transcoding of video segments starts after the upload of each video segment is completed. For example, the video segment SF1 on the server side in the figure starts transcoding on the server side as soon as it receives the upload of the video segment UF1. In one embodiment, when the transcoding of the video segment SF1 is completed and the video segment UF2 has been received at this time, the transcoding of the video segment SF2 is continued. Until all video segments are transcoded on the server side, as shown in the figure, the transcoding of video segment SF5 ends, then the complete video file can be output and the subsequent video file distribution operation can be continued.

FIG. 7 is a schematic structural diagram of a video processing terminal device in an embodiment of the application. The device includes: a first encoding module 701, an uploading module 702, and a second encoding module 703.

The first encoding module 701 is configured to perform encoding of the to-be-processed video, and generate multiple video segments one by one according to the encoding sequence;

The uploading module 702 is configured to perform every time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server After receiving the video segment, decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

The second encoding module 703 is configured to perform the process of uploading each video segment, if the encoding for the to-be-processed video is not finished, at the same time continue to encode the to-be-processed video to obtain the next video segment .

In an embodiment, after the first encoding module 701 is configured to perform the generation of multiple video segments one by one in the encoding sequence, the apparatus further includes:

The output module is configured to output the video segments generated one by one to a file, and record the offset of each video segment in the one file;

The upload module 702 is configured to execute:

For each video segment, read data corresponding to the video segment from the one file according to the recorded offset of the video segment;

Upload the data of the video segment and the segment identifier to the server.

The file creation module is configured to execute after each video segment is obtained, a corresponding video segment file is created for each video segment, and the storage address of each video segment file is recorded;

The upload module 702 is configured to execute:

Read the data of each video segment according to the storage address of the video segment file of each video segment;

And upload each video segment and the corresponding segment identifier to the server.

In one embodiment, the upload module 702 is configured to execute:

Each time a video segment is obtained, an upload task for the video segment is created;

When there are multiple upload tasks, the segments corresponding to each upload task are uploaded to the server at the same time.

In an embodiment, after the upload module 702 is configured to execute uploading each video segment to the server, the apparatus further includes:

The notification module is configured to output a notification that the video upload is completed if it is determined that the upload of each video segment of the video is completed.

Refer to FIG. 8, which is a schematic structural diagram of a video processing server device in an embodiment of the application. The device includes: a first receiving module 801, a decoding module 802, and a second receiving module 803.

The first receiving module 801 is configured to receive each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

The decoding module 802 is configured to perform decoding of each video segment according to the segment identifier;

The second receiving module 803 is configured to perform, in the process of decoding each video segment, if there is an upload of the next video segment, simultaneously receive the next video segment.

In an embodiment, the first receiving module 801 is configured to execute:

Store the video segments uploaded by the terminal in the receiving queue;

Sort the video fragments in the receiving queue according to the fragment identifier;

The decoding module is configured to execute:

The video segments are decoded in sequence according to the segment identifiers.

In an embodiment, after the first receiving module 801 is configured to perform the sequential decoding of video segments, the apparatus further includes:

The sorting module is configured to perform the sorting of the video frames obtained by decoding each video segment according to the segment identifiers.

After introducing the video processing method and device in the exemplary embodiment of the present application, next, the electronic device of another exemplary embodiment of the present application is introduced.

Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, each aspect of the present application can be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as "Circuit", "Module" or "System".

In some possible implementation manners, the electronic device according to the present application may at least include at least one processor and at least one memory. Wherein, the memory stores a program code, and when the program code is executed by the processor, the processor executes the steps in the image processing method according to various exemplary embodiments of the application described above in this specification. For example, the processor may execute step 401 to step 403 shown in FIG. 4 and step 501 to step 503 shown in FIG. 5.

The electronic device 130 according to this embodiment of the present application will be described below with reference to FIG. 9. The electronic device 130 shown in FIG. 9 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present application.

As shown in FIG. 9, the electronic device 130 is represented in the form of a general-purpose computing device. The components of the electronic device 130 may include, but are not limited to: the aforementioned at least one processor 131, the aforementioned at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

The bus 133 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a processor, or a local bus using any bus structure among multiple bus structures.

The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and/or a cache memory 1322, and may further include a read-only memory (ROM) 1323.

The memory 132 may also include a program/utility tool 1325 having a set of (at least one) program modules 1324. Such program modules 1324 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each of the examples or some combination may include the realization of a network environment.

The electronic device 130 may also communicate with one or more external devices 134 (such as keyboards, pointing devices, etc.), and may also communicate with one or more devices that enable the target object to interact with the electronic device 130, and/or communicate with the The electronic device 130 can communicate with any device (such as a router, a modem, etc.) that communicates with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 135. In addition, the computing device 130 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 136. As shown in the figure, the network adapter 136 communicates with other modules for the electronic device 130 through the bus 133. It should be understood that although not shown in the figure, other hardware and/or software modules can be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

In some possible implementation manners, various aspects of the video processing method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to make the computer The device executes the steps in the image processing method according to various exemplary embodiments of the application described above in this specification. For example, the computer device can execute steps 401 to 403 shown in FIG. 4 and the steps shown in FIG. 5 501-Step 503.

The program product can adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

The program product for video processing of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a computing device. However, the program product of this application is not limited to this. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

The readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.

The program code contained on the readable medium can be transmitted by any suitable medium, including, but not limited to, wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

The program code for performing the operations of this application can be written in any combination of one or more programming languages. Programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming. Language-such as "C" language or similar programming language. The program code can be executed entirely on the target computing device, partly executed on the target target device, executed as an independent software package, partly executed on the target computing device and partly executed on the remote computing device, or completely executed on the remote computing device Execute on the device or server. In the case of a remote computing device, the remote computing device can be connected to the target electronic device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using the Internet) The service provider comes to connect via the Internet).

It should be noted that although several units or subunits of the device are mentioned in the above detailed description, this division is only exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that these operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired result. . Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution.

Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

This application is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device, so The instruction device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of this application. These variations, uses, or adaptive changes follow the general principles of this application and include common knowledge or customary technical means in the technical field that are not disclosed in this application. . The description and embodiments are only regarded as exemplary, and the true scope and spirit of the application are pointed out by the following claims.

Claims

A video processing method, including:

Encode the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

Each time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server receives the video segment Then decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

In the process of uploading each video segment, if the encoding of the to-be-processed video is not completed, the encoding of the to-be-processed video is continued at the same time to obtain the next video segment.
The method according to claim 1, after the multiple video segments are generated one by one according to the encoding sequence, the method further comprises:

Output the video segments generated one by one to a file, and record the offset of each video segment in the one file;

The uploading of the video segment and its corresponding segment identifier to the server includes:

For each video segment, read data corresponding to the video segment from the one file according to the recorded offset of the video segment;

Upload the data of the video segment and the segment identifier to the server.
The method according to claim 1, after the multiple video segments are generated one by one according to the encoding sequence, the method further comprises:

After each video segment is obtained, a corresponding video segment file is created for each video segment, and the storage address of each video segment file is recorded;

The uploading of the video segment and its corresponding segment identifier to the server includes:

Read the data of each video segment according to the storage address of the video segment file of each video segment;

And upload each video segment and the corresponding segment identifier to the server.
The method according to claim 1, wherein the uploading the video segment and its corresponding segment identifier to the server includes:

Each time a video segment is obtained, an upload task for the video segment is created;

When there are multiple upload tasks, the segments corresponding to each upload task are uploaded to the server at the same time.
The method according to any one of claims 1-4, after uploading each video segment to the server, the method further comprises:

If it is determined that the upload of each video segment of the video is completed, a notification of the completion of the video upload is output.
A video processing method, including:

Receiving each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

According to the segment identifier, decode each video segment;

In the process of decoding each video segment, if there is an upload of the next video segment, the next video segment is received at the same time.
The method according to claim 6, wherein said receiving each video segment of the video uploaded by the terminal and the segment identifier of each video segment comprises:

Store the video segments uploaded by the terminal in the receiving queue;

Sort the video fragments in the receiving queue according to the fragment identifier;

The decoding of each video segment according to the segment identifier includes:

The video segments are decoded in sequence according to the segment identifiers.
The method according to claim 7, after the decoding the video slices in sequence, further comprising:

The video frames obtained by decoding each video segment are sorted according to the segment identifiers.
A video processing device includes:

The first encoding module is configured to perform encoding of the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

The uploading module is configured to execute each time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server is After receiving the video segment, decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

The second encoding module is configured to perform the process of uploading each video segment, if the encoding for the to-be-processed video is not finished, at the same time continue to encode the to-be-processed video to obtain the next video segment.
The apparatus according to claim 9, after the first encoding module is configured to perform the generation of multiple video segments one by one in the encoding sequence, the apparatus further comprises:

The output module is configured to output the video segments generated one by one to a file, and record the offset of each video segment in the one file;

The upload module is configured to execute:

For each video segment, read data corresponding to the video segment from the one file according to the recorded offset of the video segment;

Upload the data of the video segment and the segment identifier to the server.
The apparatus according to claim 9, after the first encoding module is configured to perform the generation of multiple video segments one by one in the encoding sequence, the apparatus further comprises:

The file creation module is configured to execute after each video segment is obtained, a corresponding video segment file is created for each video segment, and the storage address of each video segment file is recorded;

The upload module is configured to execute:

Read the data of each video segment according to the storage address of the video segment file of each video segment;

And upload each video segment and the corresponding segment identifier to the server.
The apparatus according to claim 9, wherein the upload module is configured to execute:

Each time a video segment is obtained, an upload task for the video segment is created;

When there are multiple upload tasks, the segments corresponding to each upload task are uploaded to the server at the same time.
The apparatus according to any one of claims 9-12, after the upload module is configured to execute uploading each video segment to the server, the apparatus further comprises:

The notification module is configured to output a notification that the video upload is completed if it is determined that the upload of each video segment of the video is completed.
A video processing device includes:

The first receiving module is configured to receive each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

The decoding module is configured to perform decoding of each video segment according to the segment identifier;

The second receiving module is configured to perform, in the process of decoding each video segment, if there is an upload of the next video segment, simultaneously receive the next video segment.
The apparatus according to claim 14, wherein the first receiving module is configured to execute:

Store the video segments uploaded by the terminal in the receiving queue;

Sort the video fragments in the receiving queue according to the fragment identifier;

The decoding module is configured to execute:

The video segments are decoded in sequence according to the segment identifiers.
The apparatus according to claim 15, after the first receiving module is configured to perform the sequential decoding of video segments, the apparatus further comprises:

The sorting module is configured to perform the sorting of the video frames obtained by decoding each video segment according to the segment identifiers.
An electronic device including:

A memory for storing computer programs, candidate intermediate data and result data generated by executing the computer programs;

Processor for:

Encode the video to be processed, and generate multiple video segments one by one according to the encoding sequence;

Each time a video segment is obtained, after assigning a segment identifier to the video segment, upload the video segment and its corresponding segment identifier to the server, so that the server receives the video segment Then decode the video segment according to the segment identifier of the video segment; wherein, the segment identifier is used to mark the sequence of generating multiple video segments;

In the process of uploading each video segment, if the encoding of the to-be-processed video is not completed, the encoding of the to-be-processed video is continued at the same time to obtain the next video segment.
The electronic device according to claim 17, after the processor executes the generation of multiple video segments one by one in the encoding sequence, the processor is further configured to:

Output the video segments generated one by one to a file, and record the offset of each video segment in the one file;

The uploading of the video segment and its corresponding segment identifier to the server includes:

For each video segment, read data corresponding to the video segment from the one file according to the recorded offset of the video segment;

Upload the data of the video segment and the segment identifier to the server.
The electronic device according to claim 17, after the processor executes the generation of multiple video segments one by one in the encoding sequence, the processor is further configured to:

After each video segment is obtained, a corresponding video segment file is created for each video segment, and the storage address of each video segment file is recorded;

The uploading of the video segment and its corresponding segment identifier to the server includes:

Read the data of each video segment according to the storage address of the video segment file of each video segment;

And upload each video segment and the corresponding segment identifier to the server.
The electronic device according to claim 17, when the processor executes the uploading of the video segment and its corresponding segment identifier to the server, it is specifically configured to:

Each time a video segment is obtained, an upload task for the video segment is created;

When there are multiple upload tasks, the segments corresponding to each upload task are uploaded to the server at the same time.
According to the electronic device according to any one of claims 17-20, the processor executes after uploading each video segment to the server, and is further configured to:

If it is determined that the upload of each video segment of the video is completed, a notification of the completion of the video upload is output.
An electronic device including:

A memory for storing computer programs, candidate intermediate data and result data generated by executing the computer programs;

Processor for:

Receiving each video segment of the to-be-processed video uploaded by the terminal and the segment identifier corresponding to each video segment;

According to the segment identifier, decode each video segment;

In the process of decoding each video segment, if there is an upload of the next video segment, the next video segment is received at the same time.
The electronic device according to claim 22, when the processor executes each video segment of the video uploaded by the receiving terminal and the segment identifier of each video segment, it is specifically configured to:

Store the video segments uploaded by the terminal in the receiving queue;

Sort the video fragments in the receiving queue according to the fragment identifier;

The decoding of each video segment according to the segment identifier includes:

The video segments are decoded in sequence according to the segment identifiers.
The electronic device according to claim 23, after the processor is further configured to: after performing the decoding of video segments in sequence:

The video frames obtained by decoding each video segment are sorted according to the segment identifiers.
A computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the method according to any one of claims 1-8.