WO2022082457A1

WO2022082457A1 - Video processing method, apparatus, and device, unmanned aerial vehicle, and image transmission system

Info

Publication number: WO2022082457A1
Application number: PCT/CN2020/122279
Authority: WO
Inventors: 王强; 王博; 朱超
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-10-20
Filing date: 2020-10-20
Publication date: 2022-04-28
Also published as: CN113491122A

Abstract

The present disclosure provides a video processing method, apparatus, and device, an unmanned aerial vehicle, and an image transmission system, which are applied to a video processing device comprising a first encoder and a second encoder. The method comprises: obtaining a first video frame generated by a first encoder and a second video frame generated by a second encoder; and writing the first video frame to a main video file, writing the second video frame to a proxy file, and synchronizing the first video frame in the main video file with the second video frame in the proxy file, wherein the proxy file allows a user to preview and select a video segment to be processed, before a video display device processes the main video file.

Description

Video processing method, device and equipment, unmanned aerial vehicle, image transmission system

technical field

The present disclosure relates to the technical field of video processing, and in particular, to a video processing method, device and device, unmanned aerial vehicle, and image transmission system.

Background technique

The proxy file is a video file that is basically synchronized with the main video file but has a lower image quality. It occupies a small space and requires less hardware when decoding and playing. By using the proxy file, the time-consuming to download the original video can be effectively reduced and the load on the playback hardware can be reduced. For example, when using editing software for video editing, you can preview, crop, and add special effects to the proxy file first, and then call the main video file for rendering. The proxy file needs to be synchronized with the main video file to avoid the inconsistency between the proxy file previewed by the user and the main video content. The traditional way of synchronizing proxy files generally uses an additional device outside the video capture device to downsample the main video file to obtain the proxy file. Since the downsampling method takes a long time, the process of generating the proxy file is less efficient.

SUMMARY OF THE INVENTION

The present disclosure provides a video processing method, device and equipment, an unmanned aerial vehicle, and a picture transmission system, which can improve the efficiency of generating proxy files.

In a first aspect, an embodiment of the present disclosure provides a video processing method, which is applied to a video processing device including a first encoder and a second encoder, the method comprising: acquiring a first video frame generated by the first encoder and the second video frame generated by the second encoder; the first video frame is written into the main video file, the second video frame is written into the proxy file, and the first video frame in the main video file is written. The video frame is synchronized with the second video frame in the proxy file, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.

In a second aspect, an embodiment of the present disclosure provides a video processing apparatus, including a processor, wherein the processor is configured to perform the following steps: acquiring a first video frame generated by the first encoder and the second video frame a second video frame generated by an encoder; writing the first video frame into a main video file, writing the second video frame into a proxy file, and combining the first video frame in the main video file with the The second video frame in the proxy file is synchronized, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.

In a third aspect, embodiments of the present disclosure provide a video processing device, including a first encoder, a second encoder, and a processor; the first encoder is configured to acquire a video code stream, and generate a first video code stream according to the video code stream. video frame, and send the first video frame to the processor; the second encoder is configured to obtain the video code stream, generate a second video frame according to the video code stream, and convert the second video frame The video frame is sent to the processor; the processor is configured to perform the following operations: obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder; The video frame is written into the main video file, the second video frame is written into the proxy file, and the first video frame in the main video file is synchronized with the second video frame in the proxy file, wherein all the The proxy file is used for the user to preview and select the video segment to be processed before the main video file is processed by the video display device.

In a fourth aspect, an embodiment of the present disclosure provides an unmanned aerial vehicle, comprising: a body; a camera disposed on the body for capturing video streams; The video processing device described above.

In a fifth aspect, an embodiment of the present disclosure provides an image transmission system, the image transmission system includes: a pan-tilt camera, and the pan-tilt camera includes: a housing; a pan-tilt mounted on the housing; The camera on the PTZ is used for collecting video streams; and the video processing device according to any embodiment of the present disclosure is provided in the casing.

In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to execute the video processing method described in any of the embodiments of the present disclosure.

In the embodiment of the present disclosure, the first video frame and the second video frame are generated in parallel by the two-channel encoder in the video processing device, the main video file is generated based on the first video frame, and the second video frame is generated synchronously based on the second video frame. proxy file, so that after the video processing device collects the video stream, the proxy file can be generated inside the video processing device, which ensures the generation efficiency of the proxy file and the synchronization between the proxy file and the main video file.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

FIG. 1 is a schematic diagram of a traditional way of generating a proxy file.

FIG. 2 is a flowchart of a video processing method according to an embodiment of the present disclosure.

FIG. 3 is a schematic diagram of a generation method of a proxy file according to an embodiment of the present disclosure.

FIG. 4 is a schematic diagram of a synchronization process when an encoder is started according to an embodiment of the present disclosure.

FIG. 5 is a schematic diagram illustrating the flow of video frames of the second encoder during a normal recording process according to an embodiment of the present disclosure.

FIG. 6 is a schematic diagram of a synchronization process in a pause and resume process according to an embodiment of the present disclosure.

FIG. 7 is a schematic diagram of a synchronization process in a file splitting process according to an embodiment of the present disclosure.

FIG. 8 is a schematic diagram of an audio and video synchronization manner according to an embodiment of the present disclosure.

FIG. 9 is a schematic diagram of a video processing apparatus according to an embodiment of the present disclosure.

FIG. 10 is a schematic diagram of a video processing device according to an embodiment of the present disclosure.

11A and 11B are schematic diagrams of drones of embodiments of the present disclosure.

FIG. 12 is a schematic diagram of an image transmission system according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

Most of the traditional proxy files use external devices such as workstations, mobile phones, etc. to obtain the main video file, and then convert the main video file into a low-quality proxy file through image quality compression (such as sampling below). As shown in Figure 1, it is a traditional way of generating proxy files. First, the image sensor 101 a in the camera 101 captures a video stream, and then the encoder 101 b in the camera 101 generates a video frame based on the video stream, and the video frame is written into the main video file 103 . At the same time, the camera 101 transmits the main video file 103 to the mobile phone 102 in real time, and the mobile phone 102 downsamples the received main video file to generate a proxy file 104 . In the above process, since the downsampling process takes more time, the efficiency of generating the proxy file is low. In addition, although there are some solutions that can generate proxy files in the video capture device, they need to unify the main parameters of the proxy file and the main video file, such as bit rate, frame rate (Frame Per Second, FPS), and group of pictures (Group). of Pictures, GOP), etc., it is difficult to meet the needs of different application scenarios.

Based on this, an embodiment of the present disclosure provides a video processing method, as shown in FIG. 2 , the method may include:

Step 201: Obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder;

Step 202: Write the first video frame into a main video file, write the second video frame into a proxy file, and write the first video frame in the main video file with the second video frame in the proxy file. The video frames are synchronized, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.

In the embodiment of the present disclosure, the main video file refers to the original video file generated based on the video code stream collected by the image sensor. The main video file generally has high image quality, but occupies a large storage space, and requires more hardware during decoding and playback. high. A proxy file is a video file with roughly the same content as the main video file, but with lower quality. Compared with the main video file, the proxy file occupies less storage space and requires less hardware when decoding and playing.

The video processing method of the embodiment of the present disclosure may be executed by a processor. The processor may be a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and other types. The processor may be a single processor or a processor cluster including multiple processors.

The video processing device in the embodiment of the present disclosure may be a device with a video capture function, for example, a camera. In step 201, a video code stream may be collected through an image sensor in a video processing device. The first encoder and the second encoder can obtain the original video code stream, and respectively generate the first video frame and the second video frame according to the original video code stream. Wherein, the original video code stream obtained by the first encoder and the second encoder may be the video code stream collected by the same image sensor, so as to reduce the hardware cost.

The first encoder and the second encoder may be encoders internal to the video processing device. By arranging two encoders inside the video processing device, after the video processing device captures the video stream, the proxy file can be generated inside the video processing device, and no additional device is needed to generate the proxy file. After the first encoder and the second encoder generate corresponding video frames, the video frames may be sent to the processor.

In step 202, the processor may invoke a write process to write the first video frame to the main video file and the second video frame to the proxy file, a process referred to as video recording. The recorded main video file and proxy file can be stored in a storage unit (eg, SD card) of the video processing device.

During the video recording process, the first video frame in the main video file needs to be synchronized with the second video frame in the proxy file. It should be noted that the synchronization here does not mean absolute synchronization, but basic synchronization within a certain error. The certain error may be less than or equal to the duration corresponding to one video frame. That is to say, when the time difference between the first video frame in the main video file and the second video frame in the proxy file is less than the duration corresponding to one video frame, it can be regarded as the first video frame and the second video frame Frames are synchronized. When the frame rates of the main video file and the proxy file are different, the duration corresponding to one video frame may be the duration corresponding to one video frame in the file with a higher frame rate.

The proxy file of the present application can be used on a video display device for a user to preview and select video clips to be processed. Specifically, it can be used to implement two functions of segment download and proxy editing. Among them, the segmented download function means that the camera body stores the main video file, and the video display device downloads the proxy file. The user selects the video clip to be downloaded in the proxy file, and then requests the camera side to transmit the main video file with a higher resolution. High raw footage. By adopting the segment download function, only a part of the main video file is downloaded, and the time for downloading the complete main video file and the time for decoding the complete main video file can be saved. The proxy editing function means that both the proxy file and the original file are completely downloaded to the video display device. The preview, editing, and cropping operations performed by the user on the video display device are all performed for the proxy file. Only the corresponding video clips in the main video file are processed. By using the proxy editing function, the time required to decode the complete main video file can be saved.

When performing segment download, the processor can read the proxy file from the storage space of the video processing device in response to receiving the video preview instruction, and send the read proxy file to the video display device for decoding and display, and obtain the decoded proxy file. The post video image for the user to preview. Wherein, the video preview instruction may be sent by the user to the processor through an interactive control (eg, button, touch screen) on the video processing device, or sent by the user through the video display device, or may also be sent in other ways sent to the processor. After receiving the selection request for the first video segment in the video image sent by the video display device, the selection request in the main video file corresponding to the first video segment may be responsive to the selection request. The second video segment is sent to the video display device for processing.

When a user needs to download a video file, a download request can be sent to the processor through the video display device. The download request may carry identification information of the first video segment in the proxy file. In response to the download request, the processor may send the second video segment in the main video file that is synchronized with the first video segment to the video display device, which is called segmented download. Wherein, the second video clip includes a first video clip, and the duration of the second video clip may be slightly longer than the first video clip.

An application scenario is: the video processing device transmits the captured real-time image to the video playback device, such as a mobile phone for preview display, and the user can trigger the video processing device to record the video through the preview image. During the recording process, the video processing device will separately store For the main video file and the proxy file, when the recording is over, if the user wants to play the recorded video through the video display device, the video processing device can first transmit the proxy file to the video display device for playback, saving transmission overhead and time. The video display device decodes the received proxy file, and then plays the video image corresponding to the second video frame, and the user determines the video segment to be edited by watching the video played by the video display device. After the user selects the video clip to be edited from the playing video screen, the video display device sends the time stamp information corresponding to the video clip to the video processing device, and the video processing device searches the main video file based on the time stamp information. The corresponding file data is sent to the video display device, and after the video display device receives and decodes this part of the data, it performs the post-editing process.

When performing proxy editing, the processor may send the main video file and the proxy file to the video display device respectively, wherein the proxy file is used for the user to preview and edit the video display device on the video display device. The display device may perform the first processing on the first video clip in the proxy file based on the user operation, and perform the first processing on the second video clip corresponding to the first video clip in the main video file based on the first processing. The second process is to obtain the processing result of the high-resolution video frames in the main video file.

Referring to FIG. 3 , in a practical application scenario, a first encoder 301 and a second encoder 302 may be set in the camera 300 , and the first encoder 301 and the second encoder 302 respectively receive video streams from the image sensor 303 , and generate the first video frame and the second video frame respectively. The first video frame is used to generate the main video file, and the second video frame is used to generate the proxy file.

In the embodiment of the present disclosure, the video parameters of the main video file and the video parameters of the proxy file may be different. The video parameters may include, but are not limited to, at least one of resolution, bit rate, frame rate, and group of pictures. The video parameters of the proxy file can be set according to actual needs. For example, it can be set according to at least one of user preferences, hardware resources, application scenarios and other conditions. For example, in a situation where a user tends to preview a proxy file with higher quality, or in a scenario where high-precision processing of a video file is required, the resolution of the proxy file can be set higher. For another example, when the transmission bandwidth is limited, the code rate of the proxy file may be set lower.

Optionally, the code rate of the proxy file can be set to an integral multiple of the code rate of the main video file, or the code rate of the main video file can be set to an integral multiple of the code rate of the proxy file. For example, the frame rate of the main video file is 60fps and the frame rate of the proxy file is 30fps; or the frame rate of the main video file is 30fps and the frame rate of the proxy file is 60fps. In this way, the synchronization accuracy of the second video frame in the proxy file and the first video frame in the main video file can be improved.

The manner of synchronizing the first video frame and the second video frame will be described in detail below. In the embodiment of the present disclosure, in the case of starting to record a video file, or in the case of resuming the recording after pausing the recording, or in the case of file splitting, the instant decoding refresh (Instantaneous Decoding Refresh) sent by the first encoder may be used. , IDR) frame and the IDR frame sent by the second encoder to synchronize the first video frame and the second video frame. The role of an IDR frame is to clear the reference frame queue, so as to block the reference of the subsequent video frame of the IDR frame to the video frame before the IDR frame during video decoding. Specifically, in the case of starting to record a video file, or in the case of resuming recording after pausing the recording, or in the case of file splitting, if the first IDR frame sent by the first encoder and the For the second IDR frame sent by the second encoder, the first video frame (including the first IDR frame) sent by the first encoder is synchronized with the second video frame (including the second IDR frame) sent by the second encoder to the corresponding video file.

During the startup process of the first encoder and the second encoder, if the startup of the first encoder and the second encoder are synchronized, if the first IDR sent by the first encoder is simultaneously received frame and the second IDR frame sent by the second encoder, the first video frame including the first IDR frame can be directly written into the main video file, and the second video frame including the second IDR frame can be written Write the proxy file. Simultaneous here does not mean the same moment, but means that the time interval is less than a certain threshold, for example, less than the duration of one video frame.

In the case that the start of the first encoder and the second encoder are not synchronized, if only the first IDR frame or the second IDR frame is received, the first IDR frame or the second IDR frame is sent to the The encoder of two IDR frames cyclically sends IDR requests until the IDR frame and the second IDR frame are received at the same time, then the first video frame including the first IDR frame is written into the main video file, and the first video frame including the first IDR frame A second video frame of the second IDR frame is written to the proxy file.

As shown in FIG. 4 , it is a schematic diagram of a synchronization process of a first video frame and a second video frame in the case that the startup of the first encoder and the second encoder are not synchronized. Wherein, the start-up asynchronous may be that the start-up durations of the first encoder and the second encoder are different, or the start-up time of the first encoder and the second encoder may be different. Figure 4 shows the case where the first encoder starts faster and the second encoder starts slower.

The first encoder generates a first video frame when the startup is completed. The first video frame of the first frame generated by the first encoder is the first IDR frame. The video frame generated after the first IDR frame may be a B frame or a P frame, as shown by a blank white square in the figure. After generating the first IDR frame, the first encoder may send the first IDR frame to the processor. At this time, the processor can determine whether the second IDR frame sent by the second encoder is received. If it is not received, the IDR request is sent again to the encoder that starts faster (ie, the first encoder), and the first encoder will generate the first IDR frame again when the IDR request is received. The processor does not start video recording until the processor receives the first IDR frame and the second IDR frame synchronously, that is, writes the first video frame including the first IDR frame into the main video file, and synchronously records the first IDR frame. The second video frame, including the two IDR frames, is written to the proxy file. The first video frame and the second video frame written into the corresponding video file are shown as gray squares in the dotted box in the figure. The first video frame received before the first IDR frame and the second IDR frame are received synchronously may be discarded, and the discarded video frame is shown as a white square outside the dotted box in the figure. The video frame after the IDR frame in the dashed box may be a B frame or a P frame.

Whether the first encoder and the second encoder are started synchronously is limited by hardware conditions. The synchronization mechanism during the startup of the encoder can be preset according to the hardware conditions at the factory. If the hardware conditions support the simultaneous startup of the first encoder and the second encoder, the video recording will be performed directly after startup; if the hardware conditions do not support the first encoder and the second encoder. When the encoder and the second encoder are activated at the same time, video recording is performed in the manner shown in FIG. 4 . In practical applications, it may also happen that the first encoder starts slowly and the second encoder starts faster. This situation is similar in principle to the situation shown in FIG. 4 , and can be handled with reference to the situation shown in FIG. 4 , which will not be repeated here.

Because the frame rate of the proxy file and the main video file may be different, and there may be a situation in which the frame output time of the two encoders is not synchronized. Therefore, as shown in FIG. 5 , during the video recording process, the first video frame can be directly written into the main video file, and the second video frame can be written into the buffer first. In the case that the timestamp of the second video frame in the cache satisfies the time condition for writing the proxy file, the second video frame is then written into the proxy file from the cache. Specifically, the second video frame may be written into the cache, and when a frame of the first video frame is acquired, the first timestamp corresponding to the first video frame is detected, and the second video frame in the cache is If the time interval between the second time stamp of the frame and the first time stamp is less than a preset value, write the second video frame in the buffer into the proxy file.

For example, if the first time stamp of the first video frame f1 currently received by the processor is t1, the second time stamp t2 of the second video frame in the buffer is compared with t1. If the difference between t2 and t1 is smaller than the preset Δt, write the second video frame in the buffer into the proxy file. If the difference between t2 and t1 is greater than or equal to Δt, the second video frame continues to be stored in the buffer.

In addition to generating the proxy file, the second video frame generated by the second encoder can also be used for real-time transmission to the video display device. The second video frame transmitted in real time can be directly transmitted to the video display device through the communication link without being affected by the recording process.

In some embodiments, the user may pause the video recording process. Since the video parameters of the proxy file may be different from the video parameters of the main video file, the first video frame and the second video frame may lose synchronization during the pause process, so that the proxy file cannot play its due role. In order to solve the above problem, in the case of video recording pause, it may be determined whether to write the second video frame into a proxy file based on whether the second video frame encoded by the second encoder is used for real-time image transmission, to keep the synchronization of the first video frame and the second video frame.

Specifically, if the second video frame is not used for real-time image transmission, both the first encoder and the second encoder may enter a pause state. In this case, a pause instruction may be sent to the first encoder and the second encoder synchronously, so that the first encoder and the second encoder synchronously pause to generate corresponding video frames.

When the video recording is paused, if the duration of the main video file is greater than the duration of the proxy file, the writing of the first video frame to the main video file can be suspended, and the second video frame can be continued to be written to the proxy file. video frames until the duration of the main video file and the proxy file are equal. The second video frame can be written into the cache first, and when the first video frame is received, the time stamp corresponding to the first video frame is detected. If the time stamp corresponding to the second video frame is smaller than the time stamp corresponding to the first video frame, it means that the main The duration of the video file is longer than that of the proxy file, so that at least one second video frame in the cache can be supplemented into the proxy file to fill the duration of the proxy file to a length substantially equal to that of the main video file. The basic equality may be that the difference between the durations of the main video file and the proxy file is less than the duration of one video frame.

As shown in Figure 6, after the user sends the pause command, the first encoder enters the pause state after 2 frames of the first video frame (as shown by the two gray squares between pause and resume in the figure), and then pauses the generation of first video frame. Since the main video file has two more frames than the proxy file after the pause, the second video frame continues to be read from the cache for two frames and the proxy file is added. The gray squares in the figure represent the video frames written to the proxy file or the main video file, and the white squares represent no video frames are generated.

In another case, if the time stamp of the second video frame in the buffer is greater than the time stamp of the first video frame of the last frame received after the pause state, all the second video frames in the buffer will be Writing the proxy file will make the proxy file longer than the main video file. Therefore, at least one second video frame in the cache may be discarded, and the discarded second video frame will not be written into the proxy file, so as to keep the duration of the proxy file and the main video file substantially equal.

In the case where the second video frame is used for real-time image transmission, the synchronization process in resuming video recording is similar to the startup process, that is, the IDR request is cyclically sent to the encoder that was resumed first, and the first encoder is received at the same time. The video recording is started in the case of the IDR frame of the second encoder and the IDR frame of the second encoder. For the specific process, please refer to the synchronization method of the encoder startup process, which will not be repeated here.

If the second video frame is used for real-time image transmission, the second group of encoders cannot be paused or stopped, and will always keep working because real-time image transmission (referred to as image transmission for short) is not interrupted. The encoder may send a pause instruction to the first encoder, so that the first encoder pauses to generate the first video frame, and continues to write the second video frame to the proxy file, when the duration of the proxy file is Stop writing the second video frame to the proxy file when the duration is equal to the duration of the main video file. The manner in which the duration of continuing to write the second video frame to the proxy file is equal to the duration of the main video file is similar to the manner shown in FIG. 6 , and details are not repeated here. It should be noted that the equality here does not mean absolute equality, but may also be basic equality.

In the case of resuming video recording, an IDR request may be sent to the second encoder, and a start instruction may be sent to the first encoder, to write the first video frame generated after the first encoder resumes startup. the main video file, and write into the proxy file a second video frame starting from the last IDR frame generated by the second encoder before the first encoder resumes startup. As shown in FIG. 6 , after resuming video recording, the processor first requests the second encoder to send an IDR frame, and determines whether the first encoder is restarted after receiving the IDR frame sent by the second encoder. If the first encoder is not started, the second encoder is requested again to send the IDR frame until the first encoder is restarted (R in the figure indicates that the first encoder is restarted). The processor may write the first video frame to the main video file starting from the first video frame of the first frame sent after the first encoder restarts, and start from the last IDR frame sent by the second encoder to write the second video frame. Video frames are written to proxy files.

In some embodiments, in order to prevent the file system from being unsupported or the recording duration being too long due to the excessively large file size, it is necessary to perform file splitting before the video file reaches a critical value. In order to ensure that in the main video file and proxy file before and after splitting, both the first video frame and the second video frame can maintain synchronization, when the main video file reaches the video splitting condition, the An encoder and the second encoder send an IDR request, if the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is greater than the time when the IDR frame sent by the second encoder is received Set the value to send an IDR request to the first encoder and the second encoder again.

Only when the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is not greater than the second preset value, the The main video file and the proxy file are split to write the first video frame and the second video frame into the split main video file and the split proxy file, respectively.

The video splitting condition may be that the duration or data volume of the main video file reaches a critical value. For example, when the data volume of the main video file reaches 3.8 Gbit, it is considered that the splitting condition is satisfied. As shown in Figure 7, during the splitting process, IDR requests will be sent to the first encoder and the second encoder respectively, and only when the IDR frames returned by the first encoder and the second encoder are received at the same time, The file is split, and the received first video frame and the second video frame are written into new files respectively. Otherwise, the first video frame and the second video frame are still written into the original file. The gray video frame in the figure represents the second video frame written into the original proxy file or the first video frame written into the original main video file, and the video frame in the dotted box represents the second video frame written into the new proxy file or the first video frame written into the original main video file. The first video frame of the new main video file.

Further, since the video parameters of the main video file and the proxy file are different, when one of the main video file and the proxy file satisfies the splitting condition, the other may not yet satisfy the splitting condition. That is to say, the splitting process of the main video file and the proxy file is not synchronized, and the time when the main video file starts to be divided may be earlier than the time when the proxy file starts to be divided, or it may be later than the time when the proxy file starts to be divided. Taking the moment when the main video file starts to be split is earlier than the moment when the proxy file starts to be split, the processor may split the proxy file after the main video file starts to be split. After the main video file starts to be split, the processor may set the flag bit representing the split state as the first flag bit to indicate that at least one of the main video file and the proxy file has not been split. In the case that the splitting of the main video file and the proxy file is completed, the flag bit is set as the second flag bit to indicate that the splitting of the main video file and the proxy file is completed. The first flag bit and the second flag bit may be represented by a Boolean type constant (for example, "0" or "1"), respectively.

In some embodiments, each time a first video frame is written to the main video file, one audio frame may be synchronously written to the main video file. It should be noted that the traditional audio and video synchronization method is to synchronize the audio frame and the video frame first, and then write the synchronized audio frame and video frame into a file. However, in the method of the embodiment of the present disclosure, after the first video frame and the second video frame are written into the main video file and the proxy file respectively, the corresponding audio frames are written into the main video file and the proxy file respectively. Due to the difference in the audio and video synchronization mechanisms, the problem of out-of-sync audio frames and video frames may occur during the file splitting process. In order to solve this problem, the audio frames received in the process of splitting the main video file and the proxy file can be buffered, and after the splitting of the main video file and the proxy file is completed, The audio frame in the buffer is written into the proxy file synchronously with the corresponding second video frame.

As shown in FIG. 8 , when the audio frame can be written into the main video file, one copy of the audio frame is made to obtain two identical audio frames. One of the audio frames is written to the main video file, and the other audio frame is written to the proxy file depending on whether a file split is currently in progress. If file splitting is currently in progress, write the audio frame to the cache, otherwise, determine whether the split has just been completed. If the file has just been split, the audio frames in the buffer are read out until the buffer is emptied. Wherein, the splitting just completed may be that the time difference between the current time and the time when the splitting is completed is less than a preset time interval. If the split is not just completed, the audio frames are written to the proxy file in the order in which they are written to the buffer. The file splitting is completed here means that both the main video file and the proxy file are split. The splitting process of the main video file and the proxy file may not be synchronized. As long as either the main video file or the proxy file has not completed the file splitting, it is considered that the file splitting is not completed currently.

The present disclosure adopts the first encoder and the second encoder to respectively realize the recording of the main video file and the proxy file, and ensures that the first video frame and the second video frame in the main video file are basically synchronized, and realizes the internal synchronization of the video processing device. The proxy file is generated without using additional equipment, which reduces the process expense in the process of generating the proxy file. Both the generated proxy file and the main video file are saved in the storage medium of the camera. You can download the proxy file to preview and select the area of interest, and then download the corresponding video clip from the main video file, or use the proxy file to preview and edit, and then The editing result is applied to the main video file, which saves the processing time consumption of the video file and facilitates user editing.

In addition, the proxy file generation scheme in the present disclosure can ensure that pause recovery and file splitting do not affect the synchronization effect between the proxy file and the main video file, and the user's recording process is more free and convenient. At the same time, the video parameters of the proxy file can be different from the main video file, which relieves the traditional restriction that the proxy file needs to keep the same video parameters as the main video file. The definition of the proxy file can be improved by reducing the frame rate or increasing the picture group. , so as to improve the user experience.

An embodiment of the present disclosure further provides a video processing apparatus, including a processor, wherein the processor is configured to perform the following steps:

Obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder;

Writing the first video frame to a main video file, writing the second video frame to a proxy file, and performing a comparison between the first video frame in the main video file and the second video frame in the proxy file Synchronization, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.

In some embodiments, the main video file and the proxy file have different video parameters.

In some embodiments, the video parameters include at least one of the following: resolution, bit rate, frame rate, and group of pictures.

In some embodiments, the code rate of the main video file is an integral multiple of the code rate of the proxy file, or the code rate of the proxy file is an integral multiple of the code rate of the main video file.

In some embodiments, the processor is further configured to: control the second encoder to start synchronously with the first encoder, so that the first encoder and the second encoder synchronously generate the the first video frame and the second video frame.

In some embodiments, the processor is configured to: under the condition that the startup of the first encoder and the second encoder are synchronized, if the first IDR frame and the first IDR frame sent by the first encoder are received simultaneously For the second IDR frame sent by the second encoder, write the first video frame including the first IDR frame into the main video file, and write the second video frame including the second IDR frame into the proxy file ; Or, in the case that the startup of the first encoder and the second encoder is not synchronized, if only the first IDR frame or the second IDR frame is received, send the first IDR frame to the The encoder of the frame or the second IDR frame cyclically sends the IDR request until the IDR frame and the second IDR frame are received at the same time, then the first video frame including the first IDR frame is written into the main video file, and writing a second video frame including the second IDR frame into the proxy file.

In some embodiments, the processor is configured to: write the second video frame into a cache; when a frame of the first video frame is acquired, detect a first timestamp corresponding to the first video frame; When the time interval between the second time stamp of the second video frame in the cache and the first time stamp is less than a preset value, write the second video frame in the cache to the proxy file .

In some embodiments, the processor is configured to: determine whether to convert the second video frame based on whether the second video frame encoded by the second encoder is used for real-time image transmission if the video recording is paused Frames are written to the proxy file.

In some embodiments, the processor is further configured to: if the second video frame is not used for real-time image transmission, send a pause instruction to the first encoder and the second encoder synchronously, to synchronously pause the generation of corresponding video frames by the first encoder and the second encoder.

In some embodiments, the processor is further configured to: in the case of paused video recording, if the duration of the main video file is greater than the duration of the proxy file, suspend writing the first video to the main video file frame, and continue to write second video frames to the proxy file until the duration of the main video file and the proxy file are equal.

In some embodiments, the processor is further configured to: if the second video frame is used for real-time image transmission, send a pause instruction to the first encoder, so as to cause the first encoder to pause the generation of the first encoder a video frame, and continue to write a second video frame to the proxy file, and stop writing the second video frame to the proxy file when the duration of the proxy file is equal to the duration of the main video file.

In some embodiments, the processor is configured to: in the case of resuming video recording, send an IDR request to the second encoder, and send a start instruction to the first encoder; Write the first video frame generated after the recovery start into the main video file, and write the second video frame starting from the last IDR frame generated by the second encoder before the first encoder is restarted the proxy file.

In some embodiments, the processor is further configured to send an IDR request to the first encoder and the second encoder respectively when the main video file reaches a video splitting condition; if receiving The time difference between the time when the IDR frame sent by the first encoder and the time when the IDR frame sent by the second encoder is received is greater than the second preset value, and the first encoder and the second The encoder sends an IDR request.

In some embodiments, the processor is further configured to: if the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is not greater than the time difference The second preset value is to split the main video file and the proxy file, so as to write the first video frame and the second video frame into the split main video file and the split main video file respectively. proxy file.

In some embodiments, the moment when the main video file starts to be split is not synchronized with the moment when the proxy file starts to be split.

In some embodiments, the processor is further configured to: synchronously write one audio frame into the main video file every time one frame of the first video frame is written to the main video file.

In some embodiments, the processor is further configured to: buffer audio frames received in the process of splitting the main video file and the proxy file; After the proxy file is split, the audio frame in the cache and the corresponding second video frame are synchronously written into the proxy file.

In some embodiments, the processor is further configured to: generate a flag bit according to the current split state, where the flag bit is used to indicate whether the split of the main video file and the proxy file is completed.

In some embodiments, the processor is further configured to: send the proxy file to the video display device for decoding and display, to obtain a decoded video image; receive the video image sent by the video display device for the video image A selection request of the first video clip in the main video file; in response to the selection request, sending a second video clip corresponding to the first video clip in the main video file to the video display device for processing.

In some embodiments, the processor is further configured to: send the main video file and the proxy file to the video display device respectively, so that the video display device obtains information about the first video in the proxy file A first process is performed on a video segment, and a second process is performed on a second video segment in the main video file corresponding to the first video segment based on the first process.

For details of the method executed by the processor in the embodiment of the present disclosure, refer to the foregoing embodiments of the video processing method, and details are not described herein again.

FIG. 9 shows a schematic diagram of the hardware structure of a more specific video processing apparatus provided by an embodiment of this specification. The apparatus may include: a processor 901 , a memory 902 , an input/output interface 903 , a communication interface 904 and a bus 905 . The processor 901 , the memory 902 , the input/output interface 903 and the communication interface 904 realize the communication connection between each other within the device through the bus 905 . When the clock synchronization apparatus is used to execute the above method applied to the first subsystem, the processor 901 is a first processor, and the communication interface 904 is a first communication interface. When the clock synchronization apparatus is used to execute the above method applied to the second subsystem, the processor 901 is a second processor, and the communication interface 904 is a second communication interface.

The processor 901 can be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. program to implement the technical solutions provided by the embodiments of this specification.

The memory 902 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like. The memory 902 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902 and invoked by the processor 901 for execution.

The input/output interface 903 is used for connecting input/output modules to realize information input and output. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 904 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, mobile network, WIFI, Bluetooth, etc.).

Bus 905 includes a path to transfer information between the various components of the device (eg, processor 901, memory 902, input/output interface 903, and communication interface 904).

It should be noted that although the above device only shows the processor 901, the memory 902, the input/output interface 903, the communication interface 904 and the bus 905, in the specific implementation process, the device may also include the necessary components for normal operation. other components. In addition, those skilled in the art can understand that, the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.

As shown in FIG. 10 , an embodiment of the present disclosure further provides a video processing device 1000, including a first encoder 1001, a second encoder 1002, and a processor 1003;

The first encoder 1001 is configured to obtain a video code stream, generate a first video frame according to the video code stream, and send the first video frame to the processor 1003;

The second encoder 1002 is configured to acquire the video code stream, generate a second video frame according to the video code stream, and send the second video frame to the processor 1003;

The processor 1003 is configured to perform the following operations:

Obtain the first video frame generated by the first encoder 1001 and the second video frame generated by the second encoder 1002;

For details of the method executed by the processor 1003 in the embodiment of the present disclosure, refer to the foregoing embodiments of the video processing method, and details are not described herein again.

As shown in FIG. 11A and FIG. 11B , an embodiment of the present disclosure further provides an unmanned aerial vehicle 1100, including:

body 1101;

a camera 1102 arranged on the body 1101 for capturing video streams;

and the video processing device 1000 disposed in the body 1101 .

The video stream collected by the camera 1102 can be used for real-time output to the terminal communicatively connected to the drone 1000 for the user to preview. The terminal may be a remote controller with a screen dedicated to the drone, or an intelligent terminal such as a mobile phone, a tablet computer, and a desktop computer. In some embodiments, the camera 1102 is disposed on the body 1101 , which may mean that the camera 1102 is directly disposed on the body 1101 , or is disposed on a pan/tilt connected to the body 1101 .

The UAV 1000 may further include blades 1104 disposed on the body, and a power system for providing power to the UAV 1000 . The embodiment of the video processing device 1000 used in the drone 1000 is the same as the embodiment of the video processing device 1000 described above, and details are not repeated here.

As shown in FIG. 12 , an embodiment of the present disclosure further provides an image transmission system 1200, including a pan-tilt camera 1201, and the pan-tilt camera 1201 includes:

housing 1201a;

a pan/tilt 1201b mounted on the housing 1201a;

a camera 1201c installed on the PTZ 1201b for capturing video streams; and

A video processing device (not shown in the figure) provided in the casing 1201a.

The embodiment of the video processing device used in the image transmission system 1200 is the same as the embodiment of the aforementioned video processing device 1000 , and details are not repeated here.

In some embodiments, the image transmission system 1200 may further include a video display device (eg, a display screen) 1202 communicatively connected to the video processing device. The video display device 1202 is configured to: receive the proxy file; display a video picture obtained after decoding the proxy file; and send a video for the first video segment in the proxy file to the video processing device. A download request; processing a second video segment corresponding to the first video segment in the main video file according to the selection request.

Optionally, the video display device 1202 is configured to send a download request for the second video clip to the video processing device according to the selection request; receive the second video clip sent by the video processing device; The second video segment is displayed, so as to implement a segmented download function.

Or optionally, the video display device 1202 is configured to receive the main video file sent by the video processing device; perform a clipping operation on the second video segment according to the selection request to generate a clipped video, thereby Implement proxy editing.

The video display device 1202 can be a portable mobile device such as a mobile phone and a tablet computer. The video display device 1202 and the PTZ camera 1201 can communicate via wired or wireless means such as a USB interface, Bluetooth, etc. The PTZ camera 1201 can convert the generated proxy file. The real-time image is transmitted to the video display device 1202 for preview, and the video display device 1202 can send instructions for functions such as pausing recording, resuming recording, downloading video files, etc. to the PTZ camera 1201 .

In some embodiments, a fixing bracket 1203 may also be installed on the housing 1201a of the PTZ camera 1201 for fixing the video display device 1202 to facilitate the user to view video files. The fixing bracket 1203 can be fixedly or detachably mounted on the housing 1201a.

The pan-tilt camera 1201 shown in FIG. 12 is only an example, and the pan-tilt camera 1201 may also include a display screen and the like.

Embodiments of the present disclosure also provide a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to perform the method described in any of the foregoing embodiments.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

From the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present specification can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions of the embodiments of this specification or the parts that make contributions to the prior art may be embodied in the form of software products, and the computer software products may be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments in this specification.

The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, e-mail device, game control desktop, tablet, wearable device, or a combination of any of these devices.

Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features, but due to space limitations, they are not described one by one, so the various technical features in the above embodiments can be combined arbitrarily It is also within the scope of this disclosure.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the disclosure and practice of the specification disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common general knowledge or techniques in the technical field not disclosed by this disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure. within the scope of protection.

Claims

A video processing method, characterized in that, applied to a video processing device comprising a first encoder and a second encoder, the method comprising:

Obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder;

Writing the first video frame to a main video file, writing the second video frame to a proxy file, and performing a comparison between the first video frame in the main video file and the second video frame in the proxy file Synchronization, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.
The method according to claim 1, wherein the video parameters of the main video file and the proxy file are different.
The method according to claim 2, wherein the video parameters include at least one of the following: resolution, bit rate, frame rate, and group of pictures.
The method according to claim 2, wherein the code rate of the main video file is an integer multiple of the code rate of the proxy file, or the code rate of the proxy file is the code rate of the main video file integer multiples of .
The method according to claim 1, wherein the method further comprises:

The second encoder is controlled to be activated in synchronization with the first encoder, so that the first encoder and the second encoder are synchronized to generate the first video frame and the second video frame.
The method according to claim 1, wherein the writing the first video frame into a main video file and writing the second video frame into a proxy file comprises:

In the case that the start-up of the first encoder and the second encoder are synchronized, if the first instant decoding refresh IDR frame sent by the first encoder and the first instant decoding refresh IDR frame sent by the second encoder are received simultaneously Two IDR frames, writing the first video frame including the first IDR frame into the main video file, and writing the second video frame including the second IDR frame into the proxy file; or,

In the case that the start of the first encoder and the second encoder are not synchronized, if only the first IDR frame or the second IDR frame is received, the first IDR frame or the second IDR frame is sent to the The encoder of two IDR frames cyclically sends IDR requests until the IDR frame and the second IDR frame are received at the same time, then the first video frame including the first IDR frame is written into the main video file, and the first video frame including the first IDR frame A second video frame of the second IDR frame is written to the proxy file.
The method according to claim 1, wherein the writing the second video frame into a proxy file comprises:

writing the second video frame into a cache;

When acquiring a frame of the first video frame, detecting the first timestamp corresponding to the first video frame;

In the case that the time interval between the second time stamp of the second video frame in the cache and the first time stamp is less than a preset value, write the second video frame in the cache into the proxy file.
The method according to claim 1, wherein the writing the second video frame into a proxy file comprises:

In the case where the video recording is paused, it is determined whether to write the second video frame to the proxy file based on whether the second video frame encoded by the second encoder is used for real-time image transmission.
The method according to claim 8, wherein the method further comprises:

If the second video frame is not used for real-time image transmission, a pause command is sent to the first encoder and the second encoder synchronously, so that the first encoder and the second encoder synchronously pauses the generation of the corresponding video frame.
The method according to claim 9, wherein the method further comprises:

When the video recording is paused, if the duration of the main video file is greater than the duration of the proxy file, stop writing the first video frame to the main video file, and continue to write the second video to the proxy file frames until the main video file and the proxy file are equal in duration.
The method according to claim 8, wherein the method further comprises:

If the second video frame is used for real-time image transmission, send a pause instruction to the first encoder, so that the first encoder pauses to generate the first video frame, and continues to write the first video frame to the proxy file Two video frames, when the duration of the proxy file is equal to the duration of the main video file, the writing of the second video frame to the proxy file is stopped.
The method according to claim 8, wherein the writing the first video frame into a main video file and writing the second video frame into a proxy file comprises:

In the case of resuming video recording, sending an IDR request to the second encoder, and sending a start instruction to the first encoder;

Write the first video frame generated after the first encoder is restarted into the main video file, and before the first encoder is restarted, the last frame IDR frame generated by the second encoder starts The second video frame is written to the proxy file.
The method according to claim 1, wherein the method further comprises:

When the main video file reaches the video splitting condition, send an IDR request to the first encoder and the second encoder respectively;

If the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is greater than the second preset value, the first encoder and the The second encoder sends an IDR request.
The method of claim 13, wherein the method further comprises:

If the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is not greater than the second preset value, the main video file and all The proxy file is split, so that the first video frame and the second video frame are written into the split main video file and the split proxy file, respectively.
The method according to claim 14, wherein the moment when the main video file starts to be split is not synchronized with the moment when the proxy file starts to be split.
The method according to claim 1, wherein the method further comprises:

Each time a first video frame is written to the main video file, one audio frame is synchronously written to the main video file.
The method of claim 16, wherein the method further comprises:

buffering the audio frames received in the process of splitting the main video file and the proxy file;

After the main video file and the proxy file are split, the audio frame in the buffer and the corresponding second video frame are synchronously written into the proxy file.
The method of claim 17, wherein the method further comprises:

A flag bit is generated according to the current split state, and the flag bit is used to indicate whether the split of the main video file and the proxy file is completed.
The method according to claim 1, wherein the method further comprises:

sending the proxy file to the video display device for decoding and display, and obtaining a decoded video image;

receiving a selection request for a first video segment in the video image sent by the video display device;

In response to the selection request, a second video segment in the main video file corresponding to the first video segment is sent to the video display device for processing.
The method according to claim 1, wherein the method further comprises:

Send the main video file and the proxy file to the video display device respectively, so that the video display device obtains the first processing performed on the first video clip in the proxy file, and based on the first process A process performs a second process on a second video segment corresponding to the first video segment in the main video file.
A video processing device, characterized in that it includes a processor, wherein the processor is configured to perform the following steps:

Obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder;

Writing the first video frame to a main video file, writing the second video frame to a proxy file, and performing a comparison between the first video frame in the main video file and the second video frame in the proxy file Synchronization, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.
The apparatus according to claim 21, wherein the video parameters of the main video file and the proxy file are different.
The apparatus according to claim 22, wherein the video parameters include at least one of the following: resolution, bit rate, frame rate, and group of pictures.
The apparatus according to claim 23, wherein the bit rate of the main video file is an integer multiple of the bit rate of the proxy file, or the bit rate of the proxy file is the bit rate of the main video file integer multiples of .
The apparatus of claim 21, wherein the processor is further configured to:

The second encoder is controlled to be activated in synchronization with the first encoder, so that the first encoder and the second encoder are synchronized to generate the first video frame and the second video frame.
The apparatus of claim 21, wherein the processor is configured to:

In the case that the startup of the first encoder and the second encoder are synchronized, if the first IDR frame sent by the first encoder and the second IDR frame sent by the second encoder are received simultaneously , write the first video frame including the first IDR frame into the main video file, and write the second video frame including the second IDR frame into the proxy file; or,

In the case that the start of the first encoder and the second encoder are not synchronized, if only the first IDR frame or the second IDR frame is received, the first IDR frame or the second IDR frame is sent to the The encoder of two IDR frames cyclically sends IDR requests until the IDR frame and the second IDR frame are received at the same time, then the first video frame including the first IDR frame is written into the main video file, and the first video frame including the first IDR frame A second video frame of the second IDR frame is written to the proxy file.
The apparatus of claim 21, wherein the processor is configured to:

writing the second video frame into a cache;

When acquiring a frame of the first video frame, detecting the first timestamp corresponding to the first video frame;

In the case that the time interval between the second time stamp of the second video frame in the cache and the first time stamp is less than a preset value, write the second video frame in the cache into the proxy file.
The apparatus of claim 21, wherein the processor is configured to:

In the case where the video recording is paused, it is determined whether to write the second video frame to the proxy file based on whether the second video frame encoded by the second encoder is used for real-time image transmission.
The apparatus of claim 28, wherein the processor is further configured to:

If the second video frame is not used for real-time image transmission, a pause command is sent to the first encoder and the second encoder synchronously, so that the first encoder and the second encoder synchronously pauses the generation of the corresponding video frame.
The apparatus of claim 29, wherein the processor is further configured to:

When the video recording is paused, if the duration of the main video file is greater than the duration of the proxy file, stop writing the first video frame to the main video file, and continue to write the second video to the proxy file frames until the main video file and the proxy file are equal in duration.
The apparatus of claim 28, wherein the processor is further configured to:

If the second video frame is used for real-time image transmission, send a pause instruction to the first encoder, so that the first encoder pauses to generate the first video frame, and continues to write the first video frame to the proxy file Two video frames, when the duration of the proxy file is equal to the duration of the main video file, the writing of the second video frame to the proxy file is stopped.
The apparatus of claim 28, wherein the processor is configured to:

In the case of resuming video recording, sending an IDR request to the second encoder, and sending a start instruction to the first encoder;

Write the first video frame generated after the first encoder is restarted into the main video file, and before the first encoder is restarted, the last frame IDR frame generated by the second encoder starts The second video frame is written to the proxy file.
The apparatus of claim 21, wherein the processor is further configured to:

When the main video file reaches the video splitting condition, send an IDR request to the first encoder and the second encoder respectively;

If the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is greater than the second preset value, the first encoder and the The second encoder sends an IDR request.
The apparatus of claim 33, wherein the processor is further configured to:

If the time difference between the time when the IDR frame sent by the first encoder is received and the time when the IDR frame sent by the second encoder is received is not greater than the second preset value, the main video file and all The proxy file is split, so that the first video frame and the second video frame are written into the split main video file and the split proxy file, respectively.
The apparatus according to claim 34, wherein the moment when the main video file starts to be split is not synchronized with the moment when the proxy file starts to be split.
The apparatus of claim 21, wherein the processor is further configured to:

Each time a first video frame is written into the main video file, an audio frame is written into the main video file synchronously.
The apparatus of claim 36, wherein the processor is further configured to:

buffering the audio frames received in the process of splitting the main video file and the proxy file;

After the main video file and the proxy file are split, the audio frame in the buffer and the corresponding second video frame are synchronously written into the proxy file.
The apparatus of claim 37, wherein the processor is further configured to:

A flag bit is generated according to the current split state, and the flag bit is used to indicate whether the split of the main video file and the proxy file is completed.
The apparatus of claim 21, wherein the processor is further configured to:

sending the proxy file to the video display device for decoding and display, and obtaining a decoded video image;

receiving a selection request for a first video segment in the video image sent by the video display device;

In response to the selection request, a second video segment in the main video file corresponding to the first video segment is sent to the video display device for processing.
The apparatus of claim 21, wherein the processor is further configured to:

Send the main video file and the proxy file to the video display device respectively, so that the video display device obtains the first processing performed on the first video clip in the proxy file, and based on the first process A process performs a second process on a second video segment corresponding to the first video segment in the main video file.
A video processing device, comprising a first encoder, a second encoder and a processor;

The first encoder is configured to obtain a video code stream, generate a first video frame according to the video code stream, and send the first video frame to the processor;

The second encoder is configured to obtain the video code stream, generate a second video frame according to the video code stream, and send the second video frame to the processor;

The processor is used to perform the following operations:

Obtain the first video frame generated by the first encoder and the second video frame generated by the second encoder;

Writing the first video frame to a main video file, writing the second video frame to a proxy file, and performing a comparison between the first video frame in the main video file and the second video frame in the proxy file Synchronization, wherein the proxy file is used for the user to preview and select the video segment to be processed before the video display device processes the main video file.
An unmanned aerial vehicle, characterized in that it includes:

body;

a camera set on the body for capturing video streams;

and the video processing device as claimed in claim 41 provided in the body.
An image transmission system, characterized in that the image transmission system comprises:

PTZ camera, the PTZ camera includes:

case;

a head mounted on the housing;

a camera installed on the PTZ for capturing video streams; and

The video processing apparatus of claim 41 provided in the housing.
The image transmission system according to claim 43, wherein the image transmission system further comprises:

a video display device communicatively connected to the video processing device;

The video display device is used for:

receiving said proxy document;

Decoding the proxy file and displaying the decoded video image;

receiving a user's selection request for the first video segment in the video image;

The second video segment corresponding to the first video segment in the main video file is processed according to the selection request.
The image transmission system according to claim 44, wherein the video display device is used for:

sending a download request for the second video segment to the video processing device according to the selection request;

receiving the second video segment sent by the video processing device;

The second video segment is displayed.
The image transmission system according to claim 44, wherein the video display device is further used for:

receiving the main video file sent by the video processing device;

The editing operation is performed on the second video segment according to the selection request to generate a edited video.
A computer-readable storage medium, comprising instructions, which, when executed on a computer, cause the computer to execute the video processing method according to any one of claims 1 to 20.