WO2021042341A1

WO2021042341A1 - Video display method, receiving end, system and storage medium

Info

Publication number: WO2021042341A1
Application number: PCT/CN2019/104597
Authority: WO
Inventors: 王晓东
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-05
Filing date: 2019-09-05
Publication date: 2021-03-11
Also published as: CN112154665A

Abstract

A video display method, a receiving end, an image transmission system and a storage medium. The method comprises: receiving encoded video data, and decoding the video data by means of a decoder (S201); determining whether partial data, output by the current frame, in the video data decoded by the decoder meets a pre-set condition (S202); and if so, generating a timing signal and sending the timing signal to a display controller, such that the display controller controls a display screen to display the current frame (S203).

Description

Video display method, receiving end, system and storage medium

Technical field

This application relates to the field of display technology, and in particular to a video display method, receiving end, image transmission system and storage medium.

Background technique

At present, in the image transmission system, in order to reduce the delay of the image transmission, the sub-frame level pipeline processing is generally adopted, and the relevant steps of the video stream processing are arranged in a streamlined manner, including the video acquisition part, the image processing part, and the video compression. The steps corresponding to the part, the video transmission part, the video decompression part, and the video display part (such as LCD display) require that the timing of each step can match each other to achieve. However, in a common image transmission system, the timing between some steps may not match. For example, the timing of the video frame received at the receiving end and the LCD display video frame may be asynchronous. The position of the video frame is uncertain, so the previous frame may be displayed or the next frame may be displayed. Therefore, it will cause end-to-end delay jitter. In order to overcome the delay jitter, frame buffer display is adopted, that is, a frame of data or multiple frames of data is buffered and then the screen is refreshed regularly. However, this will introduce a frame-level delay. And long-term timing drift will also cause the problem of repeated display or dropped frame display.

Summary of the invention

Based on this, the present application provides a video display method, receiving end, image transmission system, and storage medium to reduce display delay and solve the problem of repeated display and dropped frame display.

In the first aspect, this application provides a video display method, the method including:

Receiving encoded video data, and decoding the video data through a decoder;

Determining whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, where the preset condition is used to trigger the generation of a timing signal;

If part of the output data meets the preset condition, a timing signal is generated and sent to the display controller, so that the display controller controls the display screen to display the current frame.

In a second aspect, the present application also provides a receiving end, the receiving end including a memory and a processor;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Receiving encoded video data, and decoding the video data through a decoder;

In a third aspect, the present application also provides an image transmission system, the system includes a sending end and a receiving end, and the sending end and the receiving end are in communication connection;

The sending end obtains video data and encodes the video data;

The sending end sends the encoded video data to the receiving end;

The receiving end receives the encoded video data, and decodes the video data through a decoder;

Determining, by the receiving end, whether part of the data output by the current frame in the video data decoded by the decoder satisfies a preset condition, and the preset condition is used to trigger the generation of a timing signal;

If the receiving end determines that part of the output data meets the preset condition, it generates a timing signal and sends the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.

In a fourth aspect, the present application also provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor implements the above-mentioned video display method.

The embodiments of the present application provide a video display method, a receiving end, an image transmission system, and a storage medium. After receiving the encoded video data, the video data is decoded by a decoder; and it is determined that the decoder decodes the video data. Whether part of the data output by the current frame in the video data meets a preset condition; and when the preset condition is satisfied, a timing signal is generated and sent to the display controller, so that the display controller Controlling the display screen to display the current frame according to the timing signal. This display method reduces display delay, and is different from fixed-frequency refresh or frame buffer display methods, and solves the problem of repeated display or dropped frame display caused by long-term timing drift.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic block diagram of an image transmission system provided by an embodiment of the present application;

2 is a schematic flowchart of steps of a video display method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a display process of a video display method provided by an embodiment of the present application;

4 is a schematic flowchart of steps of a video display method provided by another embodiment of the present application;

FIG. 5 is a schematic block diagram of a sending end according to an embodiment of the present application;

Fig. 6 is a schematic block diagram of a receiving end according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1. FIG. 1 is a schematic block diagram of an image transmission system provided by the present application. The image transmission system in the embodiment of the present application will be described below with reference to FIG. 1.

As shown in Figure 1, the image transmission system 10 includes a sending end 11 and a receiving end 12. The sending end 11 is equipped with a shooting device 110. The shooting device 110 can collect video, and then send the collected video through wireless image transmission technology. To the receiving end 12. Of course, the video collected by the shooting device 110 can also be sent to the receiving end 12 by means of wired image transmission, which is not limited here.

The sending end 11 includes a movable device, the movable device may be, for example, a drone, an unmanned vehicle, a handheld pan/tilt, or a robot, etc. The movable device is equipped with a camera 110 for capturing video. For example, the drone in Figure 1 is equipped with a camera.

In some embodiments, the photographing device 110 may be, for example, a mobile phone or a camera, of course, it may also be other photographing equipment, such as a camera module.

It can be understood that the hand-held pan/tilt includes a pan/tilt system or an integrated pan/tilt camera on which the photographing device 110 is mounted. The pan/tilt system to which the camera 110 is mounted is a handheld pan/tilt mounted with a mobile phone or a handheld pan/tilt mounted with a camera.

The receiving end 12 may include a processor, a display, and a display controller. The display controller is electrically connected to the processor and connected to a display, and is used to control the display to display corresponding content, such as displaying a video screen.

Among them, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The processor may communicate with the display controller, for example, to generate a timing signal for triggering the display controller to control the display to display relevant content.

It can be understood that the display controller may be a part of the display, or may be separate from the display. The displays include, but are not limited to, LCD displays, LED displays, and OLED displays.

It is understandable that the display may be a part of the receiving end 12, for example, the receiving end 12 is a mobile phone, and the display is a display screen of the mobile phone; of course, it may also be an external display of the receiving end 12. Correspondingly, the display controller may also be a part of the receiving end 12 or a separate part from the receiving end 12.

In some embodiments, the receiving end 12 specifically includes, but is not limited to, a remote control, a mobile phone, or a tablet computer. Of course, electronic devices, such as VR glasses or other wearable devices, can also be used.

It should be noted that, in FIG. 1, the sending end 11 is an unmanned aerial vehicle and the receiving end 12 is a mobile phone as an example, but it constitutes a limitation on the sending end 11 and the receiving end 12.

In the image transmission system, in order to obtain end-to-end low latency, it is necessary to adopt sub-frame level pipeline processing, that is, to arrange the sequence of each step corresponding to video processing and transmission in a streamlined manner, including the video acquisition part, the image processing part, and the video Compression part, video transmission part, video decompression part, video display part. This requires the timing of each step to match each other.

However, in a common image transmission system, the timing between some steps may not match. For example, the video frame received at the receiving end 12 may be asynchronous with the frame timing displayed on the display. At the moment when the display is refreshed, the new video The arrival of the frame is uncertain, so the previous frame may be displayed, the next frame may be displayed, or the next frame may be skipped to display the next frame. This causes the end-to-end delay jitter, and the increase in delay caused by the buffer added to overcome the delay jitter.

In the image transmission system, the sending end 11 is used to collect video data, compress and encode the collected video data, and send the compressed and encoded video data to the receiving end 12, and the receiving end 12 decodes and displays the encoded video data. By encoding the video, the smoothness and transmission speed of video transmission are improved.

For example, the drone as shown in Figure 1 captures video data through the camera 110, compresses and encodes the captured video data, and sends the compressed and encoded video data to a mobile phone on the ground, and the user decodes and displays it through the mobile phone. Watch videos taken by drones in real time20.

Among them, the way of encoding the video data can be inter-frame coding or intra-frame coding, of course, other coding methods can also be used, such as a mixed coding method of inter-frame coding and intra-frame coding. Correspondingly, a decoding method corresponding to the encoding method is adopted to decode the encoded video data.

When encoding video data, the video images in the video data need to be divided into macroblocks, that is, each macroblock includes multiple pixels, such as 16×16 pixels as a macroblock.

Among them, one frame of video image can be encoded into one or more slices, and each slice contains an integer number of macroblocks. Among them, each slice has at least one macroblock, and at most it contains the macroblock of the entire image.

Among them, a slice group is a subset of several macroblocks in a coded image, including one or several slices.

The sending end 11 and the receiving end 12 encode and decode the video data, which are encoded and decoded by the encoder and decoder in the sending end 11 and the receiving end 12, respectively.

It can be understood that the image system in FIG. 1 and the naming of the components of the image system described above are only for identification purposes, and therefore do not limit the embodiments of the present application.

Hereinafter, based on the image transmission system in FIG. 1, the video display method provided by the embodiment of the present application will be introduced in detail.

Please refer to FIG. 2, which is a schematic flowchart of steps of a video display method provided by an embodiment of the present application. The video display method can be applied to the receiving end of the image system to reduce the display delay of the video, thereby solving the problem of repeated display or dropped frame display caused by long-term timing drift.

As shown in FIG. 2, the video display method includes steps S201 to S203.

S201: Receive encoded video data, and decode the video data through a decoder;

S202: Determine whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition;

S203: If part of the output data meets the preset condition, generate a timing signal and send the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.

After the transmitting end collects the video data, it compresses and encodes the collected video data. Specifically, the video data is encoded by an encoder using a corresponding encoding method, and the encoded video data is sent to the receiving end.

Wherein, the corresponding encoding method is, for example, an encoding method of intra-frame encoding or inter-frame encoding. Of course, other encoding methods are also included, for example, both inter-frame encoding and intra-frame encoding are used to encode video data.

After receiving the encoded video data, the receiving end decodes the video data through a decoder. The decoder uses a decoding method corresponding to the encoding method to decode the encoded video data.

In some embodiments, in order to improve decoding efficiency and reduce video display delay, decoding the video data is specifically: decoding the video data in a time less than one frame of video data by a decoder.

Video data includes multiple frames of video images, and each frame of video image corresponds to a corresponding time. For example, if 100 frames of video images are displayed in one millisecond, the time corresponding to each frame of video data is 0.1 second. Then, the video data is decoded in less than 0.01 milliseconds by the decoder.

It should be noted that the decoding of video data by the decoder includes two processes of demodulation, decoding and decompression. Decompression is to decompress the data output by the demodulation and decoding.

The preset condition is used to trigger the generation of a timing signal, and the timing signal is used to trigger the display controller to control the display screen to display the current frame.

When determining that the part of the output data meets the preset condition, the receiving end generates a timing signal and sends the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.

Thereby, a timing signal is generated by controlling and the timing signal is sent to the display controller, so that the display controller controls the display screen to display the current frame, thereby reducing the display delay.

In some embodiments, the preset condition includes: detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold; and/or determining the remaining amount of the current frame Whether the data can be transmitted to the display within a predetermined time.

The preset number threshold is used to determine the size of the decoded output data. Because each frame of the video image in the video data is divided into macroblocks during encoding, and at the same time, how many slices can be divided into the video image by macroblocks, each slice contains an integer number of macroblocks , And the slice group contains one or several slices. Therefore, the predetermined number threshold can be represented by macroblocks, slices, or slice groups.

The predetermined time is used to determine that the remaining data of the current frame can arrive at the display on time, so that the current frame can be fully displayed. The predetermined time may be determined based on empirical values, or may be determined based on the display parameters of the display, such as the refresh rate.

Exemplarily, the preset number threshold includes a preset number of macroblocks, for example, the preset number of macroblocks is specifically M macroblocks, and M is a positive integer.

Correspondingly, detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold is specifically: counting the number of macroblocks output by the decoder decoding the current frame, and determining the Whether the number of output macro blocks is greater than the preset number of macro blocks.

For example, when the number of macroblocks outputted by the decoder in decoding the current frame is greater than M, it is determined that part of the output data satisfies the preset condition to trigger the generation of a timing signal.

Exemplarily, if the encoded video data is encoded into multiple slices or slice groups, the preset number threshold includes the preset number of slices or the preset number of slice groups.

Correspondingly, detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, specifically: counting the number of slices or slice groups output by the decoder when decoding the current frame, and It is determined whether the number of output slices is greater than the preset number of slices, or whether the number of output slices is greater than the preset number of slices.

For example, when the number of slices output by the decoder in decoding the current frame is greater than the preset number of slices, it is determined that part of the output data satisfies the preset condition to trigger the generation of a timing signal.

For another example, when the number of slice groups output by the decoder in decoding the current frame is greater than the preset number of slice groups, it is determined that part of the output data satisfies the preset condition to trigger the generation of a timing signal.

In some embodiments, it is determined whether the remaining data of the current frame can be transmitted to the display within a predetermined time; if the remaining data of the current frame can be transmitted to the display within a predetermined time, it is determined that part of the data is output The preset condition is satisfied to trigger the generation of the timing signal.

Wherein, it is determined whether the remaining data of the current frame can be transmitted to the display within a predetermined time, and the determination method adopted is to determine whether the remaining data can be transmitted to the display according to the remaining data of the current frame reaching the arrival requirement time of the display It is transmitted to the display within a predetermined time.

In some embodiments, in order to quickly determine whether it can be transmitted to the display within a predetermined time. The determination method specifically includes:

Determine the arrival demand time of the remaining data of the current frame to the display, and determine whether the arrival demand time is less than or equal to a predetermined time; if the arrival demand time is less than or equal to the predetermined time, determine the current The remaining part of the frame data can be transmitted to the display within a predetermined time.

For example, if the predetermined time is 0.01 milliseconds, and it is determined that the remaining data of the current frame is transmitted to the display with a required time of 0.005 milliseconds, it can be determined that the remaining data of the current frame can be transmitted to the display within the predetermined time. monitor.

In some embodiments, determining the arrival required time for the remaining data of the current frame to be transmitted to the display is specifically: acquiring the number of macroblocks contained in the remaining data of the current frame; determining the output time of the macroblock And a delay time; and the arrival demand time is determined according to the number, output time, and delay time of the macroblocks.

Determine and obtain the number of macroblocks included in the remaining data, determine the output time and delay time of the macroblock, and thereby calculate the arrival demand time according to the number of macroblocks and the corresponding output time and delay time of the macroblock .

For example, if the number of macro blocks is 10, the corresponding output time of the macro block is 0.0001 milliseconds, and the corresponding delay time is 0.0004 milliseconds, then the calculated arrival demand time is 0.005 milliseconds.

It is understandable that the number of slices or slice groups, and the corresponding output time and delay time can also be used to calculate the arrival demand time.

In some embodiments, if the preset condition is: detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, and determining whether the remaining data of the current frame can be It is transmitted to the display within a predetermined time.

Correspondingly, the determining whether the partial data output by decoding satisfies a preset condition is specifically: if the partial data output is greater than the preset number threshold, and the remaining data can be transmitted to the display within a predetermined time, then It is determined that part of the output data satisfies the preset condition. This preset condition can effectively avoid frame-level delays, and at the same time solve the problems of repeated display and dropped frame display.

Exemplarily, as shown in FIG. 3, the receiving end receives encoded and compressed video data, where the video data includes multiple frames of video images, for example, the current frame is video image i. Decode the video data through a decoder; determine whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, and the preset condition is used to trigger the generation of a timing signal; Part of the data satisfies the preset condition, for example, when the current frame i is decoded in Figure 3, M macroblocks are output, that is, part of the output data satisfies the preset condition, a timing signal is generated and the timing signal is sent to the display controller , So that the display controller controls the display screen to display the current frame, and the current frame refers to the decoded video image I.

It should be noted that the decoding of the video data by the decoder includes two processes of demodulation, decoding and decompression. As shown in FIG. 3, the encoded and compressed video image i is first demodulated and decoded to output data. The output data is decompressed to obtain decoded video data.

It can be seen that the video display method realizes the adjustment of the frame timing of the display refreshing according to the output timing of the decoder, so that the refreshing is started immediately when the output data of the current frame meets the requirements, and the display delay of the video on the display is reduced, thereby Reduce the delay of the entire image transmission system. And the start time of the display is adjusted according to the time of data decoding and output, thereby avoiding the problem of frame dropping and frame skipping of the display caused by clock asynchronous. This improves the user's viewing experience.

In some embodiments, in order to better solve the display delay problem, a process of dynamically adjusting the preset number threshold is further included. The adjustment process is specifically: adjusting the size of the preset number threshold according to the stable state of the video data transmission.

Specifically, it is determined whether the transmission of the video data is stable; when the transmission of the video data is stable, the size of the preset number threshold is increased; and when the transmission of the video data is unstable, the value of the preset number threshold is decreased. size. In this way, the size of the preset number can be adjusted in real time according to the stable state of video transmission to ensure that the timing signal is triggered when necessary to complete the display according to the timing signal to reduce display delay.

In some embodiments, specifically, it is determined whether the transmission of the video data is stable; when the transmission of the video data is stable, the first preset number is determined as the preset number threshold; when the transmission of the video data is unstable At the time, the second preset number is determined as the preset number threshold. Wherein, the second preset number is greater than the first preset number.

For example, when the transmission of the video data is stable, the first preset number is determined as the preset macroblock data; when the transmission of the video data is unstable, the second preset number is determined as the preset macroblock data ; Wherein, the second preset number is greater than the first preset number.

When the data transmission of the image transmission system is stable enough, for each part of the data in each frame, the delay is fixed, so a smaller first preset number can be determined as the preset number of macroblocks For example, M macroblock rows correspond to 1/16 frame size.

In the image system, if the transmission is not stable enough, it is necessary to increase a certain buffer to resist the jitter of the transmission delay, and a larger second preset number can be determined as the preset number of macroblocks, for example, M macroblock lines correspond to 3 /4 frame size. Through the first preset number, the second preset number and their magnitude relationship, the jitter of the transmission delay can be effectively resisted, thereby more effectively reducing the display delay.

Wherein, determining whether the transmission of the video data is stable can be specifically adopted: monitoring whether the delay time of the video data transmission has changed; if the delay time of the video data transmission is fixed, it is determined that the transmission of the video data is stable If the delay time of the video data transmission changes, it is determined that the video data transmission is unstable.

For example, if the monitoring delay time is always 0.0004 milliseconds, it can be determined that the transmission of the video data is stable; if the monitoring delay time is constantly changing, such as the delay time is 0.0004 milliseconds, and the delay time is 0.0006 milliseconds, then it can be determined The transmission of the video data is unstable.

In some embodiments, the video display method further includes a step of determining a predetermined time, and the step is specifically: obtaining a refresh rate configured by the display, and determining the predetermined time according to the refresh rate. The refresh time of the display is determined according to the refresh frequency, and the refresh time is used as the predetermined time.

For example, different displays have different refresh rates. The higher the refresh rate, the better the stability of the corresponding displayed image (picture). The refresh rate of a certain display is 60Hz, and the corresponding refresh time of the display is about 0.01543 milliseconds. Then, 0.01543 milliseconds is regarded as the predetermined time.

It can be understood that, of course, a certain time greater than the refresh time may also be used as the predetermined time. For example, the refresh time corresponding to the display is about 0.01543 milliseconds, and 0.016 milliseconds can be selected as the predetermined time.

The predetermined time is determined according to the refresh rate, and the user can set different refresh rates for different types of displays or displays of the same type, and adjust the predetermined time in a timely manner. It can be determined that the display can display according to the timing signal, thereby further reducing the display delay.

Please refer to FIG. 4, which is a schematic flowchart of steps of a video display method provided by another embodiment of the present application. The video display method can be applied to a video transmission system to reduce the delay of video display, so as to solve the problem of repeated display or dropped frame display caused by long-term timing drift.

As shown in FIG. 4, the video display method includes steps S401 to S405.

S401: The sending end obtains video data and encodes the video data.

S402: The sending end sends the encoded video data to the receiving end.

S403: The receiving end receives the encoded video data, and decodes the video data through a decoder;

S404: The receiving end determines whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition;

S405. If the receiving end determines that the part of the output data meets the preset condition, it generates a timing signal and sends the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.

The sending end acquires the video data and encodes the video data after collecting the video data through the shooting device.

Specifically, acquiring video data in a time corresponding to less than one frame of video data, and encoding the video data in a time corresponding to less than one frame of video data;

Wherein, the sending end sends the encoded video data to the receiving end, specifically, sending the encoded video data to the receiving end in a time less than one frame of video data.

The receiving end decodes the video data, specifically: the decoder decodes the video data in a time less than one frame of video data.

This realizes the sub-frame level control of the video acquisition part, the image processing part, the video compression part, the video transmission part, the video decompression part, and the video display part. At the same time, it controls the generation of timing signals and sends the timing signals to the display controller, So that the display controller controls the display screen to display the current frame. This further reduces the display delay.

Please refer to FIG. 5, which is a schematic block diagram of a sending end according to an embodiment of the present application. The sending end 11 includes a photographing device 110, a processor 111, and a memory 112. The processor 111, the memory 112 and the photographing device 110 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 111 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 112 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 111 is configured to run a computer program stored in a memory, and implement the following steps when the computer program is executed:

Control the shooting device to collect video data; obtain the collected video data and encode the video data; and send the encoded video data to the receiving end.

In some embodiments, when the processor 111 realizes that the sending end obtains video data and encodes the video data, it specifically realizes:

The video data is acquired in a time less than one frame of video data, and the video data is encoded in a time less than one frame of video data.

In some embodiments, when the processor 111 implements the sending end to send the encoded video data to the receiving end, it specifically implements:

The encoded video data is sent to the receiving end in a time corresponding to less than one frame of video data.

Please refer to FIG. 6, which is a schematic block diagram of a receiving end according to an embodiment of the present application. The receiving end 12 includes a processor 121, a memory 122, a display controller 123, and a display 124. The processor 121, the memory 122 and the display controller 123 are connected by a bus, such as an I2C (Inter-integrated Circuit) bus, which controls the display. The display 123 and the display 124 are used to control the display 124 to display relevant content.

Specifically, the processor 121 may be a micro-controller unit (MCU), a central processing unit (CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 122 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 121 is configured to run a computer program stored in a memory, and implement the following steps when executing the computer program:

Receive encoded video data, decode the video data through a decoder; determine whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, and the preset condition is used for Trigger generating a timing signal; if part of the output data meets the preset condition, generate a timing signal and send the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.

In some embodiments, the preset condition includes: detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, and determining whether the remaining data of the current frame can be Transmit to the display within a predetermined time;

The determining whether the partial data output by the current frame in the video data decoded by the decoder meets a preset condition includes: if the partial data output is greater than the preset number threshold, and the remaining data can be It is transmitted to the display within a predetermined time, and it is determined that part of the output data meets the preset condition.

In some embodiments, the preset number threshold includes a preset number of macroblocks; the processor 121 is implementing the detection of whether part of the data output by the current frame in the video data decoded by the decoder is greater than When the number threshold is preset, realize:

Count the number of macroblocks output by the decoder in decoding the current frame, and determine whether the number of output macroblocks is greater than the preset number of macroblocks.

In some embodiments, the encoded video data is encoded into multiple slices or slice groups, and the preset number threshold includes a preset number of slices or a preset number of slice groups; the processor 121 is implementing all The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, to achieve:

Count the number of slices or slice groups output by the decoder in decoding the current frame, and determine whether the number of output slices is greater than the preset number of slices, or whether the number of output slices is greater than the preset number Number of film groups.

In some embodiments, the processor 121 further implements:

Adjust the size of the preset number threshold according to the stable state of the video data transmission.

In some embodiments, when the processor 121 realizes the adjustment of the size of the preset number threshold according to the stable state of the video data transmission, it realizes:

Determine whether the transmission of the video data is stable; if the transmission of the video data is stable, increase the size of the preset number threshold; if the transmission of the video data is unstable, reduce the size of the preset number threshold.

Determine whether the transmission of the video data is stable; if the transmission of the video data is stable, determine the first preset number as the preset number threshold; if the transmission of the video data is unstable, determine the second preset number as the preset The number threshold, wherein the second preset number is greater than the first preset number.

In some embodiments, when the processor 121 implements the determining whether the transmission of the video data is stable, implement:

Monitor whether the delay time of the video data transmission changes; if the delay time of the video data transmission is fixed, determine that the transmission of the video data is stable; if the delay time of the video data transmission changes, determine the The transmission of video data is unstable.

In some embodiments, when the processor 121 implements the determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time, the processor 121 implements:

In some embodiments, when the processor 121 realizes the determination of the arrival required time for the remaining data of the current frame to be transmitted to the display, it realizes:

Acquire the number of macroblocks included in the remaining data of the current frame; determine the output time and delay time of the macroblock; and determine the arrival required time according to the number, output time, and delay time of the macroblock.

In some embodiments, the processor further implements:

The refresh rate of the display configuration is acquired, and a predetermined time is determined according to the refresh rate.

In some embodiments, when the processor 121 realizes the determination of the predetermined time according to the refresh rate, it realizes:

The refresh time of the display is determined according to the refresh frequency, and the refresh time is taken as the predetermined time.

In some embodiments, when the processor 121 implements the decoding of the video data by a decoder, it implements:

The video data is decoded by the decoder in a time less than one frame of video data.

The embodiment of the present application also provides an image transmission system. As shown in FIG. 1, the image transmission system includes a sending end and a receiving end. The sending end and the receiving end are connected in communication, for example, in a wired or wireless manner. Make a communication connection; among them,

The sending end obtains video data and encodes the video data;

The sending end sends the encoded video data to the receiving end;

In some embodiments, acquiring the video data by the sending end and encoding the video data includes:

Acquiring video data in a time less than one frame of video data, and encoding the video data in a time less than one frame of video data;

The sending of the encoded video data by the sending end to the receiving end includes: sending the encoded video data to the receiving end in a time less than one frame of video data.

In some embodiments, the preset number threshold includes a preset number of macroblocks;

The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold includes:

In some embodiments, the encoded video data is encoded into multiple slices or slice groups, and the preset number threshold includes a preset number of slices or a preset number of slice groups;

In some embodiments, the receiving end also adjusts the size of the preset number threshold according to the stable state of the video data transmission.

In some embodiments, the adjusting the size of the preset number threshold according to the stable state of the video data transmission includes:

In some embodiments, the determining whether the transmission of the video data is stable includes:

In some embodiments, the determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time includes:

Determining the arrival required time for the remaining data of the current frame to be transmitted to the display, and determining whether the arrival required time is less than or equal to a predetermined time;

If the arrival required time is less than or equal to the predetermined time, it is determined that the remaining part of the data of the current frame can be transmitted to the display within the predetermined time.

In some embodiments, the determining the arrival required time for the remaining data of the current frame to be transmitted to the display includes:

In some embodiments, the receiving end also obtains the refresh rate configured by the display, and determines the predetermined time according to the refresh rate.

In some embodiments, the determining the predetermined time according to the refresh rate includes:

The refresh time of the display is determined according to the refresh frequency, and the refresh time is used as the predetermined time.

In some embodiments, the decoding the video data by a decoder includes:

In some embodiments, the sending end includes, but is not limited to, a removable device.

In some embodiments, the movable device includes, but is not limited to, a drone, a handheld pan/tilt, an unmanned vehicle, or a robot.

In some embodiments, the receiving end includes, but is not limited to, a remote control, a mobile phone, or a tablet computer.

In some embodiments, the receiving end includes, but is not limited to, a display and a display controller, and the display controller is electrically connected to the processor for controlling the display of the display.

The embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation The steps of the video display method provided in the example.

The computer-readable storage medium may be the internal storage unit of the handheld pan/tilt head described in any of the foregoing embodiments, such as the hard disk or memory of the handheld pan/tilt head. The computer-readable storage medium may also be an external storage device of the handheld PTZ, such as a plug-in hard disk equipped on the handheld PTZ, a smart memory card (Smart Media Card, SMC), and a secure digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A video display method, characterized in that it comprises:

Receiving encoded video data, and decoding the video data through a decoder;

Determining whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, where the preset condition is used to trigger the generation of a timing signal;

If part of the output data meets the preset condition, a timing signal is generated and sent to the display controller, so that the display controller controls the display screen to display the current frame.
The video display method according to claim 1, wherein the preset condition comprises:

Detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold; and/or determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time.
The video display method according to claim 2, wherein the preset condition comprises: detecting whether part of the data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, and determining Whether the remaining data of the current frame can be transmitted to the display within a predetermined time;

The determining whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition includes:

If the output part of the data is greater than the preset number threshold, and the remaining data can be transmitted to the display within a predetermined time, it is determined that the output part of the data satisfies the preset condition.
The video display method according to claim 2, wherein the preset number threshold includes a preset number of macroblocks;

The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold includes:

Count the number of macroblocks output by the decoder in decoding the current frame, and determine whether the number of output macroblocks is greater than the preset number of macroblocks.
The video display method according to claim 2, wherein the encoded video data is encoded into a plurality of slices or slice groups, and the preset number threshold includes a preset number of slices or a preset number of slice groups ；

The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold includes:

Count the number of slices or slice groups output by the decoder in decoding the current frame, and determine whether the number of output slices is greater than the preset number of slices, or whether the number of output slices is greater than the preset number Number of film groups.
The video display method according to claim 2, wherein the video display method further comprises:

Adjust the size of the preset number threshold according to the stable state of the video data transmission.
The video display method according to claim 6, wherein the adjusting the size of the preset number threshold according to the stable state of the video data transmission comprises:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, increase the size of the preset number threshold;

If the transmission of the video data is unstable, reduce the size of the preset number threshold.
The video display method according to claim 6, wherein the adjusting the size of the preset number threshold according to the stable state of the video data transmission comprises:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, determining the first preset number as the preset number threshold;

If the transmission of the video data is unstable, a second preset number is determined as the preset number threshold, where the second preset number is greater than the first preset number.
The video display method according to claim 7 or 8, wherein the determining whether the transmission of the video data is stable comprises:

Monitoring whether the delay time of the video data transmission changes;

If the delay time of the video data transmission is fixed, determining that the video data transmission is stable;

If the delay time of the video data transmission changes, it is determined that the video data transmission is unstable.
The video display method according to claim 2, wherein the determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time comprises:

Determining the arrival required time for the remaining data of the current frame to be transmitted to the display, and determining whether the arrival required time is less than or equal to a predetermined time;

If the arrival required time is less than or equal to the predetermined time, it is determined that the remaining part of the data of the current frame can be transmitted to the display within the predetermined time.
The video display method according to claim 10, wherein the determining the arrival required time for the remaining data of the current frame to be transmitted to the display comprises:

Acquiring the number of macroblocks contained in the remaining data of the current frame;

Determining the output time and delay time of the macro block; and

The arrival demand time is determined according to the number of the macroblocks, the output time and the delay time.
The video display method according to claim 3, 10 or 11, wherein the video display method further comprises:

The refresh rate of the display configuration is acquired, and a predetermined time is determined according to the refresh rate.
The video display method according to claim 12, wherein the determining the predetermined time according to the refresh rate comprises:

The refresh time of the display is determined according to the refresh frequency, and the refresh time is used as the predetermined time.
The video display method according to claim 1, wherein said decoding said video data by a decoder comprises:

The video data is decoded by the decoder in a time less than one frame of video data.
A receiving end, characterized in that the receiving end includes a memory and a processor;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Receiving encoded video data, and decoding the video data through a decoder;

Determining whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, where the preset condition is used to trigger the generation of a timing signal;

If part of the output data meets the preset condition, a timing signal is generated and sent to the display controller, so that the display controller controls the display screen to display the current frame.
The receiving end according to claim 15, wherein the preset condition comprises:

Detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold; and/or determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time.
The receiving end according to claim 16, wherein the preset condition comprises: detecting whether part of the data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, and determining Whether the remaining data of the current frame can be transmitted to the display within a predetermined time;

When the processor implements the determining whether part of the data output by the current frame in the video data decoded by the decoder meets a preset condition, implement:

If the output part of the data is greater than the preset number threshold, and the remaining data can be transmitted to the display within a predetermined time, it is determined that the output part of the data satisfies the preset condition.
The receiving end according to claim 16, wherein the preset number threshold includes a preset number of macroblocks;

When the processor implements the detection of whether part of the data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, implement:

Count the number of macroblocks output by the decoder in decoding the current frame, and determine whether the number of output macroblocks is greater than the preset number of macroblocks.
The receiving end according to claim 16, wherein the encoded video data is encoded into multiple slices or slice groups, and the preset number threshold includes a preset number of slices or a preset number of slice groups;

When the processor implements the detection of whether part of the data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, implement:

Count the number of slices or slice groups output by the decoder in decoding the current frame, and determine whether the number of output slices is greater than the preset number of slices, or whether the number of output slices is greater than the preset number Number of film groups.
The receiving end according to claim 16, wherein the processor further implements:

Adjust the size of the preset number threshold according to the stable state of the video data transmission.
The receiving end according to claim 20, wherein when the processor realizes the adjustment of the size of the preset number threshold according to the stable state of the video data transmission, it realizes:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, increase the size of the preset number threshold;

If the transmission of the video data is unstable, reduce the size of the preset number threshold.
The receiving end according to claim 20, wherein when the processor realizes the adjustment of the size of the preset number threshold according to the stable state of the video data transmission, it realizes:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, determining the first preset number as the preset number threshold;

If the transmission of the video data is unstable, a second preset number is determined as the preset number threshold, where the second preset number is greater than the first preset number.
The receiving end according to claim 21 or 22, wherein, when the processor realizes the determination of whether the transmission of the video data is stable, realizes:

Monitoring whether the delay time of the video data transmission changes;

If the delay time of the video data transmission is fixed, determining that the video data transmission is stable;

If the delay time of the video data transmission changes, it is determined that the video data transmission is unstable.
The receiving end according to claim 16, wherein the processor realizes the determination of whether the remaining data of the current frame can be transmitted to the display within a predetermined time, and realizes:

Determining the arrival required time for the remaining data of the current frame to be transmitted to the display, and determining whether the arrival required time is less than or equal to a predetermined time;

If the arrival required time is less than or equal to the predetermined time, it is determined that the remaining part of the data of the current frame can be transmitted to the display within the predetermined time.
The receiving end according to claim 24, wherein the processor realizes the determination of the arrival required time for the remaining data of the current frame to be transmitted to the display, and realizes:

Acquiring the number of macroblocks contained in the remaining data of the current frame;

Determining the output time and delay time of the macro block; and

The arrival demand time is determined according to the number of the macroblocks, the output time and the delay time.
The receiving end according to claim 17, 24 or 25, wherein the processor further implements:

The refresh rate of the display configuration is acquired, and a predetermined time is determined according to the refresh rate.
The receiving end according to claim 26, wherein when the processor realizes the determination of the predetermined time according to the refresh rate, it realizes:

The refresh time of the display is determined according to the refresh frequency, and the refresh time is used as the predetermined time.
The receiving end according to claim 15, wherein the processor realizes the decoding of the video data by the decoder to realize:

The video data is decoded by the decoder in a time less than one frame of video data.
The receiving end according to claim 15, wherein the receiving end comprises a display and a display controller, and the display controller is electrically connected with the processor for controlling the display of the display.
The receiving terminal according to claim 15, wherein the receiving terminal comprises a remote control, a mobile phone or a tablet computer.
An image transmission system, characterized in that it comprises a sending end and a receiving end, and the sending end and the receiving end are in communication connection;

The sending end obtains video data and encodes the video data;

The sending end sends the encoded video data to the receiving end;

The receiving end receives the encoded video data, and decodes the video data through a decoder;

Determining, by the receiving end, whether part of the data output by the current frame in the video data decoded by the decoder satisfies a preset condition, and the preset condition is used to trigger the generation of a timing signal;

If the receiving end determines that part of the output data meets the preset condition, it generates a timing signal and sends the timing signal to the display controller, so that the display controller controls the display screen to display the current frame.
The image transmission system according to claim 31, wherein said sending end obtains video data and encodes said video data, comprising:

Acquiring video data in a time less than one frame of video data, and encoding the video data in a time less than one frame of video data;

The sending of the encoded video data by the sending end to the receiving end includes: sending the encoded video data to the receiving end in a time less than one frame of video data.
The image transmission system according to claim 31, wherein the preset condition comprises:

Detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold; and/or determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time.
The image transmission system according to claim 33, wherein the preset condition comprises: detecting whether part of the data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold, and determining Whether the remaining data of the current frame can be transmitted to the display within a predetermined time;

The determining whether the partial data output by the current frame in the video data decoded by the decoder meets a preset condition includes:

If the output part of the data is greater than the preset number threshold, and the remaining data can be transmitted to the display within a predetermined time, it is determined that the output part of the data satisfies the preset condition.
The image transmission system according to claim 33, wherein the preset number threshold includes a preset number of macroblocks;

The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold includes:

Count the number of macroblocks output by the decoder in decoding the current frame, and determine whether the number of output macroblocks is greater than the preset number of macroblocks.
The image transmission system according to claim 33, wherein the encoded video data is encoded into multiple slices or slice groups, and the preset number threshold includes a preset number of slices or a preset number of slice groups ；

The detecting whether the partial data output by the current frame in the video data decoded by the decoder is greater than a preset number threshold includes:

Count the number of slices or slice groups output by the decoder in decoding the current frame, and determine whether the number of output slices is greater than the preset number of slices, or whether the number of output slices is greater than the preset number Number of film groups.
The image transmission system according to claim 33, wherein the receiving end further adjusts the size of the preset number threshold according to the stable state of the video data transmission.
The image transmission system according to claim 37, wherein the adjusting the size of the preset number threshold according to the stable state of the video data transmission comprises:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, increase the size of the preset number threshold;

If the transmission of the video data is unstable, reduce the size of the preset number threshold.
The image transmission system according to claim 37, wherein the adjusting the size of the preset number threshold according to the stable state of the video data transmission comprises:

Determine whether the transmission of the video data is stable;

If the transmission of the video data is stable, determining the first preset number as the preset number threshold;

If the transmission of the video data is unstable, a second preset number is determined as the preset number threshold, where the second preset number is greater than the first preset number.
The image transmission system according to claim 38 or 39, wherein the determining whether the transmission of the video data is stable comprises:

Monitoring whether the delay time of the video data transmission changes;

If the delay time of the video data transmission is fixed, determining that the video data transmission is stable;

If the delay time of the video data transmission changes, it is determined that the video data transmission is unstable.
The image transmission system according to claim 33, wherein the determining whether the remaining data of the current frame can be transmitted to the display within a predetermined time comprises:

Determining the arrival required time for the remaining data of the current frame to be transmitted to the display, and determining whether the arrival required time is less than or equal to a predetermined time;

If the arrival required time is less than or equal to the predetermined time, it is determined that the remaining part of the data of the current frame can be transmitted to the display within the predetermined time.
The image transmission system according to claim 41, wherein the determining the arrival required time for the remaining data of the current frame to be transmitted to the display comprises:

Acquiring the number of macroblocks contained in the remaining data of the current frame;

Determining the output time and delay time of the macro block; and

The arrival demand time is determined according to the number of the macroblocks, the output time and the delay time.
The image transmission system according to claim 34, 41 or 42, wherein the receiving end also obtains the refresh rate of the display configuration, and determines the predetermined time according to the refresh rate.
The image transmission system according to claim 43, wherein the determining the predetermined time according to the refresh rate comprises:

The refresh time of the display is determined according to the refresh frequency, and the refresh time is used as the predetermined time.
The image transmission system according to claim 31, wherein said decoding said video data by a decoder comprises:

The video data is decoded by the decoder in a time less than one frame of video data.
The image transmission system according to claim 31, wherein the sending end comprises a movable device.
The image transmission system according to claim 46, wherein the movable device comprises: an unmanned aerial vehicle, a handheld pan/tilt or a robot.
The image transmission system according to claim 31, wherein the receiving end comprises a remote control, a mobile phone, or a tablet computer.
The image transmission system according to claim 31, wherein the receiving end comprises a display and a display controller, and the display controller is electrically connected to the processor for controlling the display of the display.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the implementation of The video display method described.