WO2022116353A1 - 一种低延时视频传输方法及其系统、储存介质 - Google Patents

一种低延时视频传输方法及其系统、储存介质 Download PDF

Info

Publication number
WO2022116353A1
WO2022116353A1 PCT/CN2020/141937 CN2020141937W WO2022116353A1 WO 2022116353 A1 WO2022116353 A1 WO 2022116353A1 CN 2020141937 W CN2020141937 W CN 2020141937W WO 2022116353 A1 WO2022116353 A1 WO 2022116353A1
Authority
WO
WIPO (PCT)
Prior art keywords
read
write
pointer
frame
data
Prior art date
Application number
PCT/CN2020/141937
Other languages
English (en)
French (fr)
Inventor
胡波
卢伟军
Original Assignee
威创集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 威创集团股份有限公司 filed Critical 威创集团股份有限公司
Publication of WO2022116353A1 publication Critical patent/WO2022116353A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping

Definitions

  • the present invention relates to the field of video transmission, and more particularly, to a low-latency video transmission method, system and storage medium thereof.
  • the buffer space is used as the medium of the video transmission process.
  • the input end continuously writes frame-by-frame picture data to the buffer space, and the output end reads the picture data from the buffer space and plays the picture frame by frame.
  • Data is limited, and it is inevitable to reuse the frame memory. Therefore, in the process of video transmission, it is necessary to use the read pointer and the write pointer to guide the coordination of the read operation and the write operation in the limited buffer space, respectively.
  • the present invention aims to overcome at least one defect (deficiency) of the above-mentioned prior art, and provides a low-latency video transmission method, system, and storage medium thereof.
  • the method of the present invention judge whether the output frame rate is consistent with the input frame rate, if the output frame rate is consistent with the input frame rate, use one frame or more buffer space for read and write operations; if the output frame rate is consistent with the input frame rate Inconsistent, use the cache space of two frames to perform the first type of read and write operations; the first type of read and write operations are: the read pointer and the write pointer move alternately on the two frame memories, and determine whether there will be an access conflict during the movement process. , according to the judgment result, allocate the frame memory for read operation by the read pointer and the frame memory for write operation by the write pointer.
  • the read frame is the frame being written or the previous frame of the frame being written, and the maximum delay does not exceed one frame.
  • control the write pointer that guides the write operation (write) and guides the read operation (read).
  • the read pointer points to the same frame memory or different frame memory of the two frame buffer spaces, so as to guide the write operation and the read operation in an orderly manner.
  • the step of distributing the frame memory of the read pointer to carry out the read operation and the write pointer to carry out the frame memory of the write operation are specifically: the read/write pointer is when jumping different frame memories.
  • the read/write pointer will perform a normal jump, and the read/write operation will be performed on the jumped frame memory; if there is a write/read pointer on the frame memory to be jumped , and perform a second judgment operation; the second judgment operation is specifically: preset a safe relative position value; calculate the relative position value of the read pointer and the write pointer, if the relative position value of the read pointer and the write pointer is greater than or equal to the safe relative position value For the relative position value, the read/write pointer performs a normal jump, and the read/write operation is performed on the jumped frame memory; if the relative position value of the read pointer and the write pointer is smaller than the safe relative position value, the read/write pointer does not perform the read/write operation.
  • Jump perform read/write operations on the original frame memory.
  • An access conflict means that part of the data is rewritten by a write operation when reading a frame of pictures, or a read operation reads unupdated data when a frame of pictures is written.
  • the read rate and write rate change continuously in real time during the transmission process.
  • the pointer or the write pointer performs a judgment operation every time it jumps, so that the distance between the two is always greater than or equal to the safe relative position value, so as to avoid the image splitting phenomenon caused by the access conflict.
  • the safe relative position value is calculated and determined according to the speed difference between read and write operations.
  • the safe relative position value is positively correlated with the delay.
  • the delay can be controlled by changing the safe relative position value to adapt to different transmission scenarios and has flexibility .
  • the second type of read and write operations are specifically: write operations; when a certain amount of data is completed After writing, start the read operation; compare the rate of the read operation and the rate of the write operation in real time, if the former is larger than the latter, reduce the clock frequency of the read pointer; if the former is smaller than the latter, increase the clock frequency of the read pointer; complete After a frame memory read operation, the read pointer jumps to the frame memory where the write pointer is currently located.
  • the write operation in the second type of read and write operations is performed normally, and there is no need to perform judgment operations or change the clock frequency of the write pointer.
  • Most of the read and write pointers are in the same frame memory during the transmission process. , the read operation is followed by the write operation, and the delay is kept within one frame, which is positively related to the size of the certain amount of data.
  • the several quantities of data are calculated and determined according to the difference between the read operation rate and the write operation rate.
  • the buffer space of one frame can also be used to perform the third type of read and write operations; the third type of read and write operations are specifically: start the write operation at the initial line on one frame memory , start the read operation after writing the data of L rows; where L is an integer and is a preset value; each time the writing operation of one frame of data is completed, the time required to write a row of data currently and the time required to complete reading a row of data are calculated.
  • the write pointer jumps to the initial row, and performs the write operation on the initial row and other rows on the frame memory; the same is true for the read operation, and it is judged whether the read operation will be read in the initial row of the one frame memory.
  • a new frame of data that has not been written that is, whether there will be another access conflict phenomenon. If it is, the data of the original row will be read repeatedly. When there is no access conflict phenomenon, the read pointer will jump to the initial row. Then read the data of a new frame that has been written.
  • the above process can keep the read and write pointers at an ultra-low latency smaller than several lines of data in one frame, and the read and write pointers can be relatively close in the same frame memory without access conflict.
  • the L is calculated and determined according to the difference between the read operation rate and the write operation rate.
  • the system of the present invention includes: a judgment module for judging whether the output frame rate is consistent with the input frame rate and whether access conflict occurs; a control module for controlling the reading operation and the writing operation; a cache module for reading operation and write operation; if the judgment module judges that the output frame rate is consistent with the input frame rate, the control module controls the read and write operations on the cache module, and the cache module enables one frame or more buffer space; if the judgment module judges the output frame rate Inconsistent with the input frame rate, the control module controls the first type of read and write operations on the cache module, and the cache module enables two frames of buffer space; the first type of read and write operations are specifically: the control module controls the read pointer and write The pointer moves alternately on the two frame memories of the cache module. During the movement of the read pointer and the write pointer, the judgment module judges whether there will be an access conflict, and the control module allocates the read pointer to the frame memory of the read operation and the write pointer to write according to the judgment result. Frame memory for operations.
  • the judgment module judges whether there is an access conflict through the relative positions of the read pointer and the write pointer, as follows: if there is no write/read pointer on the frame memory to which the read/write pointer is about to jump, the judgment result of the judgment module is no.
  • the judgment module performs a second judgment operation: calculates the relative position value of the read pointer and the write pointer, and compares the relative position value with the preset The safety relative position value is compared, if the former is greater than the latter, the judgment result of the judgment module is no conflict; if the former is less than the latter, the judgment result of the judgment module is a conflict; if the judgment result is no conflict, the control module controls the read/write pointer For normal jumps, read/write operations are performed on the jumped frame memory; if the judgment result is a conflict, the control module controls the read/write pointer not to jump, and re-read/write operations are performed on the original frame memory.
  • the present invention has the beneficial effects of: adapting the video processing mode according to the input and output conditions, achieving a low delay of no more than one frame at the maximum, and ensuring the integrity of each frame of the picture, improving the video transmission. Efficiency while ensuring transmission quality.
  • FIG. 1 is a step diagram of the first type of read and write operations of the present invention.
  • FIG. 2 is a step diagram of the first and second types of read and write operations of the present invention.
  • FIG. 3 is a flow chart of write operation steps of the first type and the third type of read and write operations of the present invention.
  • FIG. 4 is a flow chart of the read operation steps of the first type and the third type of read and write operations of the present invention.
  • FIG. 5 is a system structure diagram of the present invention.
  • judgment module 1 judgment module 1; control module 2; cache module 3;
  • This embodiment provides a low-latency video transmission method, including the steps of: judging whether the output frame rate is consistent with the input frame rate, and if the output frame rate is consistent with the input frame rate, use one frame or more buffer space for reading Write operation; if the output frame rate is inconsistent with the input frame rate, use the buffer space of two frames to perform the first type of read and write operations; the first type of read and write operations are: the read pointer and the write pointer alternate on the two frame memories Move, judge whether there will be an access conflict during the moving process, and allocate the frame memory of the read pointer for read operation and the frame memory of write pointer for write operation according to the judgment result.
  • the step of distributing the frame memory of the read pointer to carry out the read operation and the write pointer to carry out the frame memory of the write operation are specifically: the read/write pointer is when jumping different frame memories. , if there is no write/read pointer on the frame memory to be jumped, the read/write pointer will perform a normal jump, and the read/write operation will be performed on the jumped frame memory; if there is a write/read pointer on the frame memory to be jumped , and perform a second judgment operation; the second judgment operation is specifically:
  • the safe relative position value Pre-set the safe relative position value; calculate the relative position value of the read pointer and the write pointer, if the relative position value of the read pointer and the write pointer is greater than or equal to the safe relative position value, the read/write pointer performs a normal jump, and the jump The read/write operation is performed on the frame memory; if the relative position value of the read pointer and the write pointer is smaller than the safe relative position value, the read/write pointer does not jump, and the read/write operation is performed on the original frame memory. Further, the safe relative position value is calculated and determined according to the speed difference between read and write operations. When the read and write rate fluctuates greatly, the safe relative position value can be appropriately increased to effectively avoid image splitting.
  • S1 Determine whether the output frame rate is consistent with the input frame rate; if the output frame rate is consistent with the input frame rate, go to S2; if the output frame rate is inconsistent with the input frame rate, go to S3;
  • S2 use one frame or more than one frame of cache space to perform read and write operations, the read and write operations are not limited, and can be common read and write processes well known to those skilled in the art;
  • S3 Use the buffer space of two frames to perform the first type of read and write operations, specifically:
  • S32 After completing the read/write operation of one frame, judge whether there is a write/read pointer on the frame memory to which the read/write pointer will jump; if there is no write/read pointer on the frame memory to be jumped, perform S32a; if there is a write/read pointer on the frame memory to be jumped, go to S32b;
  • S321 Set a safe relative position value; in other embodiments, the safe relative position value may be set in advance, thereby skipping S321;
  • S322 Calculate the relative position value of the read pointer and the write pointer, and compare the relative position value and the safe relative position value, if the relative position value is greater than or equal to the safe relative position value, proceed to S322a; if the relative position value is less than the safe relative position value position value, go to S322b;
  • the advanced reading is specifically: when displaying a frame of pictures, the reading rate is faster than the writing rate, the read pointer exceeds the write pointer, points to the buffer space where the write pointer has not been updated, and reads a part of the data of the old frame. , causing the displayed picture to be mixed with images or rendering effects of different frames, resulting in image splitting.
  • the overwriting is specifically as follows: the writing rate is faster than the reading rate, the write pointer completes the writing of one frame, and jumps to the frame memory where the read pointer is located to start the writing of a new frame, When it surpasses the read pointer in one frame, and writes a new frame of data before the read pointer reaches the buffer space, overwriting the data that the read pointer should have read but has not yet read, the subsequent data read by the read pointer will not be read. It belongs to the original frame, and there will also be image splitting in the same frame.
  • the relative position value of the read pointer and the write pointer can be the difference between the lines pointed to by the two in the cache space, or the difference between the amount of data read and the amount of data written to reflect the progress difference between the read operation and the write operation.
  • the reference value of can be used as the relative position value.
  • the safe relative position value is pre-set and can be determined according to transmission requirements and average read/write rate, which is flexible. When the read pointer does not jump, it is equivalent to re-reading the frame just read; when the write pointer does not jump, it is equivalent to overwriting the unread frame and writing the latest frame .
  • Embodiment 1 The difference between this embodiment and Embodiment 1 is that the transmission method when the output frame rate is the same as the input frame rate is further improved, and the second type of read-write operation is used, which includes the steps of: performing a write operation; After writing, the read operation is started, wherein the number of data can be in units of lines or bytes. In this embodiment, the number of data is set as K lines, and K may not be an integer; compare the speed of the read operation in real time If the former is larger than the latter, reduce the clock frequency of the read pointer; if the former is smaller than the latter, increase the clock frequency of the read pointer; after a frame memory read operation is completed, the read pointer jumps to the current write pointer the frame memory in which it resides.
  • the read pointer remains behind the write pointer, and most of the read and write pointers are in the same frame memory during the transmission process. It is positively related to the value of K, and the transmission environment with different delay requirements can be adapted by adjusting the value of K.
  • the optimal implementation of the low-latency video transmission method described in this embodiment is as follows:
  • A1 Determine whether the output frame rate is the same as the input frame rate; if the output frame rate is the same as the input frame rate, go to A2; if the output frame rate is the same as the input frame rate, go to A3;
  • A2 Use two or more frames of buffer space for the second type of read and write operations, specifically:
  • A21 Start the write operation on a frame memory, and start the read operation after writing K lines;
  • A22 Compare the rate of the read operation and the rate of the write operation in real time; if the rate of the read operation is higher than the rate of the write operation, perform A22a; if the rate of the read operation is lower than the rate of the write operation, perform A32b; if the rate of the read operation is equal to
  • the rate of write operation can be A22a or A32b without limitation;
  • A22a Decrease the clock frequency of the read pointer; after completing the write operation of one frame memory, the write pointer jumps to the next frame memory; after completing the read operation of one frame memory, the read pointer jumps to the frame memory where the write pointer is currently located;
  • A22b Increase the clock frequency of the read pointer; after completing the write operation of one frame memory, the write pointer jumps to the next frame memory; after completing the read operation of one frame memory, the read pointer jumps to the frame memory where the write pointer is currently located;
  • A3 Use the buffer space of two frames to perform the first type of read and write operations, specifically:
  • A31 The read pointer and the write pointer are separated on two frame memories, and the read operation and the write operation are performed on different frame memories;
  • A32 After completing the read/write operation of one frame, judge whether there is a write/read pointer in the frame memory to which the read/write pointer will jump; if there is no write/read pointer in the frame memory to be jumped, proceed A32a; If there is a write/read pointer on the frame memory to be jumped, perform A32b;
  • A32a read/write pointer for normal jump
  • A32b Perform the second judgment operation: specifically:
  • A321 Set a safe relative position value; in other embodiments, the safe relative position value may be set in advance, thereby skipping A321;
  • A322 Calculate the relative position value of the read pointer and the write pointer, and compare the relative position value with the safe relative position value. If the relative position value is greater than or equal to the safe relative position value, perform A322a; if the relative position value is less than the safe relative position value position value, perform A322b;
  • A322a read/write pointer for normal jump
  • A322b The read/write pointer does not jump.
  • the above processing process controls the delay to be equal to or less than one frame, while ensuring the integrity of each frame.
  • two or more frames of buffer space are used for read and write operations. Cache space, repeatedly write multiple frames of data on one frame memory, and the write pointer jumps in a circular manner; the read pointer trails the write pointer, after completing the reading of one frame of data, the read pointer jumps, pointing to the only one frame memory that has already been read. Write a new frame of data row, read a new frame of data.
  • Embodiment 2 The difference between this embodiment and Embodiment 2 is that when the output frame rate is the same as the input frame rate, a buffer space of one frame is used to perform the third type of read and write operations.
  • the third type of read and write operations are specifically:
  • L is an integer and is a preset value; each time the write operation of one frame of data is completed, the current written line of data is calculated.
  • the jumping methods of the read pointer and the write pointer are different, and they occur in the entire transmission process at the same time, so the write operation and the read operation are discussed separately.
  • the optimal implementation of the write operation is:
  • B1 Write L lines of data on a frame memory in the cache space; before the writing of L lines of data is not completed, no judgment is required before the write pointer jumps, and the jump is normal; where L is an integer and is a preset value;
  • B2 After completing the writing of L lines of data, each time the writing operation completes the writing of one frame of data, when the write pointer is about to jump to the next line, calculate the time required to write a line of data currently and the time required to complete reading a line of data time, and compare the two, if the former is larger than the latter, go to B2a; if the former is smaller than the latter, go to B2b;
  • B2a The write pointer jumps normally, and the write operation of a new frame of data starts at the initial row;
  • B2b The write pointer does not jump, and the data of the original frame is repeatedly written in the original row. After completion, the write pointer jumps to the initial row, and the writing of a new frame of data starts at the initial row.
  • B2a or B2b can be arbitrarily performed when the time required to write a row of data currently and the time required to complete reading a row of data.
  • C2 When the read operation completes the reading of one frame of data, when the read pointer is about to jump, calculate the time required to write a row of data and the time required to complete reading a row of data, and compare the two. If the latter is larger, go to C2a; if the former is smaller than the latter, go to C2b;
  • C2a The read pointer does not jump, and re-reads the original row. After completion, the read pointer jumps to the initial row, and starts the read operation of a new frame of data in the initial row;
  • C2b The read pointer jumps normally, and a new frame of data is read in the initial row.
  • C2a or C2b can be arbitrarily performed when the current time required for writing a row of data and the time required to complete reading a row of data.
  • this embodiment can achieve lower delay without the phenomenon of image splitting caused by access conflict, and is suitable for scenarios where delay is emphasized but image quality is not required.
  • the optimal implementation of the above-mentioned read operation and write operation occurs after it has been judged that the input frame rate is consistent with the output frame rate, and there is no specific process for judging whether the input frame rate and the output frame rate are consistent. Those skilled in the art may refer to Example 2 shows the complete process.
  • Embodiment 3 when the output frame rate is the same as the input frame rate, the buffer space of one frame is used for reading and writing, and the second type of read and write operations and the third type of read and write operations are used for video transmission. details as follows:
  • the read/write time difference threshold is used to determine whether to use the second type of read/write operation or the third type of read/write operation; when the output frame rate and the input frame rate are consistent, start the read/write operation Before and before starting the reading and writing of a new frame, calculate the difference between the time required to write a line and the time required to read a line, and compare the difference with the read and write time difference threshold. If the former is greater than the latter, The second type of operation is used; if the former is smaller than the latter, the third type of read and write operation is used.
  • the specific steps can be known by those skilled in the art with reference to Example 2 and Example 3.
  • this embodiment balances the transmission delay and the picture quality, the delay is lower than that of the second embodiment, and the picture quality is better than that of the third embodiment.
  • the present embodiment provides a system for implementing the low-latency video transmission method, including: a judgment module 1 for judging whether the output frame rate and the input frame rate are consistent and whether there is an access conflict;
  • the control module 2 is used to control the read operation and the write operation;
  • the cache module 3 is used to perform the read operation and the write operation, and the output end and the input end carry out data transmission through the cache space; if the judgment module 1 judges the output frame rate Consistent with the input frame rate, the control module 2 controls the read and write operations on the cache module 3, and the cache module 3 enables one frame or more buffer space; if the judgment module 1 judges that the output frame rate and the input frame rate are inconsistent, the control module 2.
  • the first type of read-write operation is controlled on the cache module 3, and the cache module 3 enables two frames of cache space; the first type of read-write operation is specifically: the control module 2 controls the read pointer and the write pointer in the cache module.
  • the two frame memories of 3 move alternately.
  • the judgment module 1 judges whether there will be an access conflict, and the control module 2 allocates the frame memory and the write pointer of the read operation according to the judgment result to perform the write operation. frame memory.
  • the judgment module 1 judges whether there will be an access conflict through the relative positions of the read pointer and the write pointer, as follows: if there is no write/read pointer on the frame memory to which the read/write pointer is about to jump, the judgment result of the judgment module 1 is determined.
  • the judgment module 1 performs a second judgment operation: calculates the relative position value of the read pointer and the write pointer, and compares the relative position value with The preset safe relative position value is compared, if the former is greater than the latter, the judgment result of the judgment module 1 is no conflict; if the former is less than the latter, the judgment result of the judgment module 1 is a conflict; if the judgment result is no conflict, the control module 2 Control the read/write pointer to jump normally, and perform read/write operations on the jumped frame memory; if the judgment result is a conflict, control module 2 controls the read/write pointer not to jump, and re-read/write on the original frame memory write operation.
  • the low-latency video transmission system of this embodiment can also implement the second type of read and write operations and the third type of read and write operations. If the judgment module 1 judges that the output frame rate is consistent with the input frame rate, the control module 2 executes the second type of read-write operation or the third type of read-write operation.
  • the process of performing the second type of read and write operations is specifically: the cache module 3 uses one frame or more of the cache space to perform read operations and write operations, and the control module 2 controls to start a write operation on a frame memory of the cache module 3 , start the read operation after writing a certain amount of data; the judgment module 1 compares the rate of the read operation and the rate of the write operation in real time, if the former is larger than the latter, the control module 2 reduces the clock frequency of the read pointer; if the former is smaller than the latter, The control module 2 increases the clock frequency of the read pointer.
  • the control module 2 controls the write pointer to jump to the next frame memory of the cache module 3; after completing the read operation of a frame memory, the control module 2 controls the read pointer to jump to the current location of the write pointer. frame save.
  • the cache module 3 may also use the cache space of one frame to perform the second type of read and write operations.
  • the process of performing the third type of read and write operations is as follows: the cache module 3 uses a frame of cache space to perform read operations and write operations, the control module 2 controls the start of the write operation on a frame memory, and starts to read after writing the data of the L line.
  • the judgment module 1 calculates the time required to write a line of data currently and completes the reading The time required for one line of data, and compare the two, if the former is larger than the latter, the control module 2 controls the write pointer to jump normally, and starts the writing operation at the initial line of the one frame memory; if the former is smaller than the latter, The control module 2 controls the write pointer not to jump, repeats writing the original frame data in the original row, and starts the writing operation of a new frame of data in the initial row after completion; every time the read operation of one frame of data is completed, the read pointer is about to jump to the next When there is one row, the judgment module 1 calculates the time required to write a row of data currently and the time required to complete reading a row of data, and compares the two.
  • control module 2 controls the read pointer not to jump. Perform repeated reading in the original row, after completing one repeated reading of the original row, the control module 2 controls the read pointer to jump to the initial row, and starts the read operation at the initial row; if the former is smaller than the latter, the control module 2 controls the read pointer to jump normally Turn, start the read operation at the initial row.
  • This embodiment provides a computer storage medium, which stores software for implementing the low-latency video transmission method, so that when the output frame rate and the input frame rate in video transmission are inconsistent, the low-latency of not more than one frame is maintained, and the minimum Ultra-low latency in line units can be achieved, and the integrity of each frame is guaranteed.
  • the low-latency video transmission method is as follows: judging whether the output frame rate is consistent with the input frame rate, and if the output frame rate is consistent with the input frame rate, a buffer space of one frame or more is used for read and write operations; If the input frame rate is inconsistent with the input frame rate, the first type of read and write operations are performed using the buffer space of two frames. Whether there will be an access conflict, according to the judgment result, allocate the frame memory of the read pointer for the read operation and the frame memory of the write pointer to perform the write operation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Systems (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

一种低延时视频传输方法及其系统、储存介质，所述方法为：判断输出帧率和输入帧率是否一致，若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行读写操作；若输出帧率和输入帧率不一致，采用两帧的缓存空间进行第一类读写操作；所述第一类读写操作具体为：读指针和写指针在两个帧存上交替移动，移动过程中判断是否会出现访问冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。本发明可降低视频传输的延时，并且保证每帧画面的完整性，提高视频传输效率的同时保证传输质量。

Description

一种低延时视频传输方法及其系统、储存介质
本申请要求于2020年12月03日提交中国专利局、申请号为202011412867.X、发明名称为“一种低延时视频传输方法及其系统、储存介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及视频传输领域，更具体地，涉及一种低延时视频传输方法及其系统、储存介质。
背景技术
随着视频处理技术的发展，在视频处理领域为方便视频传输设备之间的同步传输，而需要用缓存技术，但是目前为了保证每帧图像输出的完整性采用多帧(N)缓存进行处理，读写异步钟下，当读取比写入快时，延时为一帧；当读取比写入慢时，延时为N帧。在一些延时要求比较高的场合下满足不了应用需求，而如果采用专有低延时处理芯片处理则会增加产品成本。
缓存空间作为视频传输过程的介质，输入端向缓存空间不断写入一帧帧的画面数据，输出端从缓存空间读取所述画面数据并将画面一帧帧地播放，但缓存空间可容量的数据是有限的，避免不了重复使用其中的帧存，因此在视频传输过程中需要使用读指针和写指针分别引导读操作和写操作在有限的缓存空间的协调进行。
发明内容
本发明旨在克服上述现有技术的至少一种缺陷(不足)，提供一种低延时视频传输方法及其系统、储存介质。
本发明的方法：判断输出帧率和输入帧率是否一致，若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行读写操作；若输出帧率和输入帧率不一致，采用两帧的缓存空间进行第一类读写操作；所述第一类读写操作具体为：读指针和写指针在两个帧存上交替移动，移动过程 中判断是否会出现访问冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。在输入帧率和输出帧率不一致时，使用两帧的缓存空间进行读写，读取的帧是正在写入的帧或者正在写入的帧的上一帧，最大延时不超过一帧。为了保证一帧画面的完整性，避免出现因读写效率不平衡导致的在一帧内超前读取或覆盖写入，控制引导写操作(写入)的写指针和引导读操作(读取)的读指针指向两帧缓存空间的同一帧存或不同帧存，从而引导写操作和读操作有序进行。
进一步地，所述移动过程中判断访问是否冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存的步骤具体为：读/写指针在跳转不同帧存时，若即将跳转的帧存上没有写/读指针，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若即将跳转的帧存上有写/读指针，进行第二判断操作；所述第二判断操作具体为：预先设置安全相对位置值；计算读指针和写指针的相对位置值，若读指针和写指针的相对位置值大于或等于所述安全相对位置值，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若读指针和写指针的相对位置值小于所述安全相对位置值，读/写指针不进行跳转，在原帧存上进行读/写操作。访问冲突指读取一帧画面时中途被写操作改写了一部分数据或写入一帧画面时读操作读取了未更新的数据，读速率和写速率在传输过程中不断地进行实时变化，在读指针或者写指针每次跳转都进行判断操作，使二者始终保持大于或等于安全相对位置值的距离，避免出现访问冲突造成的图像分裂现象。
进一步地，所述安全相对位置值根据读、写操作的速率差值进行计算确定。在读速率和写速率相近时，安全相对位置值与延时正相关，安全相对位置值越小，延时越小，可通过改变安全相对位置值控制延时，适应不同的传输情景，具有灵活性。
在输出帧率和输入帧率一致时，则采用一帧或一帧以上的缓存空间进行第二类读写操作；所述第二类读写操作具体为：进行写操作；当完成若干量数据的写入后，开始读操作；实时对比读操作的速率和写操作的速率，若前者比后者大，降低读指针的时钟频率；若前者比后者小，提高读指针的时钟频率；完成一个帧存的读操作后读指针跳转至写指针当前所在的帧 存。在输出帧率和输入帧率一致时，第二类读写操作中写操作正常进行，不需进行判断操作或更改写指针的时钟频率，读写指针在传输过程中大部分处于同一帧存内，读操作尾随写操作，延时保持在一帧以内，其与所述若干量数据的大小正相关。
进一步地，所述若干量数据根据读操作速率和写操作速率的差值进行计算确定。
或者，在输出帧率和输入帧率一致时也可采用一帧的缓存空间进行第三类读写操作；所述第三类读写操作具体为：在一个帧存上的初始行开始写操作，写入L行的数据后开始读操作；其中，L为整数，并且是预设值；每完成一帧数据的写操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在初始行开始进行新一帧数据的写操作；若前者比后者小，在原行进行重复写入的写操作，完成重复写入原行数据后，在初始行开始新一帧数据的写操作；每完成一帧数据的读操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在原行进行读操作，对原行的数据进行重新读取，完成原行的数据重新读取后，在初始行进行新一帧数据的读操作；若前者比后者小，在初始行进行新一帧数据的读操作。此方案应用在需要超低延时的视频传输情景中，每完成一帧数据的写入，以完成一行数据量的写操作时间和完成一行数据量的读操作时间作为判断依据，判断在所述一个帧存的初始行开始新一帧的写入是否会覆盖仍未读取的数据，即是否会出现访问冲突的现象，是则在原行上开始新一帧的写入，完成原行的写入后写指针再跳转至初始行，在初始行以及帧存上的其他行进行写操作；读操作同理，判断在所述一个帧存的初始行进行读取是否会读取到写操作未写入的新一帧数据，即是否会出现另一种访问冲突的现象，是则先重复读取原行的数据，待不会出现访问冲突的现象，读指针再跳转至初始行，再读取已经写入的新一帧的数据。上述过程可使读写指针保持小于一帧的若干行数据量的超低延时，读写指针可在同一帧存内相对位置较近亦不会出现访问冲突的情况。
进一步地，所述L根据读操作速率和写操作速率的差值进行计算确定。
本发明的系统包括：判断模块，用于进行判断输出帧率和输入帧率是否一致及是否会出现访问冲突；控制模块，用于控制读操作和写操作的进行；缓存模块，用于进行读操作和写操作；若判断模块判断输出帧率和输入帧率一致，控制模块控制在缓存模块上进行读写操作，缓存模块启用一帧或一帧以上的缓存空间；若判断模块判断输出帧率和输入帧率不一致，控制模块控制在缓存模块上进行第一类读写操作，缓存模块启用两帧的缓存空间；所述第一类读写操作具体为：所述控制模块控制读指针和写指针在缓存模块的两个帧存上交替移动，读指针和写指针的移动过程中判断模块判断是否会出现访问冲突，控制模块根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
进一步地，判断模块通过读指针和写指针的相对位置进行判断是否会出现访问冲突，具体如下：若读/写指针即将跳转的帧存上没有写/读指针，判断模块的判断结果为不冲突；若读/写指针即将跳转的帧存上有写/读指针，判断模块进行第二判断操作：计算读指针和写指针的相对位置值，并比较所述相对位置值与预设的安全相对位置值作比较，若前者大于后者，判断模块的判断结果为不冲突；若前者小于后者，判断模块的判断结果为冲突；若判断结果为不冲突，控制模块控制读/写指针正常跳转，在跳转的帧存上进行读/写操作；若判断结果为冲突，控制模块控制读/写指针不进行跳转，在原帧存上重新进行读/写操作。
与现有技术相比，本发明的有益效果为：根据输入和输出的情况，自适应视频处理方式，达到最大不超过一帧的低延时，并且保证每帧画面的完整性，提高视频传输效率的同时保证传输质量。
附图说明
图1为本发明的第一类读写操作步骤图。
图2为本发明的第一类和第二类读写操作步骤图。
图3为本发明的第一类和第三类读写操作之写操作步骤图。
图4为本发明的第一类和第三类读写操作之读操作步骤图。
图5为本发明的系统结构图。
附图标记：判断模块1；控制模块2；缓存模块3；
具体实施方式
本发明附图仅用于示例性说明，不能理解为对本发明的限制。为了更好说明以下实施例，附图某些部件会有省略、放大或缩小，并不代表实际产品的尺寸；对于本领域技术人员来说，附图中某些公知结构及其说明可能省略是可以理解的。
实施例1
本实施例提供一种低延时视频传输方法，包括步骤：判断输出帧率和输入帧率是否一致，若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行读写操作；若输出帧率和输入帧率不一致，采用两帧的缓存空间进行第一类读写操作；所述第一类读写操作具体为：读指针和写指针在两个帧存上交替移动，移动过程中判断是否会出现访问冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
进一步地，所述移动过程中判断访问是否冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存的步骤具体为：读/写指针在跳转不同帧存时，若即将跳转的帧存上没有写/读指针，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若即将跳转的帧存上有写/读指针，进行第二判断操作；所述第二判断操作具体为：
预先设置安全相对位置值；计算读指针和写指针的相对位置值，若读指针和写指针的相对位置值大于或等于所述安全相对位置值，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若读指针和写指针的相对位置值小于所述安全相对位置值，读/写指针不进行跳转，在原帧存上进行读/写操作。进一步地，所述安全相对位置值根据读、写操作的速率差值进行计算确定。在读写速率波动较大的情况下，可适当增大安全相对位置值，有效避免图像分裂。
如图1所示，本实施例最优的实施方式如下：
S1：判断输出帧率和输入帧率是否一致；若输出帧率和输入帧率一致，进行S2；若输出帧率和输入帧率不一致，进行S3；
S2：采用一帧或一帧以上的缓存空间进行读写操作，所述读写操作不作限制，可为本领域技术人员熟知的普通读写过程；
S3：采用两帧的缓存空间进行第一类读写操作，具体为：
S31：读指针和写指针分居在两个帧存上，读操作和写操作在不同的帧存上进行；
S32：每完成了一帧的读/写操作后，对读/写指针即将跳转的帧存上是否存在写/读指针进行判断；若即将跳转的帧存上没有写/读指针，进行S32a；若即将跳转的帧存上有写/读指针，进行S32b；
S32a：读/写指针进行正常跳转；
S32b：进行第二判断操作：具体为：
S321：设置安全相对位置值；在其他实施例中，所述安全相对位置值可预先进行设置，从而跳过S321；
S322：计算读指针和写指针的相对位置值，并对比所述相对位置值和所述安全相对位置值，若相对位置值大于或等于安全相对位置值，进行S322a；若相对位置值小于安全相对位置值，进行S322b；
S322a：读/写指针进行正常跳转；
S322b：读/写指针不进行跳转。
实际传输情景中，读写速率往往在不断波动且互不相等，收到多方面因素的干扰，为了保证一帧画面的完整性，避免出现因读写效率不平衡导致的在一帧内超前读取或覆盖写入，控制引导写操作(写入)的写指针和引导读操作(读取)的读指针指向两帧缓存空间的同一帧存或不同帧存，从而引导写操作和读操作有序进行。所述超前读取具体为：显示一帧画面时，读取的速率较写入的速率快，读指针超越了写指针，指向了写指针未更新的缓存空间，读取了一部分旧帧的数据，导致显示的画面混杂了不同帧的图像或渲染效果，造成图像分裂。同理，所述覆盖写入具体为：写入的速率较读取的速率快，写指针完成了一帧的写入，跳转至读指针所在的帧存上开始新一帧的写入，当它在一帧内超越了读指针在读指针未读取到 缓存空间写入了新一帧数据，覆盖了读指针本应读取但仍未读取的数据，读指针后续读取的数据不属于原帧，亦会出现同一帧画面出现图像分裂。
读指针和写指针的相对位置值可以用二者在缓存空间上指向的行的差值，也可以用已读取数据量和已写入数据量的差值，反映读操作和写操作进度差异的参考量均可作为相对位置值。安全相对位置值预先进行设置，可以根据传输要求和平均读写速率进行确定，具有灵活性。当读指针不进行跳转时，相当于重新读取了刚读取的一帧画面；写指针不进行跳转时，相当于将未读取的一帧画面覆盖掉，写入最新一帧画面。可以看出，上述指针不进行跳转的情况会造成丢失帧的现象，现有技术亦存在此问题，但一般传输情景中，读写速率不会相差2倍以上，本实施例的方法可使读写指针稳定保持在安全相对位置值的距离，丢失的帧数不影响输出端的视频播放效果，对画面连续性的影响较小。
实施例2
本实施例相比于实施例1的区别在于，进一步改良了输出帧率和输入帧率一致时的传输方法，使用第二类读写操作，包括步骤：进行写操作；当完成若干量数据的写入后，开始读操作其中，所述若干量数据可以以行作单位或以字节作单位，本实施例将若干量数据设为K行，K可以不为整数；实时对比读操作的速率和写操作的速率，若前者比后者大，降低读指针的时钟频率；若前者比后者小，提高读指针的时钟频率；完成一个帧存的读操作后读指针跳转至写指针当前所在的帧存。在输出帧率和输入帧率一致时，读指针保持在写指针之后，读写指针在传输过程中大部分处于同一帧存内，读操作尾随写操作，保持在以行为单位的延时，其与所述K的数值正相关，可通过调节K值适应不同延时要求的传输环境。
如图2所示，本实施例所述的一种低延时视频传输方法最优实施方式如下：
A1：判断输出帧率和输入帧率是否一致；若输出帧率和输入帧率一致，进行A2；若输出帧率和输入帧率一致，进行A3；
A2：采用两帧或两帧以上的缓存空间进行第二类读写操作，具体为：
A21：在一个帧存上开始写操作，写入K行后开始读操作；
A22：实时对比读操作的速率和写操作的速率；若读操作的速率比写操作的速率大，进行A22a；若读操作的速率比写操作的速率小，进行A32b；若读操作的速率等于写操作的速率，进行A22a或A32b均可，不作限制；
A22a：降低读指针的时钟频率；完成一个帧存的写操作后写指针跳转至下一个帧存；完成一个帧存的读操作后读指针跳转至写指针当前所在的帧存；
A22b：提高读指针的时钟频率；完成一个帧存的写操作后写指针跳转至下一个帧存；完成一个帧存的读操作后读指针跳转至写指针当前所在的帧存；
A3：采用两帧的缓存空间进行第一类读写操作，具体为：
A31：读指针和写指针分居在两个帧存上，读操作和写操作在不同的帧存上进行；
A32：每完成了一帧的读/写操作后，对读/写指针即将跳转的帧存上是否存在写/读指针进行判断；若即将跳转的帧存上没有写/读指针，进行A32a；若即将跳转的帧存上有写/读指针，进行A32b；
A32a：读/写指针进行正常跳转；
A32b：进行第二判断操作：具体为：
A321：设置安全相对位置值；在其他实施例中，所述安全相对位置值可预先进行设置，从而跳过A321；
A322：计算读指针和写指针的相对位置值，并对比所述相对位置值和所述安全相对位置值，若相对位置值大于或等于安全相对位置值，进行A322a；若相对位置值小于安全相对位置值，进行A322b；
A322a：读/写指针进行正常跳转；
A322b：读/写指针不进行跳转。
无论输出帧率和输入帧率是否一致，上述的处理过程都控制延时等于或小于一帧，同时保证每一帧画面的完整性。需要说明的是，本实施例在输出帧率和输入帧率一致的情况下使用了两帧或两帧以上的缓存空间进行读写操作，在其他实施例的相同情况中亦可使用一帧的缓存空间，重复在一个帧存上写入多帧数据，写指针循环跳转；读指针尾随写指针，完成一 帧数据的读取后，读指针跳转，指向仅有的一个帧存上已经写入新一帧数据的行，读取新一帧的数据。
实施例3
本实施例与实施例2的区别在于，当输出帧率和输入帧率一致时，采用一帧的缓存空间进行第三类读写操作。所述第三类读写操作具体为：
在一个帧存上的初始行开始写操作，写入L行的数据后开始读操作；其中，L为整数，并且是预设值；每完成一帧数据的写操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在初始行开始进行新一帧数据的写操作；若前者比后者小，在原行进行重复写入的写操作；每完成一帧数据的读操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在原行进行读操作，对原行的数据进行重新读取，完成原行的数据重新读取后，在初始行进行新一帧数据的读操作；若前者比后者小，在初始行进行新一帧数据的读操作。所述L根据读操作速率和写操作速率的差值进行计算确定。
在第三类读写操作中，读指针和写指针的跳转方式有差异，同时发生在整个传输过程中，故对写操作和读操作进行分别论述。如图3所示，写操作的最优实施方式为：
B1：在缓存空间的一帧存上写入L行数据；其中，未完成L行数据的写入前，写指针跳转前无需进行判断，正常跳转；其中L为整数，并且是预设值；
B2：完成L行数据的写入后，写操作每完成一帧数据的写入，在写指针即将跳转至下一行时，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，进行B2a；若前者比后者小，进行B2b；
B2a：写指针正常跳转，在初始行开始新一帧数据的写操作；
B2b：写指针不进行跳转，在原行进行原帧数据的重复写入，完成后写指针跳转至初始行，在初始行开始新一帧数据的写入。
其中，当前写入一行数据需要的时间和完成读取一行数据需要的时间 时，可任意进行B2a或B2b。
如图4所示，读操作的最优实施方式为：
C1：待写操作完成L行数据的写入后，开始读操作，读指针指向写指针所在的帧存，对已经写入的数据进行读取；
C2：读操作每完成一帧数据的读取，在读指针进行即将跳转时，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，进行C2a；若前者比后者小，进行C2b；
C2a：读指针不进行跳转，在原行进行重新读取，完成后读指针跳转至初始行，在初始行开始新一帧数据的读操作；
C2b：读指针正常跳转，在初始行进行新一帧数据的读操作。
其中，当前写入一行数据需要的时间和完成读取一行数据需要的时间时，可任意进行C2a或C2b。
本实施例相较于实施例2可达到更低的延时，而不出现访问冲突造成的图像分裂现象，适用于注重延时但对画面质量要求不高的情景。上述读操作和写操作的最优实施方式均发生在已经判断为输入帧率与输出帧率一致之后，无列出具体判断输入帧率与输出帧率是否一致的过程，本领域技术人员可参考实施例2得知完整的过程。
实施例4
本实施例与实施例3的区别在于，当输出帧率和输入帧率一致时，采用一帧的缓存空间进行读写，结合第二类读写操作和第三类读写操作进行视频传输，具体如下：
预设读写时间差阀值，所述读写时间差阀值用于判断使用第二类读写操作还是第三类读写操作；在输出帧率和输入帧率一致的情况下，开始读写操作前以及开始新一帧的读写之前，计算当前写入一行所需的时间和读出一行所需的时间的差值，对比所述差值和读写时间差阀值，若前者大于后者，采用第二类操作；若前者小于后者，采用第三类读写操作。具体步骤本领域技术人员可参考实施例2和实施例3得知。
本实施例相较其他实施例，平衡了传输延时和画面质量，延时比实施例2更低，画面质量比实施例3更好。
实施例5
如图5所示，本实施例提供一种实现所述低延时视频传输方法的系统，包括：判断模块1，用于进行判断输出帧率和输入帧率是否一致及是否会出现访问冲突；控制模块2，用于控制读操作和写操作的进行；缓存模块3，用于进行读操作和写操作，输出端和输入端通过所述缓存空间进行数据传输；若判断模块1判断输出帧率和输入帧率一致，控制模块2控制在缓存模块3上进行读写操作，缓存模块3启用一帧或一帧以上的缓存空间；若判断模块1判断输出帧率和输入帧率不一致，控制模块2控制在缓存模块3上进行第一类读写操作，缓存模块3启用两帧的缓存空间；所述第一类读写操作具体为：所述控制模块2控制读指针和写指针在缓存模块3的两个帧存上交替移动，读指针和写指针的移动过程中判断模块1判断是否会出现访问冲突，控制模块2根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
进一步地，判断模块1通过读指针和写指针的相对位置进行判断是否会出现访问冲突，具体如下：若读/写指针即将跳转的帧存上没有写/读指针，判断模块1的判断结果为不冲突；若读/写指针即将跳转的帧存上有写/读指针，判断模块1进行第二判断操作：计算读指针和写指针的相对位置值，并比较所述相对位置值与预设的安全相对位置值作比较，若前者大于后者，判断模块1的判断结果为不冲突；若前者小于后者，判断模块1的判断结果为冲突；若判断结果为不冲突，控制模块2控制读/写指针正常跳转，在跳转的帧存上进行读/写操作；若判断结果为冲突，控制模块2控制读/写指针不进行跳转，在原帧存上重新进行读/写操作。
本实施例的低延时视频传输系统还可实现所述第二类读写操作和第三类读写操作。判断模块1若判断输出帧率和输入帧率一致，控制模块2则执行所述第二类读写操作或第三类读写操作。其中，执行第二类读写操作的过程具体为：缓存模块3采用一帧或一帧以上的缓存空间进行读操作和写操作，控制模块2控制在缓存模块3的一个帧存上开始写操作，写入若干量数据后开始读操作；判断模块1实时对比读操作的速率和写操作的速率，若前者比后者大，控制模块2降低读指针的时钟频率；若前者比后者 小，控制模块2提高读指针的时钟频率。完成一个帧存的写操作后，控制模块2控制写指针跳转至缓存模块3的下一个帧存；完成一个帧存的读操作后，控制模块2控制读指针跳转至写指针当前所在的帧存。在其他实施例中，缓存模块3还可采用一帧的缓存空间进行第二类读写操作。
执行第三类读写操作的过程具体为：缓存模块3采用一帧的缓存空间进行读操作和写操作，控制模块2控制在一个帧存上开始写操作，写入L行的数据后开始读操作；其中，L为整数，并且是预设值；每完成一帧数据的写操作，在写指针即将跳转至下一行时，判断模块1计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，控制模块2控制写指针正常跳转，在所述一个帧存的初始行开始写操作；若前者比后者小，控制模块2控制写指针不进行跳转，在原行重复写入原帧数据，完成后在初始行开始新一帧数据的写操作；每完成一帧数据的读操作，在读指针即将跳转至下一行时，判断模块1计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，控制模块2控制读指针不进行跳转，在原行进行重复读取，完成一次原行的重复读取后控制模块2控制读指针跳转至初始行，在初始行开始读操作；若前者比后者小，控制模块2控制读指针正常跳转，在初始行开始读操作。
实施例6
本实施例提供一种计算机储存介质，存储实现所述低延时视频传输方法的软件，使在视频传输中的输出帧率和输入帧率不一致时，保持不大于一帧的低延时，最小可达以行为单位的超低延时，并且保证每帧画面的完整性。
所述低延时视频传输方法为：判断输出帧率和输入帧率是否一致，若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行读写操作；若输出帧率和输入帧率不一致，采用两帧的缓存空间进行第一类读写操作；所述第一类读写操作具体为：读指针和写指针在两个帧存上交替移动，移动过程中判断是否会出现访问冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
显然，本发明的上述实施例仅仅是为清楚地说明本发明技术方案所作的举例，而并非是对本发明的具体实施方式的限定。凡在本发明权利要求书的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明权利要求的保护范围之内。

Claims (10)

  1. 一种低延时视频传输方法，其特征在于，包括步骤：
    判断输出帧率和输入帧率是否一致，若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行读写操作；若输出帧率和输入帧率不一致，采用两帧的缓存空间进行第一类读写操作；所述第一类读写操作具体为：
    读指针和写指针在两个帧存上交替移动，移动过程中判断是否会出现访问冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
  2. 根据权利要求1所述的一种低延时视频传输方法，其特征在于，
    所述移动过程中判断访问是否冲突，根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存的步骤具体为：
    读/写指针在跳转不同帧存时，若即将跳转的帧存上没有写/读指针，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若即将跳转的帧存上有写/读指针，进行第二判断操作；所述第二判断操作具体为：
    预先设置安全相对位置值；计算读指针和写指针的相对位置值，若读指针和写指针的相对位置值大于或等于所述安全相对位置值，读/写指针进行正常跳转，在跳转的帧存上进行读/写操作；若读指针和写指针的相对位置值小于所述安全相对位置值，读/写指针不进行跳转，在原帧存上进行读/写操作。
  3. 根据权利要求2所述的一种低延时视频传输方法，其特征在于，
    所述安全相对位置值根据读、写操作的速率差值进行计算确定。
  4. 根据权利要求1所述的一种低延时视频传输方法，其特征在于，
    若输出帧率和输入帧率一致，则采用一帧或一帧以上的缓存空间进行第二类读写操作；所述第二类读写操作具体为：
    进行写操作；
    当完成若干量数据的写入后，开始读操作；实时对比读操作的速率和写操作的速率，若前者比后者大，降低读指针的时钟频率；若前者比后者小，提高读指针的时钟频率；完成一个帧存的读操作后读指针跳转至写指 针当前所在的帧存。
  5. 根据权利要求4所述的一种低延时视频传输方法，其特征在于，
    所述若干量数据根据读操作速率和写操作速率的差值进行计算确定。
  6. 根据权利要求1所述的一种低延时视频传输方法，其特征在于，
    若输出帧率和输入帧率一致，则采用一帧的缓存空间进行第三类读写操作；所述第三类读写操作具体为：
    在一个帧存上的初始行开始写操作，写入L行的数据后开始读操作；其中，L为整数，并且是预设值；
    每完成一帧数据的写操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在初始行开始进行新一帧数据的写操作；若前者比后者小，在原行进行重复写入的写操作，完成重复写入原行数据后，在初始行开始新一帧数据的写操作；
    每完成一帧数据的读操作，计算当前写入一行数据需要的时间和完成读取一行数据需要的时间，并对二者进行比较，若前者比后者大，在原行进行读操作，对原行的数据进行重新读取，完成原行的数据重新读取后，在初始行进行新一帧数据的读操作；若前者比后者小，在初始行进行新一帧数据的读操作。
  7. 根据权利要求6所述的一种低延时视频传输方法，其特征在于，
    所述L根据读操作速率和写操作速率的差值进行计算确定。
  8. 一种低延时视频传输系统，其特征在于，包括：
    判断模块，用于进行判断输出帧率和输入帧率是否一致及是否会出现访问冲突；
    控制模块，用于控制读操作和写操作的进行；
    缓存模块，用于进行读操作和写操作；
    若判断模块判断输出帧率和输入帧率一致，控制模块控制在缓存模块上进行读写操作，缓存模块启用一帧或一帧以上的缓存空间；
    若判断模块判断输出帧率和输入帧率不一致，控制模块控制在缓存模块上进行第一类读写操作，缓存模块启用两帧的缓存空间；所述第一类读写操作具体为：
    所述控制模块控制读指针和写指针在缓存模块的两个帧存上交替移动，读指针和写指针的移动过程中判断模块判断是否会出现访问冲突，控制模块根据判断结果分配读指针进行读操作的帧存和写指针进行写操作的帧存。
  9. 根据权利要求8所述的一种低延时视频传输系统，其特征在于，
    判断模块通过读指针和写指针的相对位置进行判断是否会出现访问冲突，具体如下：
    若读/写指针即将跳转的帧存上没有写/读指针，判断模块的判断结果为不冲突；若读/写指针即将跳转的帧存上有写/读指针，判断模块进行第二判断操作：
    计算读指针和写指针的相对位置值，并比较所述相对位置值与预设的安全相对位置值作比较，若前者大于后者，判断模块的判断结果为不冲突；若前者小于后者，判断模块的判断结果为冲突；
    若判断结果为不冲突，控制模块控制读/写指针正常跳转，在跳转的帧存上进行读/写操作；若判断结果为冲突，控制模块控制读/写指针不进行跳转，在原帧存上重新进行读/写操作。
  10. 一种计算机储存介质，其上储存有计算机程序，其特征在于，
    所述计算机程序被程序执行主体执行时实现如权利1-7任一项所述的一种低延时视频传输方法。
PCT/CN2020/141937 2020-12-03 2020-12-31 一种低延时视频传输方法及其系统、储存介质 WO2022116353A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011412867.XA CN112511861B (zh) 2020-12-03 2020-12-03 一种低延时视频传输方法及其系统、储存介质
CN202011412867.X 2020-12-03

Publications (1)

Publication Number Publication Date
WO2022116353A1 true WO2022116353A1 (zh) 2022-06-09

Family

ID=74970198

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/141937 WO2022116353A1 (zh) 2020-12-03 2020-12-31 一种低延时视频传输方法及其系统、储存介质

Country Status (2)

Country Link
CN (1) CN112511861B (zh)
WO (1) WO2022116353A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115955539A (zh) * 2023-03-15 2023-04-11 广州美凯信息技术股份有限公司 基于fpga的视频帧率动态转换方法、装置及存储介质
CN116456144A (zh) * 2023-06-14 2023-07-18 合肥六角形半导体有限公司 一种无帧缓存视频流处置输出装置和方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113438502B (zh) * 2021-05-27 2022-07-12 卡莱特云科技股份有限公司 一种视频帧图像传输方法、发送卡及视频帧图像显示系统
CN113377049B (zh) * 2021-06-11 2022-09-06 西安应用光学研究所 基于fpga低延迟视频图像缓存的ddr控制方法
CN114189732B (zh) * 2021-11-27 2023-04-07 深圳曦华科技有限公司 图像数据读写控制的方法及相关装置
CN115103208A (zh) * 2022-08-26 2022-09-23 广州市保伦电子有限公司 一种视频数据的行缓存方法、行读取方法及处理系统
CN117082194B (zh) * 2023-10-16 2024-02-20 苏州元脑智能科技有限公司 视频图像处理方法、装置、系统、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102270444A (zh) * 2011-09-07 2011-12-07 东莞中山大学研究院 视频处理芯片数据流控制及帧缓存装置
CN102737691A (zh) * 2011-03-31 2012-10-17 英特赛尔美国股份有限公司 视频同步
US20160080687A1 (en) * 2014-09-12 2016-03-17 Kabushiki Kaisha Toshiba Image output device, image output method, and recording medium
CN105872432A (zh) * 2016-04-21 2016-08-17 天津大学 快速自适应帧率变换的装置与方法
CN107948546A (zh) * 2017-11-09 2018-04-20 中国航空无线电电子研究所 一种低延迟视频混合装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100493175C (zh) * 2005-08-08 2009-05-27 北京中星微电子有限公司 实现视频逐行到隔行转换的装置和转换方法
KR100747668B1 (ko) * 2005-10-31 2007-08-08 삼성전자주식회사 디스플레이 동기 신호 생성 장치를 포함하는 영상 신호수신장치 및 그 제어방법
CN101043289A (zh) * 2006-06-16 2007-09-26 华为技术有限公司 一种解除存储器读写冲突的方法和装置
CN211982026U (zh) * 2020-06-05 2020-11-20 广东高云半导体科技股份有限公司 车盲区监控电路

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102737691A (zh) * 2011-03-31 2012-10-17 英特赛尔美国股份有限公司 视频同步
CN102270444A (zh) * 2011-09-07 2011-12-07 东莞中山大学研究院 视频处理芯片数据流控制及帧缓存装置
US20160080687A1 (en) * 2014-09-12 2016-03-17 Kabushiki Kaisha Toshiba Image output device, image output method, and recording medium
CN105872432A (zh) * 2016-04-21 2016-08-17 天津大学 快速自适应帧率变换的装置与方法
CN107948546A (zh) * 2017-11-09 2018-04-20 中国航空无线电电子研究所 一种低延迟视频混合装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115955539A (zh) * 2023-03-15 2023-04-11 广州美凯信息技术股份有限公司 基于fpga的视频帧率动态转换方法、装置及存储介质
CN115955539B (zh) * 2023-03-15 2023-08-18 广州美凯信息技术股份有限公司 基于fpga的视频帧率动态转换方法、装置及存储介质
CN116456144A (zh) * 2023-06-14 2023-07-18 合肥六角形半导体有限公司 一种无帧缓存视频流处置输出装置和方法
CN116456144B (zh) * 2023-06-14 2023-09-26 合肥六角形半导体有限公司 一种无帧缓存视频流处置输出装置和方法

Also Published As

Publication number Publication date
CN112511861B (zh) 2022-05-03
CN112511861A (zh) 2021-03-16

Similar Documents

Publication Publication Date Title
WO2022116353A1 (zh) 一种低延时视频传输方法及其系统、储存介质
US7079143B2 (en) Waveform drawing system and method
US9110809B2 (en) Reducing memory traffic in DRAM ECC mode
US7739473B1 (en) Off-chip memory allocation for a unified shader
US5890210A (en) Magnetic disk apparatus and command processing method thereof
US11373622B2 (en) Dynamic display switching
US9524247B2 (en) Fractured erase system and method
CN111465966A (zh) 用于命令流优化和增强的装置和方法
CN113205779B (zh) 适用于电子墨水屏的多屏同步控制方法、装置及系统
US20080036764A1 (en) Method and apparatus for processing computer graphics data
CN104834478B (zh) 一种基于异构混合存储设备的数据写入及读取方法
CN114189732B (zh) 图像数据读写控制的方法及相关装置
US20200327010A1 (en) Performant inline ecc architecture for dram controller
CN112256203B (zh) Flash存储器的写入方法、装置、设备、介质及系统
US7805573B1 (en) Multi-threaded stack cache
CN109671127B (zh) 一种实现波形绘制的方法及装置
CN112767978B (zh) 一种ddr命令调度方法、装置、设备及介质
US11061571B1 (en) Techniques for efficiently organizing and accessing compressible data
US20220291993A1 (en) Information processing apparatus and information processing method
WO2021096883A1 (en) Methods and apparatus for adaptive display frame scheduling
WO2021000226A1 (en) Methods and apparatus for optimizing frame response
CN101866634B (zh) 显示码流的显示系统和控制方法
CN117406935B (zh) 数据读取方法及装置、读写控制器
US20020021591A1 (en) Plural line buffer type memory LSI
CN115955539B (zh) 基于fpga的视频帧率动态转换方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20964198

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20964198

Country of ref document: EP

Kind code of ref document: A1