CN117061789A - Video transmission frame, method, device and storage medium - Google Patents

Video transmission frame, method, device and storage medium Download PDF

Info

Publication number
CN117061789A
CN117061789A CN202311300597.7A CN202311300597A CN117061789A CN 117061789 A CN117061789 A CN 117061789A CN 202311300597 A CN202311300597 A CN 202311300597A CN 117061789 A CN117061789 A CN 117061789A
Authority
CN
China
Prior art keywords
image data
frame image
processing module
module
current frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311300597.7A
Other languages
Chinese (zh)
Other versions
CN117061789B (en
Inventor
袁秀阳
邹晓峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to CN202311300597.7A priority Critical patent/CN117061789B/en
Publication of CN117061789A publication Critical patent/CN117061789A/en
Application granted granted Critical
Publication of CN117061789B publication Critical patent/CN117061789B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The embodiment of the invention provides a video transmission frame, a method, equipment and a storage medium, wherein the frame comprises a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module and a register configuration module, wherein the register configuration module is respectively connected with the current frame image processing module, the previous frame image processing module and the video stream processing module.

Description

Video transmission frame, method, device and storage medium
Technical Field
The present invention relates to the field of video processing technologies, and in particular, to a video transmission frame, a video transmission method, apparatus, and storage medium.
Background
With the development of technology, the requirements for video streaming are increasingly higher, in the related art, for scenes with higher definition requirements, the server host and the remote terminal are required to transmit the original video data, and the adopted scheme is that the original video streaming data to be transmitted is firstly read from the memory of the server host, then the original video streaming data is directly transmitted to the remote terminal after being subjected to format conversion, so that the transmission delay is particularly large.
Disclosure of Invention
In view of the above problems, embodiments of the present invention have been made to provide a video transmission frame, a video transmission method, apparatus, and storage medium that overcome or at least partially solve the above problems.
In order to solve the above problems, the embodiment of the invention discloses a video transmission frame, which comprises a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module and a register configuration module;
the register configuration module is respectively connected with the current frame image processing module, the previous frame image processing module and the video stream processing module, and is used for configuring a starting address of the current frame image data, a starting address of the previous frame image data and video source parameters; the starting address of the current frame image data is sent to the current frame image processing module, the starting address of the previous frame image data is sent to the previous frame image processing module, and the video source parameters are sent to the video stream processing module;
The previous frame image processing module is connected with the frame difference processing module, and is used for reading previous frame image data from original video data according to a starting address of the previous frame image data, converting the format of the previous frame image data and transmitting the previous frame image data to the frame difference processing module;
the current frame image processing module is respectively connected with the frame difference processing module and the compression module, and is used for reading current frame image data from the original video data according to the starting address of the current frame image data, converting the format of the current frame image data and then respectively transmitting the current frame image data to the frame difference processing module and the compression module;
the frame difference processing module is connected with the video stream processing module and is used for calculating the difference between the current frame image data and the previous frame image data after format conversion to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data and transmitting the encoded frame difference image data to the video stream processing module;
The compression module is connected with the video stream processing module and is used for compressing the current frame image data after format conversion to obtain current frame compressed image data and transmitting the current frame compressed image data to the video stream processing module;
the video stream processing module is used for determining and outputting processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters.
Optionally, the framework includes a double-rate synchronous dynamic random access memory controller, the double-rate synchronous dynamic random access memory controller is connected with the current frame image processing module through an AXI bus, and the current frame image processing module is used for reading current frame image data from the original video data through the double-rate synchronous dynamic random access memory controller.
Optionally, the double-rate synchronous dynamic random access memory controller is further connected to the previous frame image processing module through an AXI bus, and the previous frame image processing module is configured to read the previous frame image data from the original video data through the double-rate synchronous dynamic random access memory controller.
Optionally, the double-rate synchronous dynamic random access memory controller is further connected with the video stream processing module, and the video stream processing module is used for outputting the processed video data through the double-rate synchronous dynamic random access memory controller.
Optionally, the format of the original video data is RGB format.
Optionally, the current frame image processing module includes a current frame image reading module, and the current frame image reading module is configured to read current frame image data in RGB format from the original video data according to a start address of the current frame image data.
Optionally, the current frame image processor module further includes a first format conversion module, where the first format conversion module is connected to the current frame image reading module, and the first format conversion module is configured to convert the current frame image data in the RGB format into current frame image data in the YUV format.
Optionally, the first format conversion module is configured to convert the current frame image data in the RGB format into the current frame image data in the YUV format according to a first frame difference conversion formula.
Optionally, the previous frame image processing module includes a previous frame image reading module, where the previous frame image reading module is configured to read previous frame image data in RGB format from the original video data according to a start address of the previous frame image data.
Optionally, the previous frame image processing module further includes a second format conversion module, where the second format conversion module is connected to the previous frame image reading module, and the second format conversion module is configured to convert the previous frame image data in the RGB format into the previous frame image data in the YUV format.
Optionally, the second format conversion module is configured to convert the previous frame image data in the RGB format into the previous frame image data in the YUV format according to a second format conversion formula.
Optionally, the video stream processing module includes a video source selection module, where the video source selection module is configured to determine the processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameter.
Optionally, the video source parameter includes the encoded frame difference image data, and the video source selection module is configured to determine that the processed video data is the encoded frame difference image data.
Optionally, the video stream processing module further includes a video stream output module, where the video stream output module is connected to the video source selection module, and the video stream output module is configured to output the processed video data.
Optionally, the frame difference processing module includes a frame difference module, and the frame difference module is configured to calculate a difference between the current frame image data and the previous frame image data after format conversion, so as to obtain frame difference image data.
Optionally, the frame difference module is configured to obtain a current pixel value in the current frame image data and a previous frame pixel value in the previous frame image data, and subtract the previous frame pixel value from the current pixel value to obtain a frame difference pixel value.
Optionally, the frame difference processing module further includes an encoding module, where the encoding module is connected to the frame difference module, and the encoding module is configured to encode the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmit the encoded frame difference image data to the video stream processing module.
The invention also discloses a video transmission method, wherein the frame comprises a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module and a register configuration module; the method comprises the following steps:
configuring a start address of the current frame image data, a start address of the previous frame image data and video source parameters through the register configuration module; the starting address of the current frame image data is sent to the current frame image processing module, the starting address of the previous frame image data is sent to the previous frame image processing module, and the video source parameters are sent to the video stream processing module;
Reading the previous frame image data from the original video data according to the starting address of the previous frame image data by the previous frame image processing module, converting the format of the previous frame image data, and transmitting the previous frame image data to the frame difference processing module;
reading current frame image data from the original video data according to the starting address of the current frame image data through the current frame image processing module, converting the format of the current frame image data, and then respectively transmitting the current frame image data to the frame difference processing module and the compression module;
calculating the difference between the current frame image data and the previous frame image data after format conversion through the frame difference processing module to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmitting the encoded frame difference image data to the video stream processing module;
compressing the current frame image data after format conversion by the compression module to obtain current frame compressed image data, and transmitting the current frame compressed image data to the video stream processing module;
and determining the processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters by the video stream processing module and outputting the processed video data.
The invention also discloses an electronic device, comprising: a processor, a memory and a computer program stored on the memory and capable of running on the processor, which when executed by the processor implements the steps of the video transmission method as described above.
The invention also discloses a computer readable storage medium, on which a computer program is stored, which when being executed by a processor implements the steps of the video transmission method as described above.
The embodiment of the invention has the following advantages:
the invention discloses a video transmission frame, which can read the image data of the current frame through a current frame image processing module to process the image data of the previous frame through the image data of the previous frame, namely, the original video data is divided into the current frame image data and the image data of the previous frame, then the current frame image data and the image data of the previous frame are transmitted to a frame difference processing module, the frame difference processing module carries out frame difference processing and compression processing on the current frame image data and the image data of the previous frame and then carries out transmission, the original video stream is subjected to frame-to-frame compression, the inter-frame redundancy is removed, a plurality of pieces of information of the previous frame can be reduced in a frame difference mode, redundant information can be removed during encoding, and the delay of transmission is reduced.
Drawings
Fig. 1 is a block diagram of a video transmission frame according to an embodiment of the present invention;
fig. 2 is a block diagram of another video transmission frame according to an embodiment of the present invention;
fig. 3 is a schematic diagram of current frame image data according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of previous frame image data according to an embodiment of the present invention;
fig. 5 is a schematic diagram of frame difference image data according to an embodiment of the present invention;
fig. 6 is a flowchart of steps of a video transmission method according to an embodiment of the present invention;
FIG. 7 is a block diagram of an electronic device according to an embodiment of the present invention;
fig. 8 is a block diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
BMC (Baseboard Management Controller) baseboard management controller can manage the running state of the server locally and remotely, support a visual console interface, and can easily carry out hardware management and fault elimination on the server.
The KVM is a module in the BMC, the host transmits the image data to the PCIe controller through the PCIe (Peripheral Component Interconnect express, high-speed serial computer expansion bus standard), the PCIe controller separates the video data to the VGA (Video Graphics Array ) module, the VGA module transmits the video data to the local module for display, and stores the digital image in the DDR (Double Data Rate Synchronous Dynamic Random Access Memory, double-rate synchronous dynamic random access memory), the video compression module reads the image data in the DDR according to the video frame rate (how many images are in one second), compresses the image data by Jpeg (Joint Photographic Experts Group, standard image file format), and re-stores the compressed data in the DDR, the ethernet controller reads the compressed image data in the DDR, and transmits the compressed image data to the remote terminal through ethernet after packaging, and the remote terminal decompresses the image data by Jpeg.
The related art is that video data of a server host is compressed into image data in a JPEG format and transmitted to a remote terminal, a large amount of image details are lost in the compression process, after the image of the remote terminal is recovered, the lost details cannot be recovered, so that the definition of a video is poor, and for an application scene with a high-definition image, the image transmission mode cannot meet the requirements. For scenes with higher definition, the original YUV data needs to be transmitted to a remote terminal, so that the data size is very large, and delay jamming is easy to occur.
One of the core ideas of the embodiment of the invention is that the original video data can be divided into the current frame image data and the previous frame image data, then the current frame image data and the previous frame image data are subjected to frame difference processing and compression processing and then are transmitted, the original video stream is subjected to frame-to-frame compression, the inter-frame redundancy is removed, a plurality of previous frame information can be reduced in a frame difference mode, redundant information can be removed during encoding, and the transmission delay is reduced.
Referring to fig. 1, a block diagram of a video transmission frame provided by an embodiment of the present invention is shown, where the frame includes a current frame image processing module 101, a previous frame image processing module 102, a frame difference processing module 103, a compression module 104, a video stream processing module 105, and a register configuration module 106;
the register configuration module 106 is respectively connected with the current frame image processing module 101, the previous frame image processing module 102 and the video stream processing module 105, and the register configuration module 106 is used for configuring the start address of the current frame image data, the start address of the previous frame image data and the video source parameters; the start address of the image data of the current frame is sent to the image processing module 101 of the current frame, the start address of the image data of the previous frame is sent to the image processing module 102 of the previous frame, and the video source parameters are sent to the video stream processing module 105;
The previous frame image processing module 102 is connected with the frame difference processing module 103, and the previous frame image processing module 102 is used for reading previous frame image data from original video data according to a starting address of the previous frame image data, performing format conversion on the previous frame image data, and transmitting the previous frame image data to the frame difference processing module 103;
in the embodiment of the present invention, the original video data may be unprocessed video data stored in the external DDR, and after the previous frame image processing module 102 receives the start address of the previous frame image data transmitted by the register configuration module 106, the previous frame image data may be read from the start address of the previous frame image data of the external DDR.
The current frame image processing module 101 is respectively connected with the frame difference processing module 103 and the compression module 104, and the current frame image processing module 101 is used for reading current frame image data from original video data according to a start address of the current frame image data, performing format conversion on the current frame image data, and then respectively transmitting the current frame image data to the frame difference processing module 103 and the compression module 104.
In the embodiment of the present invention, after the current frame image processing module 101 receives the start address of the current frame image data transmitted by the register configuration module 106, the current frame image data may be read from the start address of the current frame image data of the external DDR.
The frame difference processing module 103 is connected with the video stream processing module 105, and the frame difference processing module 103 is used for calculating the difference between the current frame image data and the previous frame image data after format conversion to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmitting the encoded frame difference image data to the video stream processing module 105;
the compression module 104 is connected to the video stream processing module 105, and the compression module 104 is configured to compress the current frame image data after format conversion, obtain current frame compressed image data, and transmit the current frame compressed image data to the video stream processing module 105.
The video stream processing module 105 is configured to determine and output processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters.
The invention discloses a video transmission frame, which can divide original video data into current frame image data and previous frame image data, then carry out frame difference processing and compression processing on the current frame image data and the previous frame image data, then carry out transmission, carry out frame-to-frame compression on the original video stream, remove the inter-frame redundancy, reduce the information of a plurality of previous frames in a frame difference mode, remove redundant information during encoding and reduce the delay of transmission.
In one embodiment of the present invention, as shown in fig. 2, another video transmission framework of the present invention is shown, where the framework further includes a double rate synchronous dynamic random access memory controller 107, where the double rate synchronous dynamic random access memory controller 107 is connected to the current frame image processing module 101 through an AXI bus, and the current frame image processing module 101 is configured to read the current frame image data from the original video data through the double rate synchronous dynamic random access memory controller 107.
In the embodiment of the present invention, the current frame image processing module 101 may read the current frame image data from the start address of the current frame image data in the external DDR through the DDR controller 107.
In one embodiment of the present invention, as shown in fig. 2, the DDR controller 107 is further connected to the previous frame image processing module 102 through an AXI bus, and the previous frame image processing module 102 is configured to read the previous frame image data from the original video data through the DDR controller 107.
In the embodiment of the invention, the original video data may include multi-frame image data, and the previous frame image processing module may read the previous frame image data from a start address of the previous frame image data in the original video data of the external DDR through the DDR controller.
In one embodiment of the present invention, as shown in fig. 2, the DDR controller 107 is further connected to the video stream processing module 105, and the video stream processing module 105 is configured to output the processed video data through the DDR controller 107.
In the embodiment of the present invention, the video stream processing module 105 may write the processed video data into the designated storage location of the external DDR through the DDR controller, and the storage location is not limited herein.
In one embodiment of the present invention, the format of the original video data may be an RGB format.
In one embodiment of the present invention, the current frame image processing module 101 may include a current frame image reading module 1011, and the current frame image reading module 1011 may read the current frame image data in RGB format from the original video data according to the start address of the current frame image data.
In one embodiment of the present invention, the current frame image processing module 101 further includes a first format conversion module 1012, where the first format conversion module 1012 is connected to the current frame image reading module 1011, and the first format conversion module 1012 may convert the current frame image data in RGB format into the current frame image data in YUV format, and then transmit the current frame image data to the frame difference processing module 103 and the compression module 104.
In one embodiment of the present invention, the first format conversion module 1012 is configured to convert current frame image data in RGB format into current frame image data in YUV format according to a first frame difference conversion formula.
In the embodiment of the present invention, the first format conversion module 1012 may convert the current frame image data in RGB format into the current frame image data in YUV format through the following 3 formulas, specifically, may calculate the gray value Y of the current frame image data through the formula (1)
Y= (0.257×r) + (0.504×g) + (0.098×b) +16 formula (1)
And (3) calculating to obtain the color value of the current frame image data through a formula (2).
U= - (0.148×r) - (0.291×g) + (0.439×b) +128 formula (2)
And (3) calculating to obtain the color saturation value of the current frame image data through a formula (3).
V= (0.439×r) - (0.368×g) - (0.071×b) +128 formula (3)
In one embodiment of the present invention, the previous frame image processing module 102 includes a previous frame image reading module 1021, where the previous frame image reading module 1021 is configured to read the previous frame image data in RGB format from the original video data according to the start address of the previous frame image data, and specifically, may read the previous frame image data in RGB format from the start address of the previous frame image data in the original video data of DDR through the DDR controller.
In an embodiment of the present invention, the previous frame image processing module 102 further includes a second format conversion module 1022, where the second format conversion module 1022 is connected to the previous frame image reading module 1021, and is configured to convert the previous frame image data in RGB format into the previous frame image data in YUV format.
In this embodiment of the present invention, as shown in fig. 2, the previous frame image processing module 102 further includes a second format conversion module 1022, where the second format conversion module 1022 is connected to the previous frame image reading module 1021, and can convert the previous frame image data in RGB format read by the previous frame image reading module into the previous frame image data in YUV format, and then transmit the previous frame image data to the frame difference processing module 103.
In one embodiment of the present invention, the second format conversion module 1022 is configured to convert the previous frame image data in RGB format into the previous frame image data in YUV format according to the second format conversion formula.
In the embodiment of the invention, the second format conversion module can convert the previous frame of image data into the previous frame of image data in the YUV format through the formulas (1) - (3).
In one embodiment of the present invention, the video stream processing module 105 includes a video source selection module 1051, where the video source selection module 1051 is configured to determine processed video data from the current frame compressed image data and the encoded frame difference image data according to video source parameters.
In the embodiment of the present invention, the video source selection module 1051 may select video data to be uploaded, that is, processed video data, from the compressed image data of the current frame and the encoded frame difference image data, and the video source parameters are used to characterize the type of video data to be output by the video stream processing module 105, and may be set according to the user requirement;
in one embodiment of the present invention, the video source parameters include encoded frame difference image data, and the video source selection module 1051 is configured to determine the processed video data as encoded frame difference image data.
In one example, the video source selection module 1051 selects the video data to be uploaded as the current frame compressed image data, and the processed video data is the current frame compressed image data, and in another example, if the video source selection module selects the video data to be uploaded as the encoded frame difference image data, the processed video data is the encoded frame difference image data.
In one embodiment of the present invention, the video stream processing module 105 further includes a video stream output module 1052, where the video stream output module 1052 is connected to the video source selection module 1051, and the video stream output module 1052 is configured to output the processed video data.
In the embodiment of the present invention, the video stream output module 1052 may write the processed video data determined by the video source selection module 1051 into the designated storage location of the external DDR.
In one embodiment of the present invention, the frame difference processing module 103 includes a frame difference module 1031, where the frame difference module 1031 is configured to calculate a difference between the current frame image data and the previous frame image data after the format conversion, to obtain frame difference image data.
In this embodiment of the present invention, as shown in fig. 2, the frame difference module 1031 is respectively connected to the second format conversion module 1022 and the first format conversion module 1012, and can calculate the difference between the current frame image data after format conversion and the previous frame image data after format conversion according to the obtained current frame image data after format conversion, so as to obtain frame difference image data.
In one embodiment of the present invention, the frame difference module 1031 is configured to obtain a current pixel value in the current frame image data and a previous frame pixel value in the previous frame image data, and subtract the previous frame pixel value from the current pixel value to obtain a frame difference pixel value.
Specifically, as shown in fig. 3, a schematic diagram of current frame image data provided by the embodiment of the present invention is shown, where a pixel point of a current frame is Ps (Xs, ys), as shown in fig. 4, a schematic diagram of previous frame image data provided by the embodiment of the present invention is shown, a pixel point of a previous frame is Pm (Xs, ys), ps (Xs, ys) -Pm (Xs, ys) may be obtained to obtain a frame difference pixel point value, as shown in fig. 5, a schematic diagram of frame difference image data provided by the embodiment of the present invention is shown, and a pixel point after a frame difference is Pd (Xs, ys).
The YUV values of Ps (Xs, ys) are Y0, U0, V0, respectively, the YUV values of Pm (Xs, ys) are Y1, U1, V1, respectively, the YUV values of Pd (Xs, ys) are y=y0-Y1, u=u0-U1, v=v0-V1, and the YUV values range from 0 to 255, assuming that the bit widths of the YUV values are all 8.
In one example, the pixel value of the current frame of the same pixel is 5, and the pixel value of the previous frame is 251, and then the frame difference of the pixel is 5-251=10 (9' b100001010, the 9 th bit is omitted).
In another example, the pixel value of the current frame of the same pixel is 251, and the pixel value of the previous frame is 5, and then the frame difference of the pixel is 251-5=246 (9' b 01110110, the 9 th bit is omitted).
It should be noted that, when the current frame is the 1 st frame, no image data is generated in the previous frame, that is, YUV values of the previous frame are all 0.
In an embodiment of the present invention, the frame difference processing module 103 further includes an encoding module 1032, where the encoding module 1032 is connected to the frame difference module 1031, and the encoding module 1032 is configured to encode the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmit the encoded frame difference image data to the video stream processing module 105.
In the embodiment of the present invention, the encoding module 1032 may encode the frame difference image data output by the frame difference module 1031 according to the number of preset pixel values in the frame difference image data, and then transmit the encoded frame difference image data to the video stream processing module 105, and the video stream processing module 105 may select the image data to be uploaded according to the received encoded frame difference image data and the compressed current frame YUV image data.
Specifically, table 1 shows coding information of frame difference image data provided by an embodiment of the present invention.
TABLE 1
The method and the device for encoding the frame difference image data from the first pixel of one frame of image start to encode, directly output the pixel value if the pixel value is not 0, and output 00, the count value and 00 when the number of the pixels after starting to count is continuously 0 until the pixel value is not 0 if the pixel value is not 0, and sequentially process all the pixel values of one frame of image until one frame of image is finished, as shown in table 2, the encoding information of the encoded frame difference image data provided by the embodiment of the invention can remove redundant information during encoding, and reduce the transmission delay.
TABLE 2
The invention discloses a video transmission frame, which can divide original video data into current frame image data and previous frame image data, then carry out frame difference processing and compression processing on the current frame image data and the previous frame image data, then carry out transmission, carry out frame-to-frame compression on the original video stream, remove the inter-frame redundancy, reduce the information of a plurality of previous frames in a frame difference mode, remove redundant information during encoding and reduce the delay of transmission.
Referring to fig. 6, a flowchart illustrating steps of a video transmission method according to an embodiment of the present invention is applied to a video transmission frame, where the frame includes a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module, and a register configuration module; the method may comprise the steps of:
step 201, configuring a start address of current frame image data, a start address of previous frame image data and video source parameters through a register configuration module; the method comprises the steps of sending a starting address of current frame image data to a current frame image processing module, sending a starting address of previous frame image data to a previous frame image processing module, and sending video source parameters to a video stream processing module;
step 202, reading the previous frame image data from the original video data according to the start address of the previous frame image data by the previous frame image processing module, performing format conversion on the previous frame image data, and transmitting the previous frame image data to the frame difference processing module;
step 203, reading the current frame image data from the original video data according to the start address of the current frame image data by the current frame image processing module, performing format conversion on the current frame image data, and then respectively transmitting the current frame image data to the frame difference processing module and the compression module;
Step 204, calculating the difference between the current frame image data and the previous frame image data after format conversion by a frame difference processing module to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmitting the encoded frame difference image data to a video stream processing module;
step 205, compressing the current frame image data after format conversion by a compression module to obtain current frame compressed image data, and transmitting the current frame compressed image data to a video stream processing module;
and 206, determining the processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters by a video stream processing module and outputting the processed video data.
The invention discloses a video transmission method, which can divide original video data into current frame image data and previous frame image data, then carry out frame difference processing and compression processing on the current frame image data and the previous frame image data, then carry out transmission, carry out frame-to-frame compression on the original video stream, remove the inter-frame redundancy, reduce the information of a plurality of previous frames in a frame difference mode, remove redundant information during encoding, and reduce the delay of transmission.
In one embodiment of the present invention, the frame includes a DDR controller, and the double rate synchronous dynamic random access memory controller is connected to a current frame image processing module through an AXI bus, where the current frame image processing module is configured to read current frame image data from original video data through the DDR controller.
In one embodiment of the present invention, the double rate synchronous dynamic random access memory controller is further connected to a previous frame image processing module through an AXI bus, where the previous frame image processing module is configured to read the previous frame image data from the original video data through the DDR controller.
In one embodiment of the present invention, the double rate synchronous dynamic random access memory controller is further connected to a video stream processing module through an AXI bus, and the video stream processing module is configured to output the processed video data through the DDR controller.
In one embodiment of the present invention, the format of the original video data is an RGB format.
In one embodiment of the present invention, the current frame image processing module includes a current frame image reading module, and the current frame image reading module is configured to read current frame image data in RGB format from the original video data according to a start address of the current frame image data.
In an embodiment of the present invention, the current frame image processor module further includes a first format conversion module, where the first format conversion module is configured to convert current frame image data in RGB format into current frame image data in YUV format.
In one embodiment of the present invention, the first format conversion module is configured to convert current frame image data in RGB format into current frame image data in YUV format according to a first frame difference conversion formula.
In one embodiment of the present invention, the previous frame image processing module includes a previous frame image reading module for reading previous frame image data in RGB format from the original video data according to a start address of the previous frame image data.
In an embodiment of the present invention, the previous frame image processing module further includes a second format conversion module, where the second format conversion module is configured to convert the previous frame image data in RGB format into the previous frame image data in YUV format.
In one embodiment of the present invention, the second format conversion module is configured to convert the previous frame image data in RGB format into the previous frame image data in YUV format according to the second format conversion formula.
In one embodiment of the present invention, the video stream processing module includes a video source selection module for determining processed video data from the current frame compressed image data and the encoded frame difference image data according to video source parameters.
In one embodiment of the present invention, the video source parameter includes encoded frame difference image data, and the video source selection module is configured to determine the processed video data as encoded frame difference image data.
In an embodiment of the present invention, the video stream processing module further includes a video stream output module, where the video stream output module is configured to output the processed video data.
In one embodiment of the present invention, the frame difference processing module includes a frame difference module, and the frame difference module is configured to calculate a difference between the current frame image data and the previous frame image data after format conversion, to obtain frame difference image data.
In one embodiment of the present invention, the frame difference module is configured to obtain a current pixel value in the current frame image data and a previous frame pixel value in the previous frame image data, and subtract the previous frame pixel value from the current pixel value to obtain a frame difference pixel value.
In an embodiment of the present invention, the frame difference processing module further includes an encoding module, where the encoding module is configured to encode the frame difference image data according to a number of preset pixel values in the frame difference image data, and transmit the encoded frame difference image data to the video stream processing module.
The invention discloses a video transmission method, which can divide original video data into current frame image data and previous frame image data, then carry out frame difference processing and compression processing on the current frame image data and the previous frame image data, then carry out transmission, carry out frame-to-frame compression on the original video stream, remove the inter-frame redundancy, reduce the information of a plurality of previous frames in a frame difference mode, remove redundant information during encoding, and reduce the delay of transmission.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 7, a block diagram of an electronic device 30 according to an embodiment of the present invention is provided, including:
the processor 301, the memory 302, and the computer program 3021 stored in the memory 302 and capable of running on the processor 301, where the computer program 3021 when executed by the processor 301 implements the processes of the video transmission method embodiment described above and achieves the same technical effects, and for avoiding repetition, details are not repeated here.
Referring to fig. 8, a block diagram of a computer readable storage medium 40 according to an embodiment of the present invention is shown, where a computer program 401 is stored on the computer readable storage medium 40, and when the computer program 401 is executed by a processor, the processes of the above-mentioned embodiments of the video transmission method are implemented, and the same technical effects can be achieved, so that repetition is avoided and redundant description is omitted.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has described in detail a video transmission frame, a video transmission method, apparatus and storage medium, and specific examples have been provided herein to illustrate the principles and embodiments of the present invention, the above examples being provided only to assist in understanding the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (20)

1. The video transmission framework is characterized by comprising a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module and a register configuration module;
the register configuration module is respectively connected with the current frame image processing module, the previous frame image processing module and the video stream processing module, and is used for configuring a starting address of the current frame image data, a starting address of the previous frame image data and video source parameters; the starting address of the current frame image data is sent to the current frame image processing module, the starting address of the previous frame image data is sent to the previous frame image processing module, and the video source parameters are sent to the video stream processing module;
The previous frame image processing module is connected with the frame difference processing module, and is used for reading previous frame image data from original video data according to a starting address of the previous frame image data, converting the format of the previous frame image data and transmitting the previous frame image data to the frame difference processing module;
the current frame image processing module is respectively connected with the frame difference processing module and the compression module, and is used for reading current frame image data from the original video data according to the starting address of the current frame image data, converting the format of the current frame image data and then respectively transmitting the current frame image data to the frame difference processing module and the compression module;
the frame difference processing module is connected with the video stream processing module and is used for calculating the difference between the current frame image data and the previous frame image data after format conversion to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data and transmitting the encoded frame difference image data to the video stream processing module;
The compression module is connected with the video stream processing module and is used for compressing the current frame image data after format conversion to obtain current frame compressed image data and transmitting the current frame compressed image data to the video stream processing module;
the video stream processing module is used for determining and outputting processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters.
2. The framework of claim 1, wherein the framework comprises a double rate synchronous dynamic random access memory controller connected to the current frame image processing module through an AXI bus, the current frame image processing module configured to read current frame image data from the raw video data through the double rate synchronous dynamic random access memory controller.
3. The framework of claim 2 wherein the double rate synchronous dynamic random access memory controller is further coupled to the previous frame image processing module via an AXI bus, the previous frame image processing module being configured to read previous frame image data from the original video data via the double rate synchronous dynamic random access memory controller.
4. The framework of claim 2, wherein the double rate synchronous dynamic random access memory controller is further coupled to the video stream processing module via an AXI bus, the video stream processing module configured to output the processed video data via the double rate synchronous dynamic random access memory controller.
5. The framework of claim 1, wherein the format of the raw video data is RGB format.
6. The framework of claim 5, wherein the current frame image processing module includes a current frame image reading module for reading current frame image data in RGB format from the original video data according to a start address of the current frame image data.
7. The framework of claim 5, wherein the current frame image processor module further comprises a first format conversion module, the first format conversion module being coupled to the current frame image reading module, the first format conversion module being configured to convert the current frame image data in RGB format to current frame image data in YUV format.
8. The framework of claim 7, wherein the first format conversion module is configured to convert the current frame image data in RGB format to the current frame image data in YUV format according to a first frame difference conversion formula.
9. The frame of claim 5, wherein the previous frame image processing module includes a previous frame image reading module for reading previous frame image data in RGB format from the original video data according to a start address of the previous frame image data.
10. The framework of claim 9, wherein the previous frame image processing module further comprises a second format conversion module, the second format conversion module being connected to the previous frame image reading module, the second format conversion module being configured to convert the previous frame image data in RGB format to the previous frame image data in YUV format.
11. The framework of claim 10, wherein the second format conversion module is configured to convert the previous frame image data in RGB format to the previous frame image data in YUV format according to a second format conversion formula.
12. The framework of claim 1 wherein the video stream processing module comprises a video source selection module for determining the processed video data from the current frame compressed image data and the encoded frame difference image data based on the video source parameters.
13. The framework of claim 12 wherein the video source parameters include the encoded frame difference image data and the video source selection module is configured to determine the processed video data as the encoded frame difference image data.
14. The framework of claim 12, wherein the video stream processing module further comprises a video stream output module coupled to the video source selection module, the video stream output module configured to output the processed video data.
15. The framework of claim 1, wherein the frame difference processing module includes a frame difference module configured to calculate a difference between the current frame image data and the previous frame image data after format conversion to obtain frame difference image data.
16. The framework of claim 15, wherein the frame difference module is configured to obtain a current pixel value in current frame image data and a previous frame pixel value in the previous frame image data, and subtract the previous frame pixel value from the current pixel value to obtain a frame difference pixel value.
17. The frame of claim 16, wherein the frame difference processing module further comprises an encoding module, the encoding module is connected to the frame difference module, the encoding module is configured to encode the frame difference image data according to a number of preset pixel values in the frame difference image data, and transmit the encoded frame difference image data to the video stream processing module.
18. The video transmission method is characterized by being applied to a video transmission frame, wherein the frame comprises a current frame image processing module, a previous frame image processing module, a frame difference processing module, a compression module, a video stream processing module and a register configuration module; the method comprises the following steps:
configuring a start address of the current frame image data, a start address of the previous frame image data and video source parameters through the register configuration module; the starting address of the current frame image data is sent to the current frame image processing module, the starting address of the previous frame image data is sent to the previous frame image processing module, and the video source parameters are sent to the video stream processing module;
reading the previous frame image data from the original video data according to the starting address of the previous frame image data by the previous frame image processing module, converting the format of the previous frame image data, and transmitting the previous frame image data to the frame difference processing module;
Reading current frame image data from the original video data according to the starting address of the current frame image data through the current frame image processing module, converting the format of the current frame image data, and then respectively transmitting the current frame image data to the frame difference processing module and the compression module;
calculating the difference between the current frame image data and the previous frame image data after format conversion through the frame difference processing module to obtain frame difference image data, encoding the frame difference image data according to the number of preset pixel values in the frame difference image data, and transmitting the encoded frame difference image data to the video stream processing module;
compressing the current frame image data after format conversion by the compression module to obtain current frame compressed image data, and transmitting the current frame compressed image data to the video stream processing module;
and determining the processed video data from the current frame compressed image data and the encoded frame difference image data according to the video source parameters by the video stream processing module and outputting the processed video data.
19. An electronic device, comprising: a processor, a memory and a computer program stored on the memory and capable of running on the processor, which when executed by the processor carries out the steps of the video transmission method as claimed in claim 18.
20. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the video transmission method as claimed in claim 18.
CN202311300597.7A 2023-10-09 2023-10-09 Video transmission frame, method, device and storage medium Active CN117061789B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311300597.7A CN117061789B (en) 2023-10-09 2023-10-09 Video transmission frame, method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311300597.7A CN117061789B (en) 2023-10-09 2023-10-09 Video transmission frame, method, device and storage medium

Publications (2)

Publication Number Publication Date
CN117061789A true CN117061789A (en) 2023-11-14
CN117061789B CN117061789B (en) 2024-02-09

Family

ID=88669547

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311300597.7A Active CN117061789B (en) 2023-10-09 2023-10-09 Video transmission frame, method, device and storage medium

Country Status (1)

Country Link
CN (1) CN117061789B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070110155A1 (en) * 2005-11-15 2007-05-17 Sung Chih-Ta S Method and apparatus of high efficiency image and video compression and display
CN102333174A (en) * 2011-09-02 2012-01-25 深圳市万兴软件有限公司 Video image processing method and device for the same
CN110944197A (en) * 2018-09-25 2020-03-31 中国移动通信有限公司研究院 Method and device for coding images and audios
CN112632426A (en) * 2020-12-22 2021-04-09 新华三大数据技术有限公司 Webpage processing method and device
CN113554008A (en) * 2021-09-18 2021-10-26 深圳市安软慧视科技有限公司 Method and device for detecting static object in area, electronic equipment and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070110155A1 (en) * 2005-11-15 2007-05-17 Sung Chih-Ta S Method and apparatus of high efficiency image and video compression and display
CN102333174A (en) * 2011-09-02 2012-01-25 深圳市万兴软件有限公司 Video image processing method and device for the same
CN110944197A (en) * 2018-09-25 2020-03-31 中国移动通信有限公司研究院 Method and device for coding images and audios
CN112632426A (en) * 2020-12-22 2021-04-09 新华三大数据技术有限公司 Webpage processing method and device
CN113554008A (en) * 2021-09-18 2021-10-26 深圳市安软慧视科技有限公司 Method and device for detecting static object in area, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN117061789B (en) 2024-02-09

Similar Documents

Publication Publication Date Title
CN101729840A (en) Storage processing method utilizing video image characteristic sequence
US20140267583A1 (en) Augmented Video Calls on Mobile Devices
CN106412473B (en) Image processing device, video subsystem and video processing circuit
WO2018129924A1 (en) Image processing method and electronic device
CN113573072B (en) Image processing method and device and related components
TW201132110A (en) Bitstream syntax for graphics-mode compression in wireless HD 1.1
CN205005201U (en) Audio frequency and video playback devices
CN117061789B (en) Video transmission frame, method, device and storage medium
CN111741343B (en) Video processing method and device and electronic equipment
CN111970564A (en) Optimization method and device for HDR video display processing, storage medium and terminal
US11838503B2 (en) Video processing method and apparatus, storage medium, and electronic device
WO2021224056A1 (en) 3d scene transmission with alpha layers
CN106658056B (en) Nonlinear editing system, device and method
CN113423016A (en) Video playing method, device, terminal and server
KR101937718B1 (en) Image Processing Method and Apparatus Using Bus Protocol Based Valid Pixel
CN107241601B (en) Image data transmission method, device and terminal
CN106954073B (en) Video data input and output method, device and system
CN104581175A (en) Image processing device and method
US7233366B2 (en) Method and apparatus for sending and receiving and for encoding and decoding a telop image
US20240048734A1 (en) Image processing method and image processing device for enhancing image processing efficiency
US20240048740A1 (en) Image processing method and image processing device for enhancing image processing efficiency
CN118338002B (en) BMC video compression method, device and system and baseboard management controller
CN115063327B (en) Image processing method and device, and video processing method and device
CN114205646B (en) Data processing method, device, electronic equipment and storage medium
CN115529465A (en) Image transmission method, system and device and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant