CN115002087A - Audio and video data transmission method, system, device and storage medium - Google Patents
Audio and video data transmission method, system, device and storage medium Download PDFInfo
- Publication number
- CN115002087A CN115002087A CN202210609559.9A CN202210609559A CN115002087A CN 115002087 A CN115002087 A CN 115002087A CN 202210609559 A CN202210609559 A CN 202210609559A CN 115002087 A CN115002087 A CN 115002087A
- Authority
- CN
- China
- Prior art keywords
- data
- network card
- audio
- intelligent network
- video data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000004891 communication Methods 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims description 25
- 238000007781 pre-processing Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 17
- 238000005538 encapsulation Methods 0.000 claims description 9
- 238000004806 packaging method and process Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 13
- 230000008569 process Effects 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/80—Responding to QoS
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
Abstract
The application discloses an audio and video data transmission method, system, device and storage medium, and relates to the field of information media. According to the audio and video data transmission method, the original common network card is replaced by the intelligent network card, the data after being unpacked are preprocessed, the processed data are subjected to hardware decoding through hardware decoding equipment, the data are repackaged through the intelligent network card, and the data are transmitted to the virtual machine through driving. Compared with the prior art that the decoding hardware is used for directly communicating with the virtual machine, in this embodiment, the multilayer virtual switch function of the intelligent network card is used to implement communication between the virtual machine and the hardware decoding device in the host machine by using the intelligent network card as an intermediate medium, and all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card for decoding, so that the problem of data transmission delay is not caused.
Description
Technical Field
The present application relates to the field of information media, and in particular, to an audio/video data transmission method, system, device, and storage medium.
Background
In recent years, with the development of self-media technology and the increasing information quantity, the original volume of high-quality audio and video is increasing, for example, 720P 30fps video, one pixel is about 3 bytes, 1280 × 720 × 33 × 3/1024/1024 — 87MB data is generated every second, and the data volume of one minute is 5.22GB, so that the video with huge volume cannot be directly transmitted in the network.
The existing audio and video data are compressed by a coding technology and then transmitted to a cloud data center, and are usually decoded by using special hardware, wherein the special hardware has the advantages of high bandwidth, high performance and low delay, and the special hardware is used for carrying out logic Processing on the audio and video data, so that the delay can be greatly reduced compared with the Processing of a Central Processing Unit (CPU), and the resource consumption is reduced. However, in the cloud computing architecture, multiple virtual machines on one host cannot simultaneously use dedicated hardware for decoding, and only decoding can be performed through the dedicated hardware, which results in data transmission delay.
In view of the above technologies, it is an urgent need for those skilled in the art to find an audio and video data transmission method capable of solving the problem of data transmission delay.
Disclosure of Invention
The application aims to provide an audio and video data transmission method, system, device and storage medium, so as to solve the problem that under the framework of cloud computing, a plurality of virtual machines on one host cannot use special hardware for decoding at the same time, and only single decoding can be carried out, so that data transmission delay is caused.
In order to solve the above technical problem, the present application provides an audio and video data transmission method, which is applied to a host machine using an intelligent network card, and includes:
decapsulating a data packet, wherein the data packet is a data packet containing audio and video data to be transmitted;
preprocessing the decapsulated data, and performing hardware decoding on the processed data through hardware decoding equipment;
and repackaging data through the intelligent network card, and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program running on the intelligent network card.
Preferably, before the preprocessing the decapsulated data and performing hardware decoding on the processed data by a hardware decoding device, the method further includes:
and arranging the data to be processed according to a priority queue, wherein the priority order of the priority queue is determined according to the size of the data.
Preferably, before the decapsulating the data packet, the method further includes:
and unloading the TCP/UDP protocol of the network flow of the host machine.
Preferably, the preprocessing the decapsulated data transmission includes:
and deleting the label of the corresponding network ID in the data, and setting the corresponding relation between the ID of the virtual machine and the data.
Preferably, after the data is repackaged through the intelligent network card and is transmitted to the virtual machine through the driver, the method further includes:
reconstructing codes of an original virtual machine, and recompiling the flow preprocessing program of the intelligent network card by adopting a GCC compiler to generate an executable program capable of running on the intelligent network card, wherein the executable program is audio and video corresponding to the audio and video data.
Preferably, the memory device of the intelligent network card is used as a data buffer and a queue, and the PCIE interface is used as the intelligent network card to communicate with the hardware decoding device.
Preferably, before the decapsulating the data packet, the method further includes:
and creating a communication cache of the hardware decoding device and the interface end of the intelligent network card, and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.
In order to solve the above problem, the present application provides an audio/video data transmission device, including:
the de-encapsulation module is used for de-encapsulating a data packet, wherein the data packet contains audio and video data to be transmitted;
the decoding module is used for preprocessing the decapsulated data transmission and performing hardware decoding on the processed data;
and the packaging module is used for repackaging the data through the intelligent network card and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program capable of running on the intelligent network card.
Preferably, the apparatus further comprises:
and the arrangement module is used for arranging the data to be processed according to a priority queue, and the priority order of the priority queue is determined according to the size of the data.
Preferably, the apparatus further comprises:
and the unloading module is used for unloading the TCP/UDP protocol of the network flow of the host machine.
Preferably, the apparatus further comprises:
in order to solve the above problem, the present application further provides an audio/video data transmission device, which is characterized by comprising a memory for storing a computer program;
a processor for implementing the steps of the audio-video data transmission method as described above when executing the computer program.
Preferably, the apparatus further comprises:
and the reconstruction module is used for reconstructing the original virtual machine codes, and adopting a GCC compiler to recompile the flow preprocessing program of the intelligent network card to generate an executable program which can run on the intelligent network card, wherein the executable program is audio and video corresponding to the audio and video data.
Preferably, the apparatus further comprises:
and the cache module is used for creating a communication cache of the hardware decoding device and the interface end of the intelligent network card and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.
In order to solve the above problem, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the audio and video data transmission method as described above.
According to the audio and video data transmission method, an original common network card is replaced by the intelligent network card, data after being unpacked are preprocessed, the processed data are subjected to hardware decoding through hardware decoding equipment, the data are repackaged through the intelligent network card and transmitted to the virtual machine through driving, so that an executable program which can run on the intelligent network card can be generated, the data are subjected to hardware decoding through the intelligent network card, the hardware decoding equipment can conduct hardware decoding on the data in the data packet, and finally after the decoding is completed, the data are packaged through the intelligent network card, and the data are transmitted to the virtual machine through driving. Compared with the prior direct communication between the decoding hardware and the virtual machine, in the embodiment, the multilayer virtual switch function of the intelligent network card is adopted, and the intelligent network card is used as an intermediate medium to realize the communication between the virtual machine and the hardware decoding device in the host machine, and as all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card to be decoded together, and are transmitted back to the intelligent network card to be repackaged after being decoded, and the data corresponding to the audio and video files are transmitted to the corresponding virtual machine, the problem of data transmission delay is avoided.
The audio and video data transmission device and the computer readable storage medium provided by the application correspond to the audio and video data transmission method, so that the beneficial effects are the same as above, and are not repeated herein.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of an audio and video data transmission method provided in an embodiment of the present application;
fig. 2 is a schematic diagram of an audio/video data transmission apparatus according to an embodiment of the present application;
fig. 3 is a structural diagram of an audio/video data transmission device according to another embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the present application.
The core of the application is to provide an audio and video data transmission method, system, device and storage medium, so as to solve the problem that under the framework of cloud computing, multiple virtual machines on one host cannot use dedicated hardware for decoding at the same time, and only single decoding can be performed, so that data transmission delay is caused.
In packet-switched networks, a single message is divided into a plurality of data blocks, called packets, which contain address information of both the sender and the recipient. These packets are then transmitted along different paths in one or more networks and recombined at the destination. The Open System Interconnection (OSI) model is a standard defined by the international organization for standardization (ISO) that defines a layered architecture in which each layer defines protocols for different communication levels. The OSI model has 7 layers, layers 1 to 7 being: physical layer, data link layer, network layer, transport layer, session layer, presentation layer, application layer. The OSI model can be logically divided into two parts: layers 1 to 3 of the lower layers are concerned with the transmission of the original data; the upper layers 4 to 7 are concerned with applications under the network. The traditional hardware network card only realizes the logic of L1-L2 layers in a network protocol, and the intelligent network card is provided with an intelligent processing chip which can process the logic of L3-L4 layers.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
Fig. 1 is a diagram of an audio and video data transmission method provided in an embodiment of the present application, which is applied to a host machine using an intelligent network card, and includes:
s10: decapsulating the data packet, wherein the data packet is a data packet containing audio and video data to be transmitted;
it should be noted that, in data encapsulation, in general, service data is mapped into a payload of an encapsulation protocol, and then a packet header of a corresponding protocol is filled to form a data packet of the encapsulation protocol, and rate adaptation is completed. It may be not well understood that, simply, when a packet is transmitted between different devices using a network, in order to be reliably and accurately transmitted to a destination and to efficiently use transmission resources (transmission devices and transmission lines), the packet is split and packed in advance, a destination address, a local address, and some bytes for error correction are added to the transmitted packet, and when security and reliability are high, encryption processing is performed, and the like. These operations are called data encapsulation. Decapsulation is the reverse process of encapsulation, and is to disassemble the protocol packet, process the information in the packet header, and take out the service information in the payload.
In this embodiment, data information in a data packet is not specifically limited, it can be understood that audio and video data that needs to be transmitted in the data packet needs to be transmitted to a plurality of virtual machines, and it can be understood that for a virtual machine, different data streams need to be received, so that ports used for communication before different virtual machines are different, and thus the virtual machine is managed.
S11: preprocessing the decapsulated data, and performing hardware decoding on the processed data through hardware decoding equipment;
it should be noted that, in this embodiment, preprocessing is performed on the decapsulated data, and in the field of programming, preprocessing generally refers to a process before a binary code is generated in a process of translating a program source code into a target code. Typically, a preprocessor (preprocessor) processes a program source code text, and the obtained result is further compiled by a compiler core, it is understood that the preprocessing is generally performed by software, and in this embodiment, specific steps of the preprocessing are not limited.
In the present embodiment, decoding is a process of restoring a digital code to its content or converting an electric pulse signal, an optical signal, a radio wave, or the like into information, data, or the like, which it represents, by a specific method. The specific process of the hardware and the specific choice of the decoding device are not limited in the present embodiment, and it is understood that the hardware decoding device may include, but is not limited to, the following devices.
A Digital Signal Processor (DSP) is a computer or a special Processing device, and performs Processing such as acquisition, transformation, filtering, estimation, enhancement, compression, and recognition on a Signal in a Digital form to obtain a Signal form meeting the needs of a user. Digital signal processing and analog signal processing are a subset of signal processing. The purpose of digital signal processing is to measure or filter a continuous analog signal of the real world. It is therefore necessary to convert the signal from the analogue domain to the digital domain before digital signal processing, which is usually done by means of an analogue to digital converter. The output of the digital signal processing is often converted to the analog domain as well, which is accomplished by a digital-to-analog converter.
A Field-Programmable Gate Array (FPGA) is a product of further development on the basis of Programmable devices. The circuit is a semi-custom circuit in the field of application-specific integrated circuits, not only overcomes the defects of the custom circuit, but also overcomes the defect that the number of gate circuits of the original programmable device is limited.
Folding in
S12: and repackaging the data through the intelligent network card, and transmitting the repackaged data to the virtual machine through the driver so as to generate the executable program running on the intelligent network card.
A virtual machine, as in an architecture in computer science, refers to a special software that creates an environment between a computer platform and an end user, who operates the software based on the environment created by the software. In computer science, a virtual machine refers to a software implementation of a computer that can run a program like a real machine. The virtual machine technology is one of virtualization technologies, the so-called virtualization technology is to convert things from one form to another form, the most common virtualization technology is virtualization of a memory in an operating system, a memory space required by a user in actual operation may be much larger than the memory size of a physical machine, and by using the memory virtualization technology, the user can virtualize a part of a hard disk into the memory, which is transparent to the user.
The driver is a small piece of code added to the operating system, and contains related hardware devices, the driver is a configuration file written by a hardware manufacturer according to the operating system, and it can be said that without the driver, the hardware in the computer cannot work. In order to ensure the compatibility of the hardware and enhance the functions of the hardware, the specific type of the driver is not limited in this embodiment, and it should be noted that the executable program mentioned in this embodiment is an original audio/video file to be transmitted, and the specific process generated and the like are not limited in this embodiment.
According to the audio and video data transmission method provided by the embodiment, an original common network card is replaced by the intelligent network card, the decapsulated data are preprocessed, the processed data are subjected to hardware decoding through the hardware decoding device, the data are repackaged through the intelligent network card and are transmitted to the virtual machine through the driver, so that an executable program capable of running on the intelligent network card can be generated, the data are subjected to hardware decoding through the hardware decoding device due to the fact that the intelligent network card conducts hardware decoding and preprocessing on the data, and finally the data in the data packet are packaged through the intelligent network card after the decoding is completed, and are transmitted to the virtual machine through the driver. Compared with the prior direct communication between the decoding hardware and the virtual machine, in the embodiment, the multilayer virtual switch function of the intelligent network card is adopted, and the intelligent network card is used as an intermediate medium to realize the communication between the virtual machine and the hardware decoding device in the host machine, and as all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card to be decoded together, and are transmitted back to the intelligent network card to be repackaged after being decoded, and the data corresponding to the audio and video files are transmitted to the corresponding virtual machine, the problem of data transmission delay is avoided.
In the foregoing embodiment, before preprocessing the decapsulated data and performing hardware decoding on the processed data by using a hardware decoding device, the method further includes:
and arranging the data to be processed according to a priority queue, wherein the priority order of the priority queue is determined according to the size of the data.
The priority is a parameter for determining the priority of each operating program to receive system resources when the computer time-sharing operating system processes a plurality of operating programs. The logical operators are the same level as all the monocular operators in the logical operators, and the logical non-operator (!) is higher than the binocular arithmetic operator, the relational operator, the logical operator AND (&) and the logical operator or (|); and operations are higher than or operations, which are both lower than arithmetic and relational operators. In this embodiment, only the priority order is limited to be related to the data size, and the specific calculation manner of the priority and other related calculation parameters are not limited.
It should be noted that, in this embodiment, the data is prioritized first, so that the smoothness of data processing is ensured, and data redundancy is prevented.
Considering the influence of the previous communication protocol of the host on the data transmission, before the decapsulating the data packet, the method further includes:
and unloading the TCP/UDP protocol of the network flow of the host machine.
The Transmission Control Protocol (TCP) is a connection-oriented, reliable Transport layer (Transport layer) communication Protocol based on a byte stream. In a simplified Open System Interconnection (OSI) model of a computer network, which implements functions specified by a fourth layer transport layer, a User Datagram Protocol (UDP) is a connectionless transport layer Protocol in the OSI reference model, which provides transaction-oriented simple unreliable messaging service, and IETF RFC 768 is a formal specification of UDP.
The application layer sends a data stream represented in 8-bit bytes to the TCP layer for internetwork transfer, and the TCP then segments the data stream into segments of appropriate length (usually limited by the Maximum Transfer Unit (MTU) of the data link layer of the network to which the computer is connected). The TCP then passes the resulting packet to the IP layer, which passes the packet through the network to the TCP layer of the receiving entity.
UDP is a connectionless transport layer protocol in the OSI reference model, which is mainly used in transport that does not require the arrival of packets in sequence, the inspection and ordering of the packet transmission sequence being done by the application layer, providing a transaction-oriented simple unreliable messaging service. The UDP protocol is basically an interface of the IP protocol with the upper layer protocol. The UDP application ports respectively run a plurality of application programs on the same equipment.
The communication mode of the host machine is cleared by unloading the protocol, so that the normal sending of audio and video data communication is ensured.
The specific manner of the preprocessing in the foregoing embodiments is not limited, and a preferred scheme is provided herein, where the preprocessing the decapsulated data transmission includes:
and deleting the label of the corresponding network ID in the data, and setting the corresponding relation between the ID of the virtual machine and the data.
It should be noted that the identification mark code (ID) is also called a serial number or an account number, and is a relatively unique code in a certain system, which is equivalent to that an "ID card" is in a specific object, and the ID number is generally unchanged, and as for what is used to identify the object, a rule formulated by a designer determines that, when preprocessing is performed, a corresponding network address in data is deleted in advance.
In view of the practicability of the final scheme, that is, after the transmission is completed, the generation of the related audio/video file, the preferred scheme is provided, and after the data is repackaged through the intelligent network card and is transmitted to the virtual machine through the driver, the method further includes:
reconstructing codes of the original virtual machine, and adopting a GCC compiler to recompile a flow preprocessing program of the intelligent network card to generate an executable program which can run on the intelligent network card, wherein the executable program is audio and video corresponding to audio and video data.
It should be noted that a virtual switch (OVS) is a high-quality, multi-layer virtual switch (network layered layer) that aims to allow large-scale network automation to be extended by programming, while still supporting standard management interfaces and protocols: sflow. The switching elements are formed by software, so that the virtual switch is called a software switch, and compared with the traditional physical switch, the virtual switch has many points: the configuration is flexible, because the software implementation is realized, tens of virtual switches or hundreds of virtual switches can be configured on one physical server, and the number of ports can be flexibly selected. The cost is low, and the switching speed of 10Gbps can be easily achieved in a software mode.
The GNU Compiler suite (GUN) contains the front-end of multiple programming languages, including libraries of these languages as free software released in GPL licenses, which are also a critical part of the GNU project. GCC originally served as the official compiler for GNU operating systems and is now adopted as a standard compiler by most Unix-like operating systems.
In the embodiment, a subsequent executable file generation mode is added, so that the completeness of the scheme is ensured, and the user experience is optimized.
In the above embodiments, neither the communication mode nor the data caching mode is limited, and an optimal scheme is provided herein, in which a memory device of the intelligent network card is used as the data cache and the queue, and a PCIE interface is used as the intelligent network card to communicate with the hardware decoding device.
The high speed serial computer extended bus standard (PCIE) includes higher maximum system bus throughput, lower I/O pin count and smaller physical size, better scaling of bus device performance, more detailed error detection and reporting mechanisms (advanced error reporting, AER) and native hot plug functionality. Newer versions of the PCIE standard provide hardware support for I/O virtualization.
In view of the communication between the hardware decoding device and the intelligent network card, the preferable scheme is provided, before the decapsulating the data packet, further comprising:
and establishing a communication cache of the hardware decoding device and an interface end of the intelligent network card, and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.
Compared with the traditional single Data Rate, the DDR technology realizes two read/write operations in one clock cycle, namely, the read/write operations are respectively executed once on the rising edge and the falling edge of the clock, so that the cache Rate is higher, and the service efficiency is higher.
In this embodiment, a more optimal caching mode is provided, so as to ensure that
In order to solve the above problem, the present application provides an audio/video data transmission device, including:
the decapsulation module is used for decapsulating the data packet, wherein the data packet is a data packet containing audio and video data to be transmitted;
the decoding module is used for preprocessing the unpackaged data transmission and performing hardware decoding on the processed data;
and the packaging module is used for repackaging the data through the intelligent network card and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program capable of running on the intelligent network card.
In the above embodiments, the audio and video data transmission method is described in detail, and the application also provides an embodiment corresponding to the audio and video data transmission device. It should be noted that the present application describes the embodiments of the apparatus portion from two perspectives, one is from the perspective of the function module, and the other is from the perspective of the hardware.
Fig. 2 is a schematic diagram of an audio/video data transmission device provided in an embodiment of the present application, where the device includes:
a decapsulation module 10, configured to decapsulate a data packet, where the data packet is a data packet that includes audio and video data that needs to be transmitted;
the decoding module 11 is configured to perform preprocessing on the decapsulated data transmission and perform hardware decoding on the processed data;
and the packaging module 12 is used for repackaging the data through the intelligent network card and transmitting the repackaged data to the virtual machine through the driver so as to generate an executable program capable of running on the intelligent network card.
Preferably, the apparatus further comprises:
and the arrangement module is used for arranging the data to be processed according to the priority queue, and the priority sequence of the priority queue is determined according to the size of the data.
Preferably, the apparatus further comprises:
and the unloading module is used for unloading the TCP/UDP protocol of the network flow of the host machine.
Preferably, the apparatus further comprises:
in order to solve the above problem, the present application further provides an audio/video data transmission device, which is characterized by comprising a memory for storing a computer program;
and the processor is used for realizing the steps of the audio and video data transmission method when executing the computer program.
Preferably, the apparatus further comprises:
and the reconstruction module is used for reconstructing the original virtual machine codes, and adopting a GCC compiler to recompile the flow preprocessing program of the intelligent network card to generate an executable program which can run on the intelligent network card, wherein the executable program is audio and video corresponding to the audio and video data.
Preferably, the apparatus further comprises:
and the cache module is used for creating a communication cache of the hardware decoding device and the interface end of the intelligent network card and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.
Since the embodiment of the apparatus portion and the embodiment of the method portion correspond to each other, reference is made to the description of the embodiment of the method portion for the embodiment of the apparatus portion and the corresponding advantageous effects, which are not repeated herein.
The audio and video data transmission device that this embodiment provided, including the decapsulation module, decoding module and packing module, can realize replacing original ordinary network card through adopting intelligent network card, and carry out the preliminary treatment to the data after the decapsulation, and carry out hardware decoding through hardware decoding equipment with the data after handling, data repackage through intelligent network card, and transmit for the virtual machine through the drive, so that generate can be at the executable program of intelligent network card operation, because intelligent network card knows encapsulation and preliminary treatment to data, thereby make hardware decoding equipment can carry out hardware decoding to the data in this data packet, at last after the completion of decoding, pack by intelligent network card again, thereby transmit for the virtual machine through the drive. Compared with the prior direct communication between the decoding hardware and the virtual machine, in the embodiment, the multilayer virtual switch function of the intelligent network card is adopted, and the intelligent network card is used as an intermediate medium to realize the communication between the virtual machine and the hardware decoding device in the host machine, and as all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card to be decoded together, and are transmitted back to the intelligent network card to be repackaged after being decoded, and the data corresponding to the audio and video files are transmitted to the corresponding virtual machine, the problem of data transmission delay is avoided.
Fig. 3 is a structural diagram of an audio/video data transmission device according to another embodiment of the present application, and as shown in fig. 3, the audio/video data transmission device includes: a memory 20 for storing a computer program;
a processor 21, configured to implement the steps of the audio-video data transmission method as mentioned in the above embodiments when executing the computer program.
The audio/video data transmission device provided by the embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The Processor 21 may be implemented in at least one hardware form of Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a Graphics Processing Unit (GPU) which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an Artificial Intelligence (AI) processor for processing computational operations related to machine learning.
The memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing the following computer program 201, wherein after being loaded and executed by the processor 21, the computer program can implement the relevant steps of the audio and video data transmission method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202, data 203, and the like, and the storage manner may be a transient storage manner or a permanent storage manner. Operating system 202 may include, among others, Windows, Unix, Linux, and the like. The data 203 may include, but is not limited to, data involved in an audio-visual data transmission method, and the like.
In some embodiments, the audio-video data transmission device may further include a display 22, an input-output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
Those skilled in the art will appreciate that the configuration shown in fig. 3 does not constitute a limitation of the audiovisual data transmission arrangement and may include more or fewer components than those shown.
The audio and video data transmission device provided by the embodiment of the application comprises a memory and a processor, wherein when the processor executes a program stored in the memory, the following method can be realized: the audio and video data transmission method in the above embodiments.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
The audio and video data transmission device provided by the embodiment comprises a memory and a processor, wherein when the processor executes a program stored in the memory, the original common network card can be replaced by the intelligent network card, the decapsulated data is preprocessed, the processed data is subjected to hardware decoding through hardware decoding equipment, the data is repackaged through the intelligent network card and is transmitted to a virtual machine through driving, so that an executable program capable of running on the intelligent network card is generated, the data is subjected to understanding encapsulation and preprocessing through the intelligent network card, the hardware decoding equipment can perform hardware decoding on the data in the data packet, and finally, after the decoding is completed, the data is packaged through the intelligent network card, so that the data is transmitted to the virtual machine through driving. Compared with the prior direct communication between the decoding hardware and the virtual machine, in the embodiment, the multilayer virtual switch function of the intelligent network card is adopted, and the intelligent network card is used as an intermediate medium to realize the communication between the virtual machine and the hardware decoding device in the host machine, and as all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card to be decoded together, and are transmitted back to the intelligent network card to be repackaged after being decoded, and the data corresponding to the audio and video files are transmitted to the corresponding virtual machine, the problem of data transmission delay is avoided.
Finally, the application also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps as set forth in the above-mentioned method embodiments.
It is to be understood that if the method in the above embodiments is implemented in the form of software functional units and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods described in the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Since the embodiment of the readable storage medium portion corresponds to the embodiment of the method portion, please refer to the description of the embodiment of the method portion for the embodiment of the apparatus portion, which is not repeated here.
The computer-readable storage medium provided in this embodiment stores a computer program, where when the computer program is executed by a processor, the original common network card is replaced by an intelligent network card, the decapsulated data is preprocessed, the processed data is hardware decoded by a hardware decoding device, the data is repackaged by the intelligent network card and is transmitted to a virtual machine through a driver, so as to generate an executable program that can run on the intelligent network card. Compared with the prior direct communication between the decoding hardware and the virtual machine, in this embodiment, the multilayer virtual switch function of the intelligent network card is used to implement communication between the virtual machine and the hardware decoding device in the host machine by using the intelligent network card as an intermediate medium, and since all the received audio and video files can be transmitted to the hardware decoding device through the multilayer virtual switch function of the intelligent network card for decoding, and then transmitted back to the intelligent network card for repackaging after decoding, and the data corresponding to the audio and video files is transmitted to the corresponding virtual machine, the problem of data transmission delay is not caused.
The method, the device and the computer-readable storage medium for transmitting audio and video data provided by the present application are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, without departing from the principle of the present application, the present application can also make several improvements and modifications, and those improvements and modifications also fall into the protection scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. An audio and video data transmission method is characterized by being applied to a host machine adopting an intelligent network card and comprising the following steps:
decapsulating a data packet, wherein the data packet is a data packet containing audio and video data to be transmitted;
preprocessing the decapsulated data, and performing hardware decoding on the processed data through hardware decoding equipment;
and repackaging data through the intelligent network card, and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program running on the intelligent network card.
2. The method for transmitting audio-video data according to claim 1, wherein before the preprocessing the decapsulated data and performing hardware decoding on the processed data by a hardware decoding device, the method further comprises:
and arranging the data to be processed according to a priority queue, wherein the priority order of the priority queue is determined according to the size of the data.
3. The method for transmitting audio-video data according to claim 2, further comprising, before the decapsulating the data packet:
and unloading the TCP/UDP protocol of the network flow of the host machine.
4. The method for transmitting audio-video data according to claim 2, wherein the preprocessing the decapsulated data transmission comprises:
and deleting the label of the corresponding network ID in the data, and setting the corresponding relation between the ID of the virtual machine and the data.
5. The audio-video data transmission method according to any one of claims 1 to 4, wherein after the data is repackaged by the smart network card and is transmitted to the virtual machine by driving, the method further comprises:
reconstructing codes of an original virtual machine, and recompiling a flow preprocessing program of the intelligent network card by adopting a GCC compiler to generate an executable program which can run on the intelligent network card, wherein the executable program is audio and video corresponding to the audio and video data.
6. The audio/video data transmission method according to claim 2, wherein the memory device of the intelligent network card is used as a data buffer and a queue, and the PCIE interface is used as the intelligent network card to communicate with the hardware decoding device.
7. The method for transmitting audio-video data according to claim 1, further comprising, before the decapsulating the data packet:
and creating a communication cache of the hardware decoding device and the interface end of the intelligent network card, and caching the data packet after the primary processing of the intelligent network card to a DDR memory of the intelligent network card.
8. An audio-video data transmission apparatus, comprising:
the de-encapsulation module is used for de-encapsulating a data packet, wherein the data packet contains audio and video data to be transmitted;
the decoding module is used for preprocessing the unpackaged data transmission and performing hardware decoding on the processed data;
and the packaging module is used for repackaging the data through the intelligent network card and transmitting the repackaged data to the virtual machine through driving so as to generate an executable program capable of running on the intelligent network card.
9. An audio-video data transmission apparatus, comprising a memory for storing a computer program;
a processor for implementing the steps of the audio-visual data transmission method as claimed in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the steps of the audio-visual data transmission method according to one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210609559.9A CN115002087A (en) | 2022-05-31 | 2022-05-31 | Audio and video data transmission method, system, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210609559.9A CN115002087A (en) | 2022-05-31 | 2022-05-31 | Audio and video data transmission method, system, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115002087A true CN115002087A (en) | 2022-09-02 |
Family
ID=83030797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210609559.9A Pending CN115002087A (en) | 2022-05-31 | 2022-05-31 | Audio and video data transmission method, system, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115002087A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883658A (en) * | 2022-12-07 | 2023-03-31 | 长安大学 | Cross-platform image information transmission method, system, equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102104772A (en) * | 2011-02-28 | 2011-06-22 | 南京邮电大学 | Method for designing wireless video sensor coding/decoding system |
| US8010692B1 (en) * | 2009-11-05 | 2011-08-30 | Adobe Systems Incorporated | Adapting audio and video content for hardware platform |
| CN207070236U (en) * | 2017-01-18 | 2018-03-02 | 北京小鸟科技股份有限公司 | IP-based audio frequency and video control Lossless transport and exchange system |
| CN107911700A (en) * | 2017-10-31 | 2018-04-13 | 深信服科技股份有限公司 | A kind of hardware coding/decoding method, decoding device and storage medium based on virtualization |
| US20190220296A1 (en) * | 2016-12-21 | 2019-07-18 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for transmitting data packet based on virtual machine |
| CN113821310A (en) * | 2021-11-19 | 2021-12-21 | 阿里云计算有限公司 | Data processing method, programmable network card device, physical server and storage medium |
| CN114024910A (en) * | 2021-10-29 | 2022-02-08 | 上海广策信息技术有限公司 | Extremely-low-delay reliable communication system and method for financial transaction system |
-
2022
- 2022-05-31 CN CN202210609559.9A patent/CN115002087A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8010692B1 (en) * | 2009-11-05 | 2011-08-30 | Adobe Systems Incorporated | Adapting audio and video content for hardware platform |
| CN102104772A (en) * | 2011-02-28 | 2011-06-22 | 南京邮电大学 | Method for designing wireless video sensor coding/decoding system |
| US20190220296A1 (en) * | 2016-12-21 | 2019-07-18 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for transmitting data packet based on virtual machine |
| CN207070236U (en) * | 2017-01-18 | 2018-03-02 | 北京小鸟科技股份有限公司 | IP-based audio frequency and video control Lossless transport and exchange system |
| CN107911700A (en) * | 2017-10-31 | 2018-04-13 | 深信服科技股份有限公司 | A kind of hardware coding/decoding method, decoding device and storage medium based on virtualization |
| CN114024910A (en) * | 2021-10-29 | 2022-02-08 | 上海广策信息技术有限公司 | Extremely-low-delay reliable communication system and method for financial transaction system |
| CN113821310A (en) * | 2021-11-19 | 2021-12-21 | 阿里云计算有限公司 | Data processing method, programmable network card device, physical server and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| DAJIANG ZHOU: "An 8K H.265/HEVC Video Decoder Chip With a New System Pipeline Design", 《 IEEE JOURNAL OF SOLID-STATE CIRCUITS》 * |
| 王升;班有容;陈佳媛;张昊;: "硬件加速在核心网转发面应用的思考和实践", 中兴通讯技术, no. 03 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115883658A (en) * | 2022-12-07 | 2023-03-31 | 长安大学 | Cross-platform image information transmission method, system, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110417780B (en) | Multi-channel high-speed data interface conversion module of customized data transmission protocol | |
| CN104899085B (en) | A kind of data processing method and device | |
| US9160659B2 (en) | Paravirtualized IP over infiniband bridging | |
| CN110912836B (en) | Method for forwarding data by forwarding device, control device, storage medium and system | |
| CN110297797B (en) | Heterogeneous protocol conversion device and method | |
| CN111158690B (en) | Desktop application framework, construction method, desktop application running method and storage medium | |
| US9965434B2 (en) | Data packet processing | |
| WO2021036407A1 (en) | Communication method and apparatus, electronic device, and storage medium | |
| US11182150B2 (en) | Zero packet loss upgrade of an IO device | |
| US8736617B2 (en) | Hybrid graphic display | |
| CN115002087A (en) | Audio and video data transmission method, system, device and storage medium | |
| US20190303740A1 (en) | Block transfer of neuron output values through data memory for neurosynaptic processors | |
| US11620250B2 (en) | Systems and methods for data transfer over a shared interface | |
| US9426033B2 (en) | Target mapping and implementation of abstract device model | |
| US10374608B1 (en) | Bridged integrated circuits | |
| CN111736844B (en) | Database cloud service standard interface and implementation method | |
| US20160070609A1 (en) | Automatically generating efficient remote procedure call (rpc) code for heterogeneous systems | |
| US7296187B1 (en) | Hardware debug device having script-based host interface | |
| JP7331178B2 (en) | Chassis simulation method, device, server, storage medium and program | |
| CN114579258A (en) | Audio processing method, device and system of virtual machine and electronic equipment | |
| CN115033348A (en) | Method, system, equipment and medium for unified management of virtual machines and containers | |
| US10886923B1 (en) | Bridged integrated circuits | |
| CN108848398B (en) | Method, device, terminal and storage medium for distributing local barrage messages | |
| EP2804103A1 (en) | System and method for forwarding a command stream | |
| CN110572423A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |