CN116132706A - Video processing method, injection method and electronic equipment - Google Patents
Video processing method, injection method and electronic equipment Download PDFInfo
- Publication number
- CN116132706A CN116132706A CN202310096728.8A CN202310096728A CN116132706A CN 116132706 A CN116132706 A CN 116132706A CN 202310096728 A CN202310096728 A CN 202310096728A CN 116132706 A CN116132706 A CN 116132706A
- Authority
- CN
- China
- Prior art keywords
- virtual machine
- video
- computing resources
- decoded
- resources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002347 injection Methods 0.000 title claims abstract description 62
- 239000007924 injection Substances 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 14
- 230000004044 response Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 abstract description 24
- 230000005540 biological transmission Effects 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000013468 resource allocation Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a video processing method, an injection method and electronic equipment. The video processing method comprises the following steps: firstly, acquiring service demand information of each virtual machine on a main server; secondly, according to the service demand information and the computing resources of the main server, virtual machine computing resources of the virtual machine are configured, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources; then acquiring file position information of a video to be decoded on a main server; and finally mapping the file position information to the virtual machine, so that the virtual machine obtains the video to be decoded from the main server according to the file position information, obtains the video to be injected after decoding by the virtual machine, and transmits the video to be injected to the video injection device, wherein the video injection device is connected with the main server. The invention can complete the transmission work of the video data to be injected by only one main server and one video injection device, reduces the use cost and can avoid unstable video processing caused by simultaneous preemption of computing resources by a plurality of tasks.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a video processing method, an injection method, and an electronic device.
Background
In the process of developing algorithms in the unmanned technology, training, verification, testing and the like of algorithms (such as a neural network) in an automobile controller are required, and various video data are required to be injected into the controller. In the process of injecting video data, a server generally transmits processed video to a video injection device, and then the video injection unit synchronously reinjects the processed video to a controller.
The controller is used as a video receiving device and is required to receive multiple paths of video data, each path of video data represents video shot by a camera, and a server transmits one path of video data to a video injection device. To meet the multiple video injection requirements, multiple video injection devices and multiple servers are often required, which will greatly increase the cost of use.
Disclosure of Invention
In order to overcome the above problems and disadvantages, an object of the present invention is to provide a video processing method, an injection method, and an electronic device, which can reduce the demands of a server and a video injection device through a virtual machine, and reduce the use cost.
To achieve the above object, a first aspect of the present invention provides a video processing method, including:
acquiring service demand information of each virtual machine on a main server;
according to the service demand information and the computing resources of the main server, virtual machine computing resources of the virtual machine are configured, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources;
acquiring file position information of a video to be decoded on a main server;
mapping the file location information to the virtual machine such that:
the virtual machine acquires the video to be decoded from the main server according to the file position information, receives and decodes the video to be decoded by utilizing the configured virtual machine computing resources, and transmits the video to be injected obtained after decoding to the video injection device, wherein the video injection device is connected with the main server.
Optionally, after the step of mapping the file location information to the virtual machine, the method further includes:
acquiring reference information, wherein the reference information comprises working state information of a virtual machine and/or video data receiving conditions, and the working state information comprises CPU utilization rate, GPU utilization rate and memory utilization rate;
and adjusting the virtual machine computing resource configuration of the virtual machine according to the reference information.
Optionally, the step of receiving and decoding the video to be decoded by using the configured virtual machine computing resource includes:
and decoding the video to be decoded by the virtual machine by utilizing the configured GPU resources to obtain the video to be injected.
Optionally, the service requirement information includes decoding means information, where the decoding means information is used to characterize whether the virtual machine needs to decode the video to be injected through GPU hardware, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine needs to decode the video to be decoded through GPU hardware;
if yes, configuring an independent GPU for the virtual machine as GPU resources.
Optionally, the service requirement information further includes response instantaneity, wherein the response instantaneity is used for representing a program instruction response speed of the virtual machine, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine needs to improve response instantaneity;
if yes, configuring exclusive CPU cores for the virtual machine.
Optionally, the decoding means information further includes information for characterizing a data bandwidth of the video to be decoded, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine computing resources of the virtual machine are matched with the data bandwidth or not;
if yes, configuring memory resources with memory capacity larger than a first threshold for the virtual machine, and if not, configuring memory resources with memory capacity smaller than a second threshold for the virtual machine.
Optionally, the service requirement information further includes resolution of the video to be decoded, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
and configuring the memory capacity of the virtual machine according to the resolution of the video to be decoded.
Optionally, after the step of configuring the virtual machine computing resources of the virtual machine according to the service requirement information, the method further includes:
synchronizing a virtual machine clock source of the virtual machine with a master server clock source of the master server, so that the virtual machine transmits the video to be injected to the video injection device according to the virtual machine clock source and the time stamp of the video to be injected.
A second aspect of the present invention provides a video injection method, including:
receiving video to be injected, wherein video data to be injected are obtained according to the video processing method;
and injecting the video to be injected into the video receiving device according to the time stamp of the video to be injected.
A third aspect of the present invention provides an electronic device comprising a processor and a memory, the memory having stored thereon a computer program which, when executed by the processor, implements the video processing method, or video injection method, described above.
Compared with the prior art, the invention has the beneficial effects that: the video processing method of the invention comprises the following steps: firstly, acquiring service demand information of each virtual machine on a main server; secondly, according to the service demand information and the computing resources of the main server, virtual machine computing resources of the virtual machine are configured, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources; then acquiring file position information of the video to be injected on the main server; and finally mapping the file position information to the virtual machine so that the virtual machine obtains the video to be decoded from the main server according to the file position information, receives and decodes the video to be decoded by utilizing the configured virtual machine computing resources, and transmits the video to be injected obtained after decoding to a video injection device which is connected with the main server. Through the steps, the method and the device for transmitting the video to be injected have the advantages that a plurality of virtual machines are configured on the main server, then the virtual machines are configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device through the plurality of virtual machines, so that the transmission work of the video data to be injected can be completed only by one main server and one video injection device, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is evident that the drawings in the following description are only some embodiments of the present invention and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art. In the drawings:
fig. 1 is a schematic diagram of a video data transmission system according to an embodiment of the invention;
FIG. 2 is a flow chart of a video processing method according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a host server and virtual machine framework according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a video processing apparatus according to an embodiment of the invention;
fig. 5 is a schematic diagram of a computer system of an electronic device according to an embodiment of the invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams depicted in the figures are merely functional entities and do not necessarily correspond to physically separate entities. That is, the functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.
An embodiment of the present invention provides a video data transmission system, as shown in fig. 1, which includes a main server 1, a video injection device 2, and a video receiving device 3.
The main server 1 may be an upper computer, an industrial personal computer, a real-time HOST or an HOST, and may perform decoding processing on video data. Data can be transmitted between the video processing unit 1 and the video injection unit 2 through an HDMI (high definition multimedia interface) line, which is a full digital video and sound transmission interface, and uncompressed audio and video signals can be transmitted. Video may also be transmitted between the video processing unit 1 and the video injection unit 2 via a PCIE (peripheral component interconnect express, a high-speed serial computer expansion bus standard) interface.
The video injection device 2 may be an FPGA (Field-Programmable Gate Array, field programmable gate array) video injection board.
The video receiving apparatus 3 may be a controller with a neural network model, have a machine learning algorithm, and may process received video data to obtain an output result, for example, may perform object recognition on a video image. The video receiving apparatus 3 can perform processing such as algorithm verification, development, or testing in unmanned automatic driving (or assisted driving) technology by using video data.
The video processing method and the video injection method of the embodiment can be used for simulation test in unmanned technology. The main server 1 processes the video and transmits the processed video to the video injection device 2, and the video injection device 2 injects the video data to the video receiving device 3.
Specifically, an embodiment of the present invention provides a video processing method, as shown in fig. 2, including a step 100, a step 200, a step 300, and a step 400, which specifically includes the following steps:
In step 200, as shown in fig. 3, virtual machine computing resources of the virtual machine are configured according to the service requirement information and computing resources of the main server 1, where the virtual machine computing resources include CPU (central processing unit) resources, GPU (graphics processor or graphics card) resources, and memory resources.
The service requirement information comprises decoding means information, and the decoding means information is used for representing whether the virtual machine needs to decode and process the video to be injected through GPU hardware. Further, step 200 specifically includes: judging whether the virtual machine needs to decode the video to be decoded through hardware; if yes, configuring an independent GPU for the virtual machine as GPU resources.
And when the GPU hardware decoding is needed to be realized by a certain virtual machine, an independent GPU is allocated to the virtual machine, otherwise, if the GPU hardware decoding is not needed to be realized by the certain virtual machine, the independent GPU is not allocated. The embodiment can configure an independent GPU for the virtual machine so as to improve the video decoding capability of the virtual machine.
In one embodiment, the service requirement information further includes response instantaneity, and the response instantaneity is used for representing a program instruction response speed of the virtual machine. Further, step 200 specifically includes: judging whether the virtual machine needs to improve response instantaneity; if yes, configuring exclusive CPU cores for the virtual machine.
When the real-time performance of the response of the virtual machine needs to be improved, an exclusive CPU core is allocated to the virtual machine; for example, a certain virtual machine needs accurate system delay, and the virtual machine needs to achieve high real-time performance, at this time, one or more exclusive CPU cores may be allocated to the virtual machine, and other virtual machines and HOST HOSTs cannot use the CPU cores.
In one embodiment, the decoding means information is also used to characterize the data bandwidth of the video to be decoded. Further, step 200 may specifically include: judging whether the virtual machine computing resources of the virtual machine are matched with the data bandwidth or not; if yes, configuring memory resources with memory capacity larger than a first threshold for the virtual machine, and if not, configuring memory resources with memory capacity smaller than a second threshold for the virtual machine. Wherein the first threshold is greater than the second threshold.
When a certain virtual machine is required to rapidly process data with large bandwidth such as a high-resolution video sequence, a large-capacity memory is allocated to the virtual machine, wherein the value of the large-capacity memory is not lower than a first threshold value A, otherwise, the memory of the virtual machine is allocated to a small-capacity memory, the value of the small-capacity memory is not higher than a second threshold value B, and B is smaller than A.
In one embodiment, the traffic demand information further includes a resolution of the video to be decoded. Further, step 200 may specifically include: and configuring the memory capacity of the virtual machine according to the resolution of the video to be decoded. The higher the resolution of the video to be decoded, the larger the memory capacity of the virtual machine configuration, and the smaller the resolution of the video to be decoded, the smaller the memory capacity of the virtual machine configuration.
The memory (or memory resource) may refer to a memory (or memory resource) that can be used by the CPU, and may also include a memory (or memory resource) used by the CPU and a memory (or memory resource) that can be used by the GPU.
According to the embodiment, through the steps, the plurality of virtual machines are configured on the main server 1, then the virtual machines are configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device 2 through the plurality of virtual machines, so that decoding and transmission of video data to be injected can be completed only by one main server 1 and one video injection device 2, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
In one embodiment, after the step of mapping the file location information to the virtual machine, further comprising:
and acquiring reference information, wherein the reference information comprises working state information of the virtual machine and/or video data receiving conditions, and the working state information comprises CPU utilization rate, GPU utilization rate and memory utilization rate.
And adjusting the virtual machine computing resource configuration of the virtual machine according to the reference information.
For example, the CPU utilization, GPU utilization, and memory utilization of the virtual machine may be monitored in real time, and when the CPU utilization, GPU utilization, or memory utilization of the virtual machine is too high, more CPU resources, GPU resources, or memory resources may be allocated to the virtual machine. When the CPU utilization rate, the GPU utilization rate or the memory utilization rate of the virtual machine is too low, part of CPU resources, GPU resources or memory resources of the virtual machine can be allocated to the virtual machine with too high CPU utilization rate, GPU utilization rate or memory utilization rate. Therefore, the CPU resource, GPU resource or memory resource of the whole main server 1 can be fully utilized, and the normal operation of each virtual machine can be ensured.
For example, the video receiving condition of the video injection device 2 may be queried and obtained, whether the goal of processing 4 paths of 4K resolution 30fps video is achieved may be determined, and if not, the configuration of the computing resources of the virtual machines may be adjusted, so that each virtual machine requiring hardware decoding achieves the goal of processing four paths of 4K resolution 30fps video.
In the prior art, the virtual machine typically uses CPU resources to perform soft decoding. In the embodiment of the invention, the virtual machine can utilize GPU resources to hard decode so as to improve the video decoding efficiency. Thus, when efficient GPU hard decoding of video is required, an independent GPU is allocated to the virtual machine.
On the other hand, in this embodiment, the virtual machine may use GPU to hard decode, or may use CPU resources to receive video to be decoded, give the video to be decoded to the GPU resources, or may use the CPU resources to output the decoded video to be injected.
In one embodiment, after the step of configuring the virtual machine computing resources of the virtual machine according to the service requirement information, the method further includes:
the virtual machine clock source of the virtual machine is synchronized with the clock source of the main server 1, so that the virtual machine transmits the video to be injected to the video injection device 2 according to the virtual machine clock source and the time stamp of the video to be injected.
Each virtual machine is kept highly synchronized with the clock source of the same main server 1, so that video synchronization injection can be maintained during injection, and the video injection device 2 can synchronously inject video frames with the same time stamp into the video receiving device 3.
By the video processing method of the embodiment, the real-time performance and the data bandwidth of the video processing can be ensured, for example, when the 3840x2160 resolution 30fps frame rate video is processed, the decoding and outputting work is completed within 33ms by the efficient data processing of the CPU and the GPU. Meanwhile, since each pixel has RGB3 bytes, the bandwidth required to be processed is 3840x2160x3x 30= 746496000 bytes per second, which is about 750MB/s. Based on the data, after the virtual machine computing resources are reasonably distributed, the CPU utilization rate, the GPU utilization rate and the memory utilization rate are fully utilized, and the goal that each virtual machine needing hardware decoding processes 4 paths of 4K30fps can be reached.
On the other hand, in the working process, the rear end of the main server 1 is connected with the video injection device 2 (for example, FPGA) according to the CPU utilization rate, GPU utilization rate, memory utilization rate and other computing resource utilization conditions, the FPGA at the rear end can judge whether the 4-path 4K30fps target is achieved or not according to the receiving condition of video data, if not, the alarm feedback or the output performance report can be conducted, and a basis is provided for changing the resource allocation of the virtual machine.
In the embodiment, a plurality of virtual machines can be configured on the main server 1, then each virtual machine is configured with proper virtual machine computing resources, and video to be injected is sent to the video injection device 2 through the plurality of virtual machines, so that the transmission work of video data to be injected can be completed only by one main server 1 and one video injection device 2, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
Meanwhile, the embodiment adopts a scheme of efficient collaborative work of multiple GPUs, and a virtualization technology is utilized, so that multiple display cards can be supported on one main server 1 to output, each display card can work independently, and no interference in performance exists between each display card and each display card. The embodiment can effectively improve the utilization rate of computing resources such as CPU, GPU, memory and the like and improve the utilization rate of electric energy. According to the embodiment, the same clock source of the main server 1 is used, each virtual machine forms high-accuracy clock source synchronization, and the time accuracy of camera video reinjection work is guaranteed. Meanwhile, the technical utilization of virtualization can be expanded to a cluster, and resource allocation can be conveniently carried out in the environment of a machine room according to requirements.
The embodiment of the invention also provides a video injection method, which comprises the following steps:
receiving video to be injected, wherein video data to be injected are obtained according to the video processing method;
the video to be injected is injected to the video receiving apparatus 3 according to the time stamp of the video to be injected.
The video injection method of the embodiment utilizes the video processing method provided by the embodiment, a plurality of virtual machines can be configured on the main server 1, then each virtual machine is configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device 2 through the plurality of virtual machines, so that the transmission work of the video data to be injected can be completed only by one main server 1 and one video injection device 2, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
Meanwhile, in some examples, a scheme of efficient collaborative work of multiple GPUs can be adopted, and a virtualization technology is utilized, so that multiple display cards can be supported on one main server 1 to output, and each display card can work independently without interference in performance. The embodiment can effectively improve the utilization rate of computing resources such as CPU, GPU, memory and the like and improve the utilization rate of electric energy. According to the embodiment, the same clock source of the main server 1 is used, each virtual machine forms high-accuracy clock source synchronization, and the time accuracy of camera video reinjection work is guaranteed. Meanwhile, the technical utilization of virtualization can be expanded to a cluster, and resource allocation can be conveniently carried out in the environment of a machine room according to requirements.
In each virtual machine, GPU resources can be utilized to carry out hard decoding of the video to be decoded, and CPU resources can also be utilized to carry out soft decoding of the video to be decoded, and as the resolution of the high-definition video is far higher than that of the video in a common format, the code rate of the high-definition video is very high. The compression rate of the coding scheme such as VC-1 and H.264 is high, and the amount of calculation of decoding is large. Therefore, the conventional direct decoding (namely, what is called "soft solution") of the CPU consumes the computing power of the CPU greatly, for example, the CPU occupancy rate is often as high as 90% when some older CPUs soft-solve the h.264, and the older ones may be still.
Therefore, in one embodiment, whether to decode with the GPU or with the CPU may be selected for the virtual machine according to at least one of the encoding mode of the video to be decoded, the encoding compression rate, the code rate, the resolution, and the like, for example, if the encoding mode of a video to be decoded is an encoding mode with a compression rate higher than a threshold value, it is determined that the video to be decoded needs to be decoded with GPU hardware, corresponding decoding mode information may be obtained correspondingly, for example, if the encoding mode of a video to be decoded is VC-1, h.264, and the like, it is determined that the video to be decoded needs to be decoded with the GPU hardware by the corresponding virtual machine, corresponding decoding mode information may be obtained correspondingly, and, in addition, if the video to be decoded is not confirmed that the video to be decoded with GPU hardware is determined that the video to be decoded with the CPU hardware by the corresponding virtual machine.
By the mode, resources can be fully utilized, the situations of excessively high occupancy rate and immovable decoding of CPU decoding are avoided, and in addition, because the quantity of the GPUs is limited, the GPU hardware decoding is only utilized under specific conditions, and the GPU resources can be effectively saved. The present embodiment also provides a video processing apparatus, as shown in fig. 4, including:
a first obtaining unit 501, configured to obtain service requirement information of each virtual machine on a main server;
the configuration unit 502 is configured to configure virtual machine computing resources of the virtual machine according to the service demand information and computing resources of the main server, where the virtual machine computing resources include CPU resources, GPU resources and memory resources;
a second obtaining unit 503, configured to obtain file location information of a video to be decoded on a main server;
the mapping unit 504 is configured to map the file location information to the virtual machine, so that the virtual machine obtains the video to be decoded from the host server according to the file location information, receives and decodes the video to be decoded by using the configured virtual machine computing resource, and transmits the video to be injected obtained after decoding to the video injection device, where the video injection device is connected with the host server.
The video processing device of the embodiment adopts the video processing method provided by the embodiment, a plurality of virtual machines can be configured on the main server 1, then each virtual machine is configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device 2 through the plurality of virtual machines, so that the transmission work of the video data to be injected can be completed only by one main server 1 and one video injection device 2, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
Meanwhile, the embodiment adopts a scheme of efficient collaborative work of multiple GPUs, and a virtualization technology is utilized, so that multiple display cards can be supported on one main server 1 to output, each display card can work independently, and no interference in performance exists between each display card and each display card. The embodiment can effectively improve the utilization rate of computing resources such as CPU, GPU, memory and the like and improve the utilization rate of electric energy. According to the embodiment, the same clock source of the main server 1 is used, each virtual machine forms high-accuracy clock source synchronization, and the time accuracy of camera video reinjection work is guaranteed. Meanwhile, the technical utilization of virtualization can be expanded to a cluster, and resource allocation can be conveniently carried out in the environment of a machine room according to requirements.
Fig. 5 shows a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
It should be noted that, the computer system of the electronic device shown in fig. 5 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present invention.
As shown in fig. 5, the computer system includes a central processing unit (Central Processing Unit, CPU) 1801, which can perform various appropriate actions and processes, such as performing the methods described in the above embodiments, according to a program stored in a Read-Only Memory (ROM) 1802 or a program loaded from a storage section 1808 into a random access Memory (Random Access Memory, RAM) 1803. In the RAM 1803, various programs and data required for system operation are also stored. The CPU 1801, ROM 1802, and RAM 1803 are connected to each other via a bus 1804. An Input/Output (I/O) interface 1805 is also connected to the bus 1804.
The following components are connected to the I/O interface 1805: an input section 1806 including a keyboard, a mouse, and the like; an output portion 1807 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and a speaker, etc.; a storage section 1808 including a hard disk or the like; and a communication section 1809 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 1809 performs communication processing via a network such as the internet. The drive 1810 is also connected to the I/O interface 1805 as needed. Removable media 1811, such as magnetic disks, optical disks, magneto-optical disks, semiconductor memory, and the like, is installed as needed on drive 1810 so that a computer program read therefrom is installed as needed into storage portion 1808.
In particular, according to embodiments of the present invention, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising a computer program for performing the method shown in the flowchart. In such embodiments, the computer program may be downloaded and installed from a network via the communication portion 1809, and/or installed from the removable medium 1811. When executed by a Central Processing Unit (CPU) 1801, performs various functions defined in the system of the present invention.
It should be noted that, the computer readable medium shown in the embodiments of the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with a computer-readable computer program embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present invention may be implemented by software, or may be implemented by hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
Specifically, the electronic device of the present embodiment includes a processor and a memory, where the memory stores a computer program, and when the computer program is executed by the processor, the video data transmission method and/or the video data injection method provided in the foregoing embodiments are implemented.
Through the electronic equipment of the embodiment, firstly, service demand information of each virtual machine on a main server is obtained; secondly, according to the service demand information and the computing resources of the main server, virtual machine computing resources of the virtual machine are configured, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources; then acquiring file position information of the video to be injected on the main server; and finally mapping the file position information to the virtual machine so that the virtual machine obtains the video to be decoded from the main server according to the file position information, receives and decodes the video to be decoded by utilizing the configured virtual machine computing resources, and transmits the video to be injected obtained after decoding to a video injection device which is connected with the main server. Through the steps, the method and the device for transmitting the video to be injected have the advantages that a plurality of virtual machines are configured on the main server, then the virtual machines are configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device through the plurality of virtual machines, so that the transmission work of the video data to be injected can be completed only by one main server and one video injection device, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
As another aspect, the present invention also provides a computer-readable storage medium that may be contained in the electronic device described in the above-described embodiment; or may exist alone without being incorporated into the electronic device. The storage medium carries one or more computer programs which, when executed by a processor of the electronic device, cause the electronic device to implement the methods provided in the embodiments described above.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the invention. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present invention may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present invention.
Specifically, through the storage medium of the present embodiment, service requirement information of each virtual machine on a main server is first obtained; secondly, according to the service demand information and the computing resources of the main server, virtual machine computing resources of the virtual machine are configured, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources; then acquiring file position information of the video to be injected on the main server; and finally mapping the file position information to the virtual machine so that the virtual machine obtains the video to be decoded from the main server according to the file position information, receives and decodes the video to be decoded by utilizing the configured virtual machine computing resources, and transmits the video to be injected obtained after decoding to a video injection device which is connected with the main server. Through the steps, the method and the device for transmitting the video to be injected have the advantages that a plurality of virtual machines are configured on the main server, then the virtual machines are configured with proper virtual machine computing resources, and the video to be injected is sent to the video injection device through the plurality of virtual machines, so that the transmission work of the video data to be injected can be completed only by one main server and one video injection device, and the use cost is reduced; and the virtual machine computing resources are configured based on the service demand information, so that unstable video processing caused by simultaneous preemption of the computing resources by a plurality of tasks can be avoided.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains.
It is to be understood that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
Claims (10)
1. A video processing method, comprising:
acquiring service demand information of each virtual machine on a main server;
according to the service demand information and the computing resources of the main server, configuring virtual machine computing resources of the virtual machine, wherein the virtual machine computing resources comprise CPU resources, GPU resources and memory resources;
acquiring file position information of a video to be decoded on the main server;
mapping the file location information to the virtual machine such that:
the virtual machine acquires the video to be decoded from the main server according to the file position information, receives and decodes the video to be decoded by utilizing the configured virtual machine computing resources, and transmits the video to be injected obtained after decoding to a video injection device, wherein the video injection device is connected with the main server.
2. The video processing method according to claim 1, further comprising, after the step of mapping the file location information to the virtual machine:
acquiring reference information, wherein the reference information comprises working state information and/or video data receiving conditions of the virtual machine, and the working state information comprises CPU utilization rate, GPU utilization rate and memory utilization rate;
and adjusting the virtual machine computing resource configuration of the virtual machine according to the reference information.
3. The video processing method according to claim 1, wherein the step of receiving and decoding the video to be decoded using the configured virtual machine computing resources comprises:
and decoding the video to be decoded by the virtual machine by utilizing the configured GPU resources to obtain the video to be injected.
4. The video processing method according to claim 1, wherein the service requirement information includes decoding means information for characterizing whether the virtual machine needs to decode the video to be decoded by GPU hardware, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine needs to decode the video to be decoded through GPU hardware;
if yes, configuring an independent GPU for the virtual machine to serve as the GPU resource.
5. The video processing method according to claim 1, wherein the service requirement information further includes response instantaneity for characterizing a program instruction response speed of the virtual machine, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine needs to improve response instantaneity or not;
if yes, configuring an exclusive CPU core for the virtual machine.
6. The video processing method according to claim 1, wherein the service requirement information further includes information for characterizing a data bandwidth of the video to be decoded, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
judging whether the virtual machine computing resources of the virtual machine are matched with the data bandwidth or not;
if yes, configuring memory resources with memory capacity larger than a first threshold for the virtual machine, and if not, configuring memory resources with memory capacity smaller than a second threshold for the virtual machine.
7. The video processing method according to claim 1, wherein the service requirement information further includes a resolution of the video to be decoded, and the step of configuring virtual machine computing resources of the virtual machine according to the service requirement information includes:
and configuring the memory capacity of the virtual machine according to the resolution of the video to be decoded.
8. The video processing method according to any one of claims 1 to 7, further comprising, after the step of configuring virtual machine computing resources of the virtual machine according to the traffic demand information:
synchronizing a virtual machine clock source of the virtual machine with a master server clock source of the master server, so that the virtual machine transmits the video to be injected to the video injection device according to the virtual machine clock source and the time stamp of the video to be injected.
9. A video injection method, comprising:
receiving video to be injected, the video data to be injected being obtained according to the video processing method of any one of claims 1 to 8;
and injecting the video to be injected into a video receiving device according to the time stamp of the video to be injected.
10. An electronic device comprising a processor and a memory, the memory having stored thereon a computer program which, when executed by the processor, implements the video processing method of any one of claims 1 to 8, or the video injection method of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310096728.8A CN116132706A (en) | 2023-02-07 | 2023-02-07 | Video processing method, injection method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310096728.8A CN116132706A (en) | 2023-02-07 | 2023-02-07 | Video processing method, injection method and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116132706A true CN116132706A (en) | 2023-05-16 |
Family
ID=86307861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310096728.8A Pending CN116132706A (en) | 2023-02-07 | 2023-02-07 | Video processing method, injection method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116132706A (en) |
-
2023
- 2023-02-07 CN CN202310096728.8A patent/CN116132706A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI483213B (en) | Integrated gpu, nic and compression hardware for hosted graphics | |
| CN102981887B (en) | Data processing method and electronic equipment | |
| CN104837052A (en) | Playing method of multimedia data and device | |
| US20120120967A1 (en) | Universal Serial Interface | |
| CN102176213A (en) | FPGA (Field Programmable Gate Array)-based AVS (Audio Video Standard) decoding chip verification platform device and method | |
| US11671678B2 (en) | Method and device, equipment, and storage medium for data processing | |
| CN113766270B (en) | Video playing method, system, server, terminal equipment and electronic equipment | |
| CN104951397A (en) | Mobile equipment testing method, mobile equipment testing device, mobile equipment testing server and mobile equipment testing system | |
| CN111464828A (en) | Virtual special effect display method, device, terminal and storage medium | |
| CN112948151A (en) | Program interface image transmission method, substrate management controller and remote control equipment | |
| CN116400998A (en) | Video hardware acceleration device and method suitable for virtual display card | |
| CN114926318A (en) | Multi-GPU aggregation method, image processing device, electronic equipment and storage medium | |
| CN116437161B (en) | Video data processing method, injection method, system, equipment and storage medium | |
| CN116132706A (en) | Video processing method, injection method and electronic equipment | |
| KR20180102605A (en) | Image drawing methods, associated devices and systems | |
| US11729349B2 (en) | Method, electronic device, and computer program product for video processing | |
| CN114554269A (en) | Data processing method, electronic device and computer readable storage medium | |
| CN113766216A (en) | Verification method and verification system for video module | |
| CN113327302A (en) | Picture processing method and device, storage medium and electronic device | |
| CN113766266A (en) | Audio and video processing method, device, equipment and storage medium | |
| CN112995543A (en) | Distributed video switching system, method and equipment | |
| CN111711800A (en) | Image processing method and device for tiled display screen and storage medium | |
| CN110969672A (en) | Image compression method and device | |
| CN110809158A (en) | Image loop filtering processing method and device | |
| CN114245108B (en) | DisplayPort test system supporting custom extension |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |