CN108924471A - A kind of conference system based on QSV fast video encoding and decoding - Google Patents
A kind of conference system based on QSV fast video encoding and decoding Download PDFInfo
- Publication number
- CN108924471A CN108924471A CN201811067169.3A CN201811067169A CN108924471A CN 108924471 A CN108924471 A CN 108924471A CN 201811067169 A CN201811067169 A CN 201811067169A CN 108924471 A CN108924471 A CN 108924471A
- Authority
- CN
- China
- Prior art keywords
- rtp
- module
- data
- qsv
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 59
- 238000005457 optimization Methods 0.000 claims abstract description 36
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 230000000694 effects Effects 0.000 claims description 10
- 238000000034 method Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 claims description 7
- 238000002844 melting Methods 0.000 claims description 3
- 230000008018 melting Effects 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 230000008901 benefit Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The present invention provides a kind of conference system based on QSV fast video encoding and decoding, including QSV hardware accelerator, video optimization modules and a plurality of RTP processing modules, each terminal is connect by a RTP processing module with QSV hardware accelerator, and QSV hardware accelerator is connect with video optimization modules；The QSV hardware accelerator includes coding module and decoder module；Terminal and conference system carry media data flow by rtp streaming, conference system receives the rtp streaming that terminal comes by RTP processing module, after RTP processing module handles rtp streaming, it is put into decoder module and is decoded, decoded data are sent to video optimization modules and optimize processing, then the data after optimization are put into coding module and are performed the encoding operation, and pass through RTP processing module, data after coding are packaged into RTP data, and are sent in corresponding terminal.CPU usage is effectively reduced in conference system of the present invention, can freely use video processing function, reduces video processing cost and improves system effectiveness simultaneously.
Description
Technical field
The present invention relates to video conferencing technology field more particularly to a kind of meeting systems based on QSV fast video encoding and decoding
System.
Background technique
Intel QSV (Quick Sync Video) technology is that a kind of hardware based on Intel GPU that Intel is released is compiled
Decoding scheme.The GPU function that by using the technology, the utilization rate of CPU can be effectively reduced, and rationally utilized Intel included
Energy.
It is largely the support that software codec is carried out using CPU, but software in existing conference system/recording and broadcasting system
Requirement of the coding and decoding scheme to CPU is relatively high, and the CPU encoding and decoding performance of some low and middle-ends is insufficient, and leading to meeting can only be audio
Meeting；Some is chip to be handled using dedicated coding and decoding video, and dedicated coding and decoding video handles chip price
It can not have always been high any more.Using hardware to there is no software processing on some special Video Compositions and some image quality optimizations and noise reduction
Come effect is good.
Summary of the invention
The technical problem to be solved in the present invention is to provide a kind of conference system based on QSV fast video encoding and decoding, have
Effect reduces CPU usage, can freely use video processing function, reduces video processing cost and improves system effectiveness simultaneously.
The invention is realized in this way：A kind of conference system based on QSV fast video encoding and decoding, including QSV hardware add
Fast module, video optimization modules and a plurality of RTP processing modules, each terminal are added by a RTP processing module with QSV hardware
Fast module connection, the QSV hardware accelerator are connect with video optimization modules；
The QSV hardware accelerator includes a coding module and a decoder module；
The RTP module is used for, and handles the rtp streaming that terminal sends over is received, processing result is put into decoding
Module, and terminal is sent to after the data from coding module received are packaged into RTP data；
The decoder module, the data for sending to RTP module are decoded, and decoding result is sent to video
Optimization module；
The video optimization modules, for carrying out special effect processing to the decoder module decoding result, melting screen processing, image quality
Enhancing processing and subtitle processing；
The coding module for encoding to the data from video optimization modules, and is sent out by RTP processing module
Give terminal；
Terminal and conference system carry media data flow by rtp streaming, and conference system is received eventually by RTP processing module
Hold the rtp streaming that comes, after RTP processing module handles rtp streaming, be put into the decoder module of QSV hardware accelerator into
Row decoding, decoded data are sent to video optimization modules and optimize processing, then the data after optimization are put into QSV hardware
It is performed the encoding operation in the coding module of accelerating module, and by RTP processing module, the data after coding is packaged into RTP number
According to, and be sent in corresponding terminal.
Further, described " to handle the rtp streaming that terminal sends over is received, processing result is put into decoding mould
Block " specifically includes：The RTP data of H264 are unpacked into the uncorrected data for H264RAW, and uncorrected data is filled into QSV's
MfxBitstream data structure is sent to decoder module.
Further, in the decoder module decoding process, when the RTP message amount received is unable to complete decoding,
It returns and needs more multidata as a result, the data for going RTP module to obtain H264 over there again export NV12 until successfully decoded
Data.
Further, the coding module obtains after the NV12 data to video optimization modules complete coding from surface
The uncorrected data for getting H264RAW is sent to RTP module and carries out RTP package, after package, sends terminal for message.
Further, the system also includes：
Conversion module, the data conversion of the yuv format of the NV12 for decoding decoder module is at YUV420 format
Data for video optimization modules processing, and the data conversion of YUV420 format that video optimization modules are handled at
The yuv format of NV12 is encoded for coding module.
The invention has the advantages that：The encoding and decoding of video conferencing system are carried out using the QSV technology of GPU and use software
Module realizes video optimized processing, realizes video conferencing system jointly in conjunction with software and hardware processing module, can effectively reduce making for CPU
With rate and then meeting number formulary (i.e. the capacity of meeting) is improved, while realizing that more preferably image quality optimization and special video effect are supported；CPU
After free time, the pressure of equipment is reduced, and more video image quality/special effect processings and media forwarding may be implemented in the CPU of surplus
The coding-decoding operation to the data of video can be completed without using dedicated coding and decoding video processing chip in business, the present invention,
Reduce cost.
Detailed description of the invention
The present invention is further illustrated in conjunction with the embodiments with reference to the accompanying drawings.
Fig. 1 is a kind of logic diagram of the conference system based on QSV fast video encoding and decoding of the present invention.
Fig. 2 is decoding process execution flow chart in one embodiment of the invention.
Fig. 3 is cataloged procedure execution flow chart in one embodiment of the invention.
Fig. 4 is present system conversion module logical schematic.
Specific embodiment
As shown in Figure 1, a kind of conference system based on QSV fast video encoding and decoding of the invention, including QSV are hardware-accelerated
Module, video optimization modules and a plurality of RTP processing modules, each terminal are hardware-accelerated by a RTP processing module and QSV
Module connection, the QSV hardware accelerator are connect with video optimization modules；
The QSV hardware accelerator includes a coding module and a decoder module；
The RTP module is used for, and handles the rtp streaming that terminal sends over is received, processing result is put into decoding
Module, and terminal is sent to after the data from coding module received are packaged into RTP data；
The decoder module, the data for sending to RTP module are decoded, and decoding result is sent to video
Optimization module；
The video optimization modules, for carrying out special effect processing to the decoder module decoding result, melting screen processing, image quality
Enhancing processing and subtitle processing；
The coding module, for being encoded to the data from video optimization modules (by calling QSV hardware-accelerated
Module realization carries out coding acceleration), and terminal is sent to by RTP processing module；
Terminal and conference system carry media data flow by rtp streaming, and conference system is received eventually by RTP processing module
Hold the rtp streaming that comes, after RTP processing module handles rtp streaming, be put into the decoder module of QSV hardware accelerator into
Row decoding, decoded data are sent to video optimization modules and optimize processing, then the data after optimization are put into QSV hardware
It is performed the encoding operation in the coding module of accelerating module, and by RTP processing module, the data after coding is packaged into RTP number
According to, and be sent in corresponding terminal.
Wherein, described " handling the rtp streaming that terminal sends over is received, processing result the is put into decoder module " tool
Body includes：The RTP data of H264 are unpacked into the uncorrected data for H264RAW, and uncorrected data is filled into the mfxBitstream of QSV
Data structure is sent to decoder module.
Wherein, it in the decoder module decoding process, when the RTP message amount received is unable to complete decoding, returns
Need more multidata as a result, the data for going RTP module to obtain H264 over there again export the number of NV12 until successfully decoded
According to.
Wherein, the coding module is got after the NV12 data to video optimization modules complete coding from surface
The uncorrected data of H264RAW is sent to RTP module and carries out RTP package, after package, sends terminal for message.
Preferably, the system also includes：
Conversion module, the data conversion of the yuv format of the NV12 for decoding decoder module is at YUV420 format
Data for video optimization modules processing, and the data conversion of YUV420 format that video optimization modules are handled at
The yuv format of NV12 is encoded for coding module.By QSV encoding and decoding relied on be NV12 yuv format data, and it is current
The video conferencing system of mainstream is mainly YUV420 for the processing format of video original data, therefore QSV decodes the number come
According to doing a conversion, and the data of YUV420 after video conferencing system is handled will also do a conversion, specific to turn
It is as shown in Figure 4 to change logic.
For meeting, maximum pressure is in encoding and decoding processing.Therefore QSV is hardware-accelerated, just plays here
Important function.It is briefly described below with the decoding process of H264 all the way, as shown in Fig. 2, the process includes
Following steps：
Step 1, RTP processing module receive corresponding terminal data, by RTP data decompression at H264RAW data；
Step 2, H264RAW data fill mfxBitstream, and wherein mfxBitstream is in SDK provided by QSV
The data structure in face；
Step 3, acquisition can use surface (i.e. abstract knot inside SDK provided by QSV about a frame image data
Structure)；
Step 4 is decoded operation, specifically, data are imparted to QSV hardware accelerator, hardware by surface
Accelerating module is decoded operation to incoming data, that is, data are sent into QSV, and QSV is allowed to carry out hardware-accelerated decoding；
Step 5 judges whether decoding result succeeds, if so, entering step 6, otherwise obtains in return step 1 more
Data are decoded；
Step 6, execution simultaneously operating (i.e. in the API of QSV, call directly and execute synchronous operation, have with confirmation output
The data of effect), export the data of NV12.
After the data of above-mentioned terminal reach server by RTP, RTP module, which needs to unpack the RTP data of H264, is
Uncorrected data is filled into the mfxBitstream data of QSV because QSV can only decode uncorrected data by the uncorrected data of H264RAW later
In structure, it is sent in decoding API later and does decoding, since the RTP data of a H264 may be divided into the report of multiple RTP
Text, therefore it is more multidata as a result, at this time just needing that RTP module is gone to obtain H264's over there again to have return needs here
Data can export the data of NV12 after successfully decoded.
As shown in figure 3, being a process of coding, since coding is complicated not as decoding, it is only necessary to by video software
NV12 data that treated are sent into QSV and are encoded, after encoding successfully, so that it may get the naked number of H264RAW from surface
According to, due to uncorrected data cannot be directly used to RTP send, at this time by uncorrected data be sent to RTP module carry out RTP package, envelope
Bao Hou, so that it may send terminal for message.
Present system rationally utilizes the QSV technology of GPU, and the utilization rate of CPU is effectively reduced and then improves meeting number formulary, benefit
It is handled with the QSV technology of GPU and the video of conference system, mutually takes strong point, reach more preferably image quality optimization and special video effect branch
It holds, after CPU is idle, reduces the pressure of equipment and idle CPU can be allowed to go to run some additional business, it can be achieved that more
Video image quality/special effect processing and media forwarding service.
Although specific embodiments of the present invention have been described above, those familiar with the art should be managed
Solution, we are merely exemplary described specific embodiment, rather than for the restriction to the scope of the present invention, it is familiar with this
The technical staff in field should be covered of the invention according to modification and variation equivalent made by spirit of the invention
In scope of the claimed protection.
Claims (5)
1. a kind of conference system based on QSV fast video encoding and decoding, it is characterised in that：Including QSV hardware accelerator, video
Optimization module and a plurality of RTP processing modules, each terminal are connect by a RTP processing module with QSV hardware accelerator,
The QSV hardware accelerator is connect with video optimization modules；
The QSV hardware accelerator includes a coding module and a decoder module；
The RTP module is used for, and handles the rtp streaming that terminal sends over is received, processing result is put into decoder module,
And terminal is sent to after the data from coding module received are packaged into RTP data；
The decoder module, the data for sending to RTP module are decoded, and decoding result are sent to video optimized
Module；
The video optimization modules, for carrying out special effect processing to the decoder module decoding result, melting screen processing, picture quality enhancement
Processing and subtitle processing；
The coding module for encoding to the data from video optimization modules, and is sent to by RTP processing module
Terminal；
Terminal and conference system carry media data flow by rtp streaming, and conference system receives terminal mistake by RTP processing module
The rtp streaming come, after RTP processing module handles rtp streaming, is put into the decoder module of QSV hardware accelerator and is solved
Code, decoded data are sent to video optimization modules and optimize processing, then that the data after optimization are put into QSV is hardware-accelerated
It is performed the encoding operation in the coding module of module, and by RTP processing module, the data after coding is packaged into RTP data, and
It is sent in corresponding terminal.
2. a kind of conference system based on QSV fast video encoding and decoding according to claim 1, it is characterised in that：It is described
" handling the rtp streaming that terminal sends over is received, processing result is put into decoder module " specifically includes：By H264's
The uncorrected data that it is H264RAW that RTP data, which unpack, and the mfxBitstream data structure that uncorrected data is filled into QSV is sent to
Decoder module.
3. a kind of conference system based on QSV fast video encoding and decoding according to claim 1, it is characterised in that：It is described
In decoder module decoding process, when the RTP message amount received is unable to complete decoding, returns and need more multidata knot
Fruit goes RTP module to obtain the data of H264 over there again, until successfully decoded, exports the data of NV12.
4. a kind of conference system based on QSV fast video encoding and decoding according to claim 1, it is characterised in that：It is described
Coding module gets the uncorrected data hair of H264RAW from surface after the NV12 data to video optimization modules complete coding
It gives RTP module and carries out RTP package, after package, send terminal for message.
5. a kind of conference system based on QSV fast video encoding and decoding according to claim 1, it is characterised in that：It is described
System further includes：
Conversion module, for by decoder module decode come NV12 yuv format data conversion at YUV420 format number
Handled according to for video optimization modules, and the YUV420 format that video optimization modules are handled data conversion at NV12's
Yuv format is encoded for coding module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811067169.3A CN108924471A (en) | 2018-09-13 | 2018-09-13 | A kind of conference system based on QSV fast video encoding and decoding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811067169.3A CN108924471A (en) | 2018-09-13 | 2018-09-13 | A kind of conference system based on QSV fast video encoding and decoding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108924471A true CN108924471A (en) | 2018-11-30 |
Family
ID=64407734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811067169.3A Pending CN108924471A (en) | 2018-09-13 | 2018-09-13 | A kind of conference system based on QSV fast video encoding and decoding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108924471A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115546329A (en) * | 2022-12-01 | 2022-12-30 | 广州市保伦电子有限公司 | Audio and video hardware accelerated transcoding detection method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102625079A (en) * | 2012-03-21 | 2012-08-01 | 厦门亿联网络技术有限公司 | Video implementation method for trilateral video conference |
| CN103491379A (en) * | 2013-09-25 | 2014-01-01 | 浙江宇视科技有限公司 | Method and device for improving intelligent analysis performance |
| US20140321554A1 (en) * | 2013-04-26 | 2014-10-30 | Kin-Hang Cheung | Transcoding management techniques |
| CN106231319A (en) * | 2016-07-14 | 2016-12-14 | 观止云（北京）信息技术有限公司 | A kind of method alignd frame by frame based on software and hardware combining |
| CN108134917A (en) * | 2017-12-20 | 2018-06-08 | 福建星网智慧软件有限公司 | A kind of video-meeting method based on Intel QSV technologies |
-
2018
- 2018-09-13 CN CN201811067169.3A patent/CN108924471A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102625079A (en) * | 2012-03-21 | 2012-08-01 | 厦门亿联网络技术有限公司 | Video implementation method for trilateral video conference |
| US20140321554A1 (en) * | 2013-04-26 | 2014-10-30 | Kin-Hang Cheung | Transcoding management techniques |
| CN103491379A (en) * | 2013-09-25 | 2014-01-01 | 浙江宇视科技有限公司 | Method and device for improving intelligent analysis performance |
| CN106231319A (en) * | 2016-07-14 | 2016-12-14 | 观止云（北京）信息技术有限公司 | A kind of method alignd frame by frame based on software and hardware combining |
| CN108134917A (en) * | 2017-12-20 | 2018-06-08 | 福建星网智慧软件有限公司 | A kind of video-meeting method based on Intel QSV technologies |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115546329A (en) * | 2022-12-01 | 2022-12-30 | 广州市保伦电子有限公司 | Audio and video hardware accelerated transcoding detection method and system |
| CN115546329B (en) * | 2022-12-01 | 2023-05-16 | 广东保伦电子股份有限公司 | Audio and video hardware acceleration transcoding detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103179431B (en) | Under VDI environment, audio frequency and video are redirected and turn code separation method | |
| CN102480618B (en) | Realize H264 video code model and play the method and system optimized | |
| US20110085602A1 (en) | Video Communication System, Device and Method Based on Feedback Reference Frames | |
| WO2011137786A1 (en) | Method, system and media resource server for playing video file | |
| CN102802024A (en) | Transcoding method and transcoding system realized in server | |
| CN104253996B (en) | The sending, receiving method and its device and Transmission system of video data | |
| US6744741B1 (en) | System and method for maintaining a plurality of media conferences | |
| CN102625079B (en) | Video implementation method for trilateral video conference | |
| CN109618170B (en) | D2D real-time video streaming transmission method based on network coding | |
| CN104093028A (en) | Device capacity negotiation method and device | |
| CN102802022A (en) | Coding and decoding system and method | |
| CN103503444A (en) | Signaling number of active layers in video coding | |
| CN109217980B (en) | Encoding and decoding capacity configuration method, device and computer storage medium | |
| CN104244025A (en) | Cluster transcoding system and method thereof | |
| CN112087631A (en) | Synchronous parallel video coding and decoding and streaming media transmission system and method based on GPU | |
| US20190356911A1 (en) | Region-based processing of predicted pixels | |
| CN107229516B (en) | Data processing method and device | |
| CN101282477A (en) | Method and system for processing multicore DSP array medium based on RapidIO interconnection | |
| CN108924471A (en) | A kind of conference system based on QSV fast video encoding and decoding | |
| CN114501025A (en) | Video transcoding method, device, equipment and medium | |
| US20140092995A1 (en) | Electronic devices for signaling a random access point picture message | |
| CN110557224A (en) | Unidirectional transmission device and system for power network | |
| CN112910899A (en) | Audio and video transparent transmission method under android system | |
| US20230389032A1 (en) | Early notification for transmission of encoded video data | |
| US11831430B2 (en) | Methods and apparatuses for encoding and decoding signal frame |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181130 |
|
| WD01 | Invention patent application deemed withdrawn after publication |