CN102625079A - Video implementation method for trilateral video conference - Google Patents
Video implementation method for trilateral video conference Download PDFInfo
- Publication number
- CN102625079A CN102625079A CN2012100759342A CN201210075934A CN102625079A CN 102625079 A CN102625079 A CN 102625079A CN 2012100759342 A CN2012100759342 A CN 2012100759342A CN 201210075934 A CN201210075934 A CN 201210075934A CN 102625079 A CN102625079 A CN 102625079A
- Authority
- CN
- China
- Prior art keywords
- video
- data
- decoding
- hosting
- rtp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention discloses a video implementation method for a trilateral video conference. A conducting party captures and codes a trilateral video combined on a screen of the conducting party by utilizing display resources of the screen, and transmits the coded trilateral video to conference participants, and screen capture data transmitted by the conducting party is displayed on screens of the conference participants respectively. Therefore, resource occupation for the mixing of video data is avoided, video resource consumption can be controlled, and three parties of the conference can be audible and visible for one another in a low-cost hardware running environment.
Description
Technical field
The present invention relates to a kind of video implementation method of 3 side video conference.
Background technology
At present support that on the market the equipment of tripartite talks function is more; But great majority are voice conference device; And video conference device often price is higher, this mainly is owing to most of realize that the 3 side video conference function algorithm is higher to hardware requirement, thereby makes whole cost promote.
Tradition tripartite talks equipment basically all is to adopt hostings side to receive all the other two number formularies to pass to other two sides after the RTP data and realize according to mixing, and wherein each is different for the implementation method of audio & video, is directed against these two kinds realization types below and analyzes:
1, the implementation method of conventional audio meeting:
As shown in Figure 1, the direct Data transmission of A and C side's reality, but pass through hosting side B as terminal; With third-party data with pass to second party after the data of B side self are mixed; Therefore, the processing of the side of hosting B is particularly important, but because processing of audio data is comparatively simple; Therefore, only need two threads of many establishments that data are mixed and can be met the demands in principle.Tripartite audio conferencing conversation step for hosting side B is:
(1) hosting side B creates the Socket to A, C two sides' RTP receiving port, and monitors wait;
(2) sound that will the side of hosting B self be sent is sampled;
(3) hosting side B receives the RTP voice data that A side transmitted, and the decoding back emits in loud speaker, and with after decoded voice data and B side self sound mix, is packaged as the RTP receiving port that the RTP packet sends to C side when consulting network parameter;
(4) (parallel with step (3)) hosting side B receives the RTP voice data that C side transmitted; The decoding back emits in loud speaker; And, be packaged as the RTP receiving port that the RTP packet sends to A side when consulting network parameter with after the mixing of decoded voice data and hosting side B self;
(5) like this, the sound that the sound that the sound that A side hears just mixes for B/C two sides, the sound that B side hears then mix for A/C both sides, the sound that the sound that C side hears mixes for A/B both sides, thus form tripartite audio conferencing.
2, the implementation method of conventional video meeting:
Under the video conference situation, if do not consider hardware cost, adopting as above, the scheme of audio conferencing also can realize; And can reach the effect that participant is seen another participant and hosting side; But because this scheme of use is higher to hardware requirement, and present most video phone does not all reach this requirement, therefore; Phone on the market basically all moves back and asks its it; As long as the tripartite sound that can both hear all the other two sides, as long as and participant can be seen the video of hosting side, as long as wherein arbitrary side's video also can be checked simultaneously by hosting side.
Take in the 3 side video conference resource at most also complicated be blended data, therefore, as shown in Figure 2; The side of hosting B has removed the process of mixed video data; And be distinguishing for the processing of A/C two number formulary certificates, mainly be because for video calling, receiving the shared CPU of data is less with respect to decoded data; Therefore; Save resource and the most effectively reduces decoding, so removed the decoding of the C number formulary certificate of non-current activation, only keep key frame to make things convenient for follow-up decoding when checking again normal.Therefore, the cpu resource of this moment takies and is the pure video calling of the tripartite audio conferencing+single channel+shared CPU summation of single channel RTP Data Receiving, and basic just conversation takies the resource loss of audio mix more compared with the single channel audio frequency and video.
For the 3 side video conference of hosting side B conversation step (the audio-frequency unit processing is referring to the implementation method of top audio conferencing) as follows:
(1) hosting side B creates the Socket to A, C two side RTP receiving ports, and monitors wait;
(2) video image that will the side of hosting B self is sampled, and is packaged as the RTP packet, sends to A, C two sides' when consulting network parameter RTP receiving port respectively;
(3) supposition hosting side B is provided with the main video that A side's video is current demonstration, and then the side of hosting B receives the RTP data that A side transmitted, and the decoding back shows on the FrameBuffer of display screen the user can be viewed;
(4) (parallel with step (3)) B side receives the RTP data that C side transmitted, but does not decode, and just stores key frame I frame data, is used for that follow-up to switch main video be the compensation of decoding behind the C side, avoids occurring when just showing mosaic phenomenon;
(5) if it is C side that hosting side B is provided with main video at this moment, then the side of hosting B utilizes communication control processor (SIP or custom protocol) request C side to retransmit the I frame, thereby upgrades up-to-date video image, and A, C two number formularies are exchanged according to processing mode.
Though top scheme has solved the problem of inadequate resource, but still has following problem:
A, side's participant can't view the video information of another participant, lose the most important principle of tripartite talks: the three parts can listen visible mutually;
B, hosting side can only view a certain participant video simultaneously, if need check that the video of another participant will switch, user's operation is trouble comparatively.
Summary of the invention
The object of the present invention is to provide and a kind ofly can control the video resource loss, under hardware running environment cheaply, the video implementation method that meeting three can be listened visible 3 side video conference mutually simultaneously.
The video implementation method of a kind of video tripartite talks of the present invention specifically comprises the steps:
Step 1, hosting side B create the Socket to the RTP receiving port of two A of conferenced party, C, and monitor wait;
Step 2, hosting side B sample the video image of self, and the viewing area of the corresponding B side on the screen of hosting side B shows the video of B side;
Step 3, hosting side B receive the RTP data that the A of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding A of conferenced party demonstration A side;
Step 4, (parallel with the step 3) side of hostinging B receive the RTP data that the C of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding C of conferenced party demonstration C side;
The associating viewing area of being made up of the tripartite viewing area of A, B, C on step 5, hosting side B intercepting self screen is packaged as the RTP packet, and the RTP receiving port of two A of conferenced party, C when it is sent to the negotiation network parameter respectively;
Step 6, this two meetings participant A, C will show on screen separately after the screenshotss RTP data decode from hosting side B.
The present invention further will receive and dispatch bag and separate with the treatment step of encoding and decoding:
The side of hosting B has created Socket in above-mentioned steps 1 after; Create separate threads T1 and be used to receive RTP packet from A or C side; And it is left in respectively in two different public queues; And the data with public queue before reception add the visit lock, will visit lock after finishing receiving and discharge, and this visit lock is used for conducting interviews synchronously with following decoding thread;
Another independent decoding thread T2 is through regularly obtaining the data in above-mentioned two public queues; After all data taking-ups in two public queues; The a plurality of packets that will be referred to whole two field picture are decoded simultaneously; Before fetching data, pass through to judge the quantity of data packets that overstocks in public queue this moment like this, as surpassing threshold value, the thread T2 that then decodes removes direct decoding I frame with the non-key frame of centre after this situation appearance at every turn; Also need the data of public queue be added the visit lock before taking out decoding, will visit lock after decoding is accomplished and discharge;
Decoding thread T2 quantity of data packets through overstocking in judgement public queue this moment before at every turn fetching data in the decode procedure; If surpass threshold value; Then should directly send RTP packet rate control command, the transmission rate that request slows down the RTP packet to data receiver by decoding thread T2 through the RTCP signaling;
Cataloged procedure is also with above-mentioned step process.
The present invention utilizes the demonstration resource of screen; To behind the tripartite video intercepting that combination is accomplished on hosting side's screen, encode and send to conferenced party; On the screen separately of conferenced party, show the screenshotss data of sending this hosting side respectively, removed the resource occupation of mixed video data like this from.
Description of drawings
Fig. 1 is the principle schematic of conventional audio meeting implementation;
Fig. 2 is the principle schematic of conventional video meeting implementation;
Fig. 3 is a schematic flow sheet of the present invention;
Fig. 4 is the further improved principle schematic of the present invention;
Fig. 5 is the principle schematic of video conference implementation of the present invention;
Below in conjunction with accompanying drawing and specific embodiment the present invention is done further detailed description.
Embodiment
Like Fig. 3, shown in 5, the video implementation method of a kind of video tripartite talks of the present invention specifically comprises the steps:
Step 1, hosting side B create the Socket to the RTP receiving port of two A of conferenced party, C, and monitor wait;
Step 2, hosting side B sample the video image of self, and the viewing area of the corresponding B side on the screen of hosting side B shows the video of B side;
Step 3, hosting side B receive the RTP data that the A of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding A of conferenced party demonstration A side;
Step 4, (parallel with the step 3) side of hostinging B receive the RTP data that the C of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding C of conferenced party demonstration C side;
The associating viewing area of forming by A, B, the tripartite viewing area of C on step 5, hosting side B intercepting self screen (as shown in Figure 3 can be rectangle); And be packaged as the RTP packet, send to when consulting network parameter the RTP receiving port of two A of conferenced party, C respectively;
Step 6, this two meetings participant A, C will show on screen separately after the screenshotss RTP data decode from hosting side B.
The implementation method of a kind of video tripartite talks of the present invention, hosting side utilizes screen display resource, sends to conferenced party after the tripartite video intercepting of combination completion is encoded; Remove the resource occupation of mixed video data like this from, as shown in Figure 3, take in the calculating and can find out from this cpu resource; Though the present invention has satisfied the tripartite requirement that can check the other side's video mutually, the resource occupation of this moment still has more the decoding of one road video calling than other phones, and for video calling; The decoding of video also is higher to resource requirement; Especially under H264-Codec, and H264 has become the first-selected Codec of video calling at present basically, therefore; Need further to improve, resources occupation rate is descended once more.
As shown in Figure 3, B cpu resource at this moment in the side of hosting takies: the pure video calling decoding+screenshotss of tripartite audio conferencing+two-way+single channel video coding+basic network transmitting-receiving bag summation.Take from cpu resource of the present invention,, when realizing, use simultaneously the metadata cache mode to reach optimal case basically because that the screenshotss own resources take is lower; Therefore emphasis still is in the utilization of decode resources, improves decoding algorithm and address this problem the most directly method yes, but because the present decoding algorithm that adopts has been comparatively ripe algorithm; Simultaneously if revise algorithm; Then possibly bring more compatibility issue, therefore, consider from another point of view; On the resource occupation when break-through point is placed on the reduction decoding; And because the operation of carrying out for a long time this moment has only coding and Network Transmission, and the modification of coding and codec class are seemingly, so consider from the resource occupation of Network Transmission.
At first, network transmission speed can not be reduced, secondly, also the transmission of network packet can not be removed, but can be from avoiding the consideration that takies of Network Transmission peak value.Because network sends with reception and encoding and decoding and binds before,, therefore, cause the speed of decoding can receive the influence of receiving and dispatching bag promptly in same thread process.
As shown in Figure 4, the present invention further will receive and dispatch bag and separate with the treatment step of encoding and decoding:
The side of hosting B has created Socket in said step 1 after; Create separate threads T1 and be used to receive RTP packet from A or C side; And it is left in respectively in two different public queues; And the data with public queue before reception add the visit lock; To visit lock after finishing receiving and discharge, this visit lock is used for conducting interviews synchronously with following decoding thread;
Another independent decoding thread T2 is through regularly obtaining the data in above-mentioned two public queues; After all data taking-ups in two public queues; A plurality of packets (promptly whole two field picture is generally about 6 packets) are decoded simultaneously, like this data volume through overstocking in judgement public queue this moment before at every turn fetching data; As surpassing 20 packets; The thread of then decoding is removed direct decoding I frame with the non-key frame (being non-I frame) of centre after this situation occurs, avoid owing to packet too much causes this locality frequently to unpack, thereby CPU takies too high phenomenon; Also need the data of public queue be added the visit lock before taking out decoding, will visit lock after decoding is accomplished and discharge.
Decoding thread T2 data volume through overstocking in judgement public queue this moment before at every turn fetching data in the decode procedure; As surpassing 30 packets; Then this thread directly sends RTP packet rate control command through the RTCP signaling to data receiver; The transmission rate that request slows down the RTP packet is avoided out of buffers.
The also similar processing of cataloged procedure.
Like this; Can avoid to receive and the decoding serial through same thread originally through the mode of a plurality of data together decode; Cause data to overstock, wait for, take the cpu resource problem of higher thereby frequently get bag; Also can come the speed of request msg transmit leg RTP packet transmission according to the amount of current overstocked data, thereby can control the overall transfer bandwidth through the decoding thread.
The above; It only is preferred embodiment of the present invention; Be not that technical scope of the present invention is done any restriction, so every foundation technical spirit of the present invention all still belongs in the scope of technical scheme of the present invention any trickle modification, equivalent variations and modification that above embodiment did.
Claims (2)
1. the video implementation method of video tripartite talks is characterized in that comprising the steps:
Step 1, hosting side B create the Socket to the RTP receiving port of two A of conferenced party, C, and monitor wait;
Step 2, hosting side B sample the video image of self, and the viewing area of the corresponding B side on the screen of hosting side B shows the video of B side;
Step 3, hosting side B receive the RTP data that the A of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding A of conferenced party demonstration A side;
Step 4, (parallel with the step 3) side of hostinging B receive the RTP data that the C of conferenced party is transmitted, after decoding on the screen of hosting side B the video of the viewing area of the corresponding C of conferenced party demonstration C side;
The associating viewing area of being made up of the tripartite viewing area of A, B, C on step 5, hosting side B intercepting self screen is packaged as the RTP packet, and the RTP receiving port of two A of conferenced party, C when it is sent to the negotiation network parameter respectively;
Step 6, this two meetings participant A, C will show on screen separately after the screenshotss RTP data decode from hosting side B.
2. the video implementation method of a kind of video tripartite talks according to claim 1 is characterized in that further will receiving and dispatching bag and separates with the treatment step of encoding and decoding:
The side of hosting B has created Socket in above-mentioned steps 1 after; Create separate threads T1 and be used to receive RTP packet from A or C side; And it is left in respectively in two different public queues; And the data with public queue before reception add the visit lock, will visit lock after finishing receiving and discharge, and this visit lock is used for conducting interviews synchronously with following decoding thread;
Another independent decoding thread T2 is through regularly obtaining the data in above-mentioned two public queues; After all data taking-ups in two public queues; The a plurality of packets that will be referred to whole two field picture are decoded simultaneously; Before fetching data, pass through to judge the quantity of data packets that overstocks in public queue this moment like this, as surpassing threshold value, the thread T2 that then decodes removes direct decoding I frame with the non-key frame of centre after this situation appearance at every turn; Also need the data of public queue be added the visit lock before taking out decoding, will visit lock after decoding is accomplished and discharge;
Decoding thread T2 quantity of data packets through overstocking in judgement public queue this moment before at every turn fetching data in the decode procedure; If surpass threshold value; Then should directly send RTP packet rate control command, the transmission rate that request slows down the RTP packet to data receiver by decoding thread T2 through the RTCP signaling;
Cataloged procedure is also with above-mentioned step process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210075934.2A CN102625079B (en) | 2012-03-21 | 2012-03-21 | Video implementation method for trilateral video conference |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210075934.2A CN102625079B (en) | 2012-03-21 | 2012-03-21 | Video implementation method for trilateral video conference |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102625079A true CN102625079A (en) | 2012-08-01 |
| CN102625079B CN102625079B (en) | 2015-01-14 |
Family
ID=46564742
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210075934.2A Active CN102625079B (en) | 2012-03-21 | 2012-03-21 | Video implementation method for trilateral video conference |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102625079B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349120A (en) * | 2013-07-26 | 2015-02-11 | 北京计算机技术及应用研究所 | Audio and video decoding system and decoding method thereof |
| CN105095123A (en) * | 2014-05-19 | 2015-11-25 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN108234432A (en) * | 2016-12-22 | 2018-06-29 | 展讯通信（上海）有限公司 | Media implementation method, device, mostly logical terminal and the media server of more logical terminals |
| CN108419125A (en) * | 2018-03-08 | 2018-08-17 | 弘成科技发展有限公司 | The long-range control method of multimedia classroom mobile terminal |
| CN108804067A (en) * | 2018-06-14 | 2018-11-13 | 上海掌门科技有限公司 | Method for information display, equipment and computer-readable medium |
| CN108924471A (en) * | 2018-09-13 | 2018-11-30 | 福建星网智慧科技股份有限公司 | A kind of conference system based on QSV fast video encoding and decoding |
| CN108965932A (en) * | 2017-05-17 | 2018-12-07 | 武汉斗鱼网络科技有限公司 | A kind of even wheat window methods of exhibiting and device |
| CN109474661A (en) * | 2018-09-25 | 2019-03-15 | 视联动力信息技术股份有限公司 | A kind of processing method and system of network request event |
| CN109669658A (en) * | 2018-12-29 | 2019-04-23 | 联想(北京)有限公司 | A kind of display methods, device and display system |
| CN109698929A (en) * | 2018-11-30 | 2019-04-30 | 视联动力信息技术股份有限公司 | A kind of screenshot method and device of view networking meeting |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101018316A (en) * | 2007-02-27 | 2007-08-15 | 深圳创维－Rgb电子有限公司 | Video conference system based on IPTV and its implementation method |
| CN101031065A (en) * | 2007-04-27 | 2007-09-05 | 华为技术有限公司 | Method, apparatus and system for switching pictures in video service |
| WO2008141539A1 (en) * | 2007-05-17 | 2008-11-27 | Huawei Technologies Co., Ltd. | A caption display method and a video communication system, apparatus |
-
2012
- 2012-03-21 CN CN201210075934.2A patent/CN102625079B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101018316A (en) * | 2007-02-27 | 2007-08-15 | 深圳创维－Rgb电子有限公司 | Video conference system based on IPTV and its implementation method |
| CN101031065A (en) * | 2007-04-27 | 2007-09-05 | 华为技术有限公司 | Method, apparatus and system for switching pictures in video service |
| WO2008141539A1 (en) * | 2007-05-17 | 2008-11-27 | Huawei Technologies Co., Ltd. | A caption display method and a video communication system, apparatus |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349120A (en) * | 2013-07-26 | 2015-02-11 | 北京计算机技术及应用研究所 | Audio and video decoding system and decoding method thereof |
| CN105095123A (en) * | 2014-05-19 | 2015-11-25 | 联想(北京)有限公司 | Data processing method and electronic device |
| CN108234432A (en) * | 2016-12-22 | 2018-06-29 | 展讯通信（上海）有限公司 | Media implementation method, device, mostly logical terminal and the media server of more logical terminals |
| CN108234432B (en) * | 2016-12-22 | 2021-02-26 | 展讯通信（上海）有限公司 | Media implementation method and device for multi-channel terminal, multi-channel terminal and media server |
| CN108965932A (en) * | 2017-05-17 | 2018-12-07 | 武汉斗鱼网络科技有限公司 | A kind of even wheat window methods of exhibiting and device |
| CN108419125A (en) * | 2018-03-08 | 2018-08-17 | 弘成科技发展有限公司 | The long-range control method of multimedia classroom mobile terminal |
| CN108804067A (en) * | 2018-06-14 | 2018-11-13 | 上海掌门科技有限公司 | Method for information display, equipment and computer-readable medium |
| CN108924471A (en) * | 2018-09-13 | 2018-11-30 | 福建星网智慧科技股份有限公司 | A kind of conference system based on QSV fast video encoding and decoding |
| CN109474661A (en) * | 2018-09-25 | 2019-03-15 | 视联动力信息技术股份有限公司 | A kind of processing method and system of network request event |
| CN109698929A (en) * | 2018-11-30 | 2019-04-30 | 视联动力信息技术股份有限公司 | A kind of screenshot method and device of view networking meeting |
| CN109669658A (en) * | 2018-12-29 | 2019-04-23 | 联想(北京)有限公司 | A kind of display methods, device and display system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102625079B (en) | 2015-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102625079B (en) | Video implementation method for trilateral video conference | |
| JP5330690B2 (en) | Data mixer for portable communication devices | |
| CN101778181B (en) | Method and system for mobile terminal to achieve three-part call of videophone | |
| KR100880150B1 (en) | Multi-point video conference system and media processing method thereof | |
| US20110131498A1 (en) | Presentation method and presentation system using identification label | |
| CN112543297A (en) | Video conference live broadcasting method, device and system | |
| CN101198008A (en) | Method and system for implementing multi-screen and multi-picture | |
| WO2015074445A1 (en) | Multipath wireless displaying method and device | |
| CN104980683A (en) | Implement method and device for video telephone conference | |
| CN101645952A (en) | Conference telephone terminal, system and method for sharing data | |
| CN201127081Y (en) | System for implementing multimedia contents share | |
| CN202918417U (en) | Video conversation system based on Android set top box | |
| CN105472306A (en) | Video conference data sharing method and related device | |
| CN102348097B (en) | Session method and multi-point control unit for video conference | |
| CN103856809A (en) | Method, system and terminal equipment for multipoint at the same screen | |
| CN101931783A (en) | Double-flow transmitting system and method for video session | |
| CN100561963C (en) | A kind of system that realizes that content of multimedia is shared | |
| CN102118602A (en) | Method and system for displaying auxiliary streaming video in multiple pictures | |
| CN100454821C (en) | Method for resource sharing among MCUs in videoconference system | |
| CN201813477U (en) | Handheld terminal capable of realizing three dimensional (3D) video call by using two cameras | |
| CN103957391A (en) | Method and system for displaying videos of all parties at same time during multi-party call in video intercom | |
| CN103139202A (en) | Thin client, communication method and device thereof | |
| WO2014177082A1 (en) | Video conference video processing method and terminal | |
| CN102438119B (en) | Audio/video communication system of digital television | |
| CN203136049U (en) | Interactive data sharing and displaying system used for remote video system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 361009, Xiamen Software Park, Fujian, two, expecting channel 63, unit 402-502 Applicant after: Xiamen Yealink Network Technology Co., Ltd. Address before: 361009, Xiamen Software Park, Fujian, two, expecting channel 63, unit 402 Applicant before: Xiamen Yilian Network Technology Co., Ltd. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |