CN108933914B - Method and system for carrying out video conference by using mobile terminal - Google Patents
Method and system for carrying out video conference by using mobile terminal Download PDFInfo
- Publication number
- CN108933914B CN108933914B CN201710375061.XA CN201710375061A CN108933914B CN 108933914 B CN108933914 B CN 108933914B CN 201710375061 A CN201710375061 A CN 201710375061A CN 108933914 B CN108933914 B CN 108933914B
- Authority
- CN
- China
- Prior art keywords
- video conference
- terminal
- streaming media
- media data
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/152—Multipoint control units therefor
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the invention provides a method and a system for carrying out a video conference by using mobile terminals, wherein the method comprises the steps of establishing communication connection between at least one mobile terminal and a video conference terminal; calling cameras of all mobile terminals to acquire video data corresponding to the cameras, acquiring audio data corresponding to the microphones, and sending acquired streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data; processing the received streaming media data in the video conference terminal according to a first preset rule, and sending the streaming media data to an MCU (micro control unit); processing the data in the MCU according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all mobile terminals connected with the other video conference terminals for playing; the conference participants can realize clear voice effect without moving positions, and can acquire close-range detail pictures of the conference room, so that the conference participants can use the pictures conveniently, and the user experience is improved.
Description
Technical Field
The invention relates to the field of video conferences, in particular to a method and a system for carrying out a video conference by using a mobile terminal.
Background
In a common video conference application scenario, a video conference terminal and its peripherals such as a video camera and a microphone are usually placed at fixed positions, for example, the video camera is placed at a certain corner of a ceiling of a conference room so as to globally observe internal pictures of the conference room, and the microphone is placed on a conference table; in such a scene, the camera is limited in vertical and horizontal rotation angle, so that certain visual blind areas may exist, and meanwhile, the camera cannot present close-range detailed pictures due to the limitation of zoom size; in addition, the position of the microphones is fixed and the number of the microphones is limited, and some conference participants may need to move the seat to a place close to the microphones or move the microphones to a place close to own seats if the conference participants want to speak in order to achieve clear voice effects. In conclusion, the video conference terminal is inconvenient to use, so that the communication effect of participants is influenced, and the usability of the video conference terminal as a communication tool is reduced.
Disclosure of Invention
The embodiment of the invention mainly solves the technical problem of providing a method and a system for carrying out a video conference by using a mobile terminal, and solves the problem that the peripheral position and the capability of the video conference terminal are limited in a limited conference room layout environment in the prior art.
In order to solve the above technical problem, an embodiment of the present invention provides a method for performing a video conference using a mobile terminal, including:
establishing communication connection between at least one mobile terminal and the video conference terminal;
calling cameras of all mobile terminals to acquire video data corresponding to the cameras, acquiring audio data corresponding to the microphones, and sending acquired streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data;
judging whether the current set result is to play the streaming media data of one mobile terminal or play the streaming media data of a plurality of mobile terminals;
determining a corresponding streaming media data processing rule according to a current set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal and processing according to a second preset rule in central control equipment of a video conference system;
processing the received streaming media data in the video conference terminal according to a first preset rule, and sending the processed streaming media data to a central control device of a video conference system; and processing the data in the central control equipment of the video conference system according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all mobile terminals connected with the other video conference terminals for playing.
To solve the above technical problem, an embodiment of the present invention provides a system for performing a video conference using a mobile terminal, including:
the system comprises at least one mobile terminal, at least two video conference terminals and video conference system central control equipment, wherein the at least one mobile terminal is in communication connection with the video conference terminal, and the video conference terminals are in communication connection through the video conference system central control equipment;
the mobile terminal is used for calling a camera of the mobile terminal to collect video data corresponding to the mobile terminal, a microphone to collect audio data corresponding to the mobile terminal, and the collected streaming media data are sent to the video conference terminal, wherein the streaming media data comprise the video data and the audio data;
the video conference terminal is used for judging whether the current set result is that the streaming media data of one mobile terminal or the streaming media data of a plurality of mobile terminals are played; determining a corresponding streaming media data processing rule according to a current set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal and processing according to a second preset rule in central control equipment of a video conference system; processing the received streaming media data according to a first preset rule, and sending the streaming media data to a video conference system central control device;
the video conference system central control equipment is used for processing the streaming media data sent by the video conference terminal according to a second preset rule and sending the processed streaming media data to other video conference terminals and all mobile terminals connected with the other video conference terminals;
the other video conference terminals are used for playing the streaming media data sent from the video conference system central control equipment;
and all the mobile terminals connected with other video conference terminals are used for playing the streaming media data sent from the central control equipment of the video conference system.
In order to solve the foregoing technical problem, an embodiment of the present invention further provides a computer storage medium, where computer-executable instructions are stored in the computer storage medium, and the computer-executable instructions are used to execute the foregoing method for performing a video conference using a mobile terminal.
The invention has the beneficial effects that:
according to the method and the system for carrying out the video conference by using the mobile terminal, which are provided by the embodiment of the invention, the communication connection between at least one mobile terminal and the video conference terminal is established; calling cameras of all mobile terminals to acquire video data corresponding to the cameras, acquiring audio data corresponding to the microphones, and sending acquired streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data; judging whether the current set result is to play the streaming media data of one mobile terminal or play the streaming media data of a plurality of mobile terminals; determining a corresponding streaming media data processing rule according to a current set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal and processing according to a second preset rule in central control equipment of a video conference system; processing the received streaming media data in the video conference terminal according to a first preset rule, and sending the processed streaming media data to a central control device of a video conference system; processing the data in the central control equipment of the video conference system according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all mobile terminals connected with the other video conference terminals for playing; by adopting the scheme, the video data and the audio data of the corresponding user, which are collected by at least one mobile terminal connected with the video conference terminal, can be played on other video conference terminals and all mobile terminals connected with other video conference terminals, the participants can realize clear voice effect without moving positions, and can obtain close-range detail pictures of a conference room, the use of the participants is greatly facilitated, the communication effect of the participants is improved, the user experience is improved, and the requirements of various flexible application scenes are met.
Drawings
Fig. 1 is a flowchart of a method for performing a video conference by using a mobile terminal according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a display on a mobile terminal according to various embodiments of the present invention;
fig. 3 is a schematic diagram of a system for performing a video conference by using a mobile terminal according to a second embodiment of the present invention.
Detailed Description
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.
Example one
The embodiment provides a method for carrying out a video conference by using a mobile terminal, and the related equipment of the method comprises the following steps: the system comprises at least one mobile terminal, at least two video conference terminals and an MCU (video conference system central control device), wherein the at least one mobile terminal is in communication connection with the video conference terminal, and the video conference terminals are in communication connection through the MCU;
the mobile terminal is provided with a camera and a microphone;
the mobile terminal can be a smart phone, a tablet computer and the like;
referring to fig. 1, fig. 1 is a flowchart of a method for performing a video conference by using a mobile terminal according to this embodiment, where the method includes the following steps:
s101: establishing communication connection between at least one mobile terminal and the video conference terminal;
the step S101 of establishing a communication connection between at least one mobile terminal and the video conference terminal specifically includes:
starting an Application (Application) for establishing communication connection with the video conference terminal in the mobile terminal to search for the connectable video conference terminal;
the App is installed on the mobile terminal and runs on the mobile terminal, and a user searches for a connectable video conference terminal through the operation of the App interface of the mobile terminal;
for the signaling interaction of the search operation between the mobile terminal and the video conference terminal, a TCP (Transmission Control Protocol) Protocol can be adopted, private messages negotiated by both parties can be adopted, and the message content can imitate a GRQ message in a way that an h.323 terminal searches for a GK, and the signaling interaction is carried out in a broadcast or multicast way;
the video conference terminal replies the information of the video conference terminal, such as information of an IP address, a video conference terminal type, a video conference terminal name and the like, under the condition that the video conference terminal receives the search request;
displaying a search result list on an App of the mobile terminal, and if at least one video conference terminal is searched, sending a connection request to the video conference terminal according to the video conference terminal selected by a user from the search result list; if the connectable video conference terminal is not searched, the process is ended;
the user can select a certain searched video conference terminal and click a 'connection' button corresponding to the video conference terminal displayed on the App so as to send a connection request to the video conference terminal; the connection request also uses a TCP protocol, and carries the name, the model and the IP address of the mobile terminal;
after receiving the connection request, the video conference terminal responds to the connection request according to the current connection state and the streaming media data processing resource of the video conference terminal;
specifically, when the number of the mobile terminals connected to the video conference terminal does not reach the set threshold and the streaming media data processing resource of the mobile terminal is sufficient, the mobile terminal is allowed to connect and a response allowing access is sent to the mobile terminal, and the set threshold is set according to an actual situation, for example, may be set to 10;
on the video conference terminal, the connection state with the mobile terminal (name identification of the mobile terminal) can be displayed, and the user can also check the detailed information of the mobile terminal, including the information of the name, the model, the IP address and the like of the mobile terminal;
after receiving the response allowing access, the App of the mobile terminal may display information of the video conference terminal connected to the mobile terminal, such as information of an IP address, a type of the video conference terminal, a name of the video conference terminal, and the like;
and under the condition that the number of the mobile terminals connected with the video conference terminal reaches a set threshold value or the processing resources of the streaming media data of the mobile terminals are insufficient, rejecting the connection request and sending a rejection response to the mobile terminals, wherein after receiving the rejection response, the mobile terminals end the process.
When a plurality of mobile terminals need to establish communication connection with the video conference terminal, the plurality of mobile terminals may send connection requests to the video conference terminal one by one or simultaneously, and the video conference terminal responds to the connection requests to the plurality of mobile terminals one by one or simultaneously.
S102: calling cameras of all mobile terminals to acquire video data corresponding to the cameras, acquiring audio data corresponding to the microphones, and sending acquired streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data;
after receiving the response allowing access, the App of the mobile terminal may call cameras and microphones of different operating systems in different mobile terminals, such as an Android system (Android), an IOS (mobile operating system of apple inc), and the like, display a local camera picture to the user, and acquire video data and audio data of the current user;
calling cameras and microphone acquisition data of all mobile terminals in communication connection with the video conference terminal; for example, a user a, a user B, a user C and a user D participating in the conference, where the user a corresponds to a mobile terminal a, the user B corresponds to a mobile terminal B, the user C corresponds to a mobile terminal C and the user D corresponds to a mobile terminal D, a camera of the mobile terminal a is called to acquire video data of the user a, and a microphone of the mobile terminal a is called to acquire audio data of the user a; calling a camera of the mobile terminal B to acquire video data of the user B, and calling a microphone of the mobile terminal B to acquire audio data of the user B; calling a camera of the mobile terminal C to acquire video data of the user C, and calling a microphone of the mobile terminal C to acquire audio data of the user C; calling a camera of the mobile terminal D to acquire video data of the user D, and calling a microphone of the mobile terminal D to acquire audio data of the user D;
after the video data and the audio data are collected, the video data and the audio data are transmitted to a video conference terminal side through a wireless network, and the transmission of the video data and the audio data can adopt a User Datagram Protocol (UDP) Protocol to ensure real-time performance.
S103: judging whether the current set result is to play the streaming media data of one mobile terminal or play the streaming media data of a plurality of mobile terminals;
the setting result can adopt default setting, under the default condition, the streaming media data of the first mobile terminal connected with the video conference terminal is configured to be played, and the setting can also be carried out by a user;
the user can set the operation interface of the video conference terminal;
when a plurality of mobile terminals establish communication connection with the video conference terminal, a user can modify the setting, that is, can select whether to play streaming media data of one mobile terminal (any connected mobile terminal) or play streaming media data of a plurality of mobile terminals.
S104: determining a corresponding streaming media data processing rule according to a current set result, wherein the streaming media data processing rule comprises processing in the video conference terminal according to a first preset rule and processing in an MCU according to a second preset rule;
s105: processing the received streaming media data in the video conference terminal according to a first preset rule, and sending the streaming media data to an MCU (micro control unit); and processing the data in the MCU according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all mobile terminals connected with the other video conference terminals for playing.
When the current setting result is that streaming media data of one mobile terminal is played, S105 specifically includes:
the video conference terminal extracts the streaming media data of one mobile terminal, re-decodes and re-encodes the streaming media data and then sends the streaming media data to the MCU, and the MCU performs encoding and decoding processing on the streaming media data and then sends the streaming media data to other video conference terminals and all the mobile terminals connected with other video conference terminals for playing;
when the current setting result is that streaming media data of a plurality of mobile terminals are played, S105 specifically includes:
and performing picture synthesis processing on all video data according to a preset video data processing sub-rule, performing sound mixing processing on all audio data according to a preset audio data processing sub-rule, sending the processed video data and audio data to an MCU (microprogrammed control unit), performing coding and decoding processing on streaming media data by the MCU, and then sending the streaming media data to other video conference terminals and all mobile terminals connected with other video conference terminals for playing.
The picture synthesis processing of all the video data according to the preset video data processing sub-rule and the sound mixing processing of all the audio data according to the preset audio data processing sub-rule comprise:
decoding the video data of each mobile terminal, then coding the video data into a uniform media format, then carrying out multi-picture synthesis, and taking the multi-picture synthesis as the video data of the video conference terminal;
removing external noise and noise from the audio data of each mobile terminal, filtering effective sound, performing audio mixing operation on audio data with a preset number of sound volume decibels and with a top ranking, and taking the audio data as the audio data of the video conference terminal; for example, three paths of audio data with the largest volume are selected for mixing operation.
Optionally, the method for performing a video conference by using a mobile terminal further includes: the video conference terminal and all the mobile terminals connected with the video conference terminal receive streaming media data from other video conference terminals;
and playing the streaming media data sent by other video conference terminals on the video conference terminal and all the mobile terminals connected with the video conference terminal.
The scheme of the present embodiment is further described below by taking the example of collecting video data.
For example, a user A and a user B participate in the video conference at the terminal side of the video conference, the user A corresponds to a mobile terminal a, the user B corresponds to a mobile terminal B, the mobile terminal a and the mobile terminal B are both in communication connection with the video conference terminal, and a camera of the mobile terminal a is called to collect video data of the user A; calling a camera of the mobile terminal B to acquire video data of the user B;
after the video data are collected, the video data are transmitted to the video conference terminal side through a wireless network, and the current setting result is that the streaming media data of a plurality of mobile terminals are played, the video data of the user A and the video data of the user B are decoded, then the video data and the video data are coded into a uniform media format, and then the two pictures are synthesized and used as the video data of the video conference terminal; sending the processed video data to an MCU (microprogrammed control Unit), and sending the video data to other video conference terminals and all mobile terminals connected with other video conference terminals for playing after the MCU carries out coding and decoding processing on the streaming media data; referring to fig. 2, fig. 2 is a schematic diagram of a display on a mobile terminal according to this embodiment, in fig. 2, an upper half is video data of a user a, a lower half is video data of a user B, and fig. 2 shows a display screen at a certain time point and on one of the mobile terminals connected to other video conference terminals.
The scheme of the embodiment can be applied to single-point and multipoint video conferences.
When the mobile terminal needs to be disconnected with the video conference terminal, the mobile terminal sends a disconnection request to the video conference terminal; the video conference terminal receives the disconnection request; the video conference terminal responds to the disconnection request; the mobile terminal updates the interface and displays the disconnection state of the video conference terminal; the mobile terminal closes the camera and the microphone and stops sending the streaming media data to the video conference terminal; the video conference terminal updates the interface and displays the disconnection state of the video conference terminal and the mobile terminal.
Through the implementation of this embodiment, can be with the video data and the audio data that correspond the user that at least one mobile terminal that is connected with this video conference terminal gathered, play on other video conference terminals, and all mobile terminals of being connected with other video conference terminals, the meeting participant need not move the position alright realize clear voice effect, and can obtain meeting room detail picture closely, the use of meeting participant has greatly been made things convenient for, meeting participant's communication effect has been improved, user experience has been promoted, the demand of various nimble application scenes has been satisfied.
The video conference terminal of the embodiment can utilize the camera and the microphone of the mobile terminal of the conference room participant, so that the communication effect is improved; on one hand, the camera position of the mobile terminal is completely controlled by the user, and the mobile terminal can shoot detail images of any corner of a meeting room at a short distance or a long distance, even text materials or articles placed on a meeting table; on the other hand, the speaker can freely speak without taking a move to the position of the microphone on the conference table, and the microphone of the mobile terminal is used as a voice collecting device, so that the tone quality effect is better. Meanwhile, the existing peripheral devices of the mobile terminal are fully utilized, and the peripheral cost can be saved by the video conference terminal. In addition, the scheme of the embodiment also supports the access of a plurality of mobile terminals, and the conference capacity is expanded.
Example two
Fig. 3 is a schematic view of a system for performing a video conference by using a mobile terminal according to this embodiment, where fig. 3 is a schematic view of a system for performing a video conference by using a mobile terminal according to this embodiment, and the system for performing a video conference by using a mobile terminal includes:
the system comprises at least one mobile terminal 301, at least two video conference terminals 302 and an MCU303, wherein the at least one mobile terminal 301 is in communication connection with the video conference terminal 302, and the video conference terminals 302 are in communication connection through the MCU 303;
the mobile terminal 301 has a camera and a microphone;
the mobile terminal 301 may be a smart phone, a tablet computer, or the like;
at least one mobile terminal 301, configured to start an App search connectable video conference terminal 302 in the mobile terminal 301, where the App search is used to establish a communication connection with the video conference terminal 302;
the App is installed on the mobile terminal 301 and runs on the mobile terminal 301, and a user operates the App interface of the mobile terminal 301 to search for a connectable video conference terminal 302;
for the signaling interaction of the search operation between the mobile terminal 301 and the video conference terminal 302, a TCP protocol may be adopted, private messages negotiated by both parties may be adopted, and the message content may imitate a GRQ message in a way that an h.323 terminal searches for a GK, and is performed in a broadcast or multicast way;
the video conference terminal 302 is configured to reply information of the video conference terminal 302, such as information of an IP address, a type of the video conference terminal 302, a name of the video conference terminal 302, and the like, when receiving the search request;
displaying a search result list on the App of the mobile terminal 301, and if at least one video conference terminal 302 is searched, sending a connection request to the video conference terminal 302 according to the video conference terminal 302 selected by the user from the search result list; if the connectable video conference terminal 302 is not searched, the process is ended;
the user can select a certain searched video conference terminal 302, and click a "connection" button corresponding to the video conference terminal 302 displayed on App to send a connection request to the video conference terminal 302; the connection request also uses a TCP protocol, and carries the name, the model and the IP address of the mobile terminal 301;
after receiving the connection request, the video conference terminal 302 responds to the connection request according to the current connection state and the streaming media data processing resource of the video conference terminal;
specifically, when the number of mobile terminals 301 connected to the video conference terminal 302 does not reach the set threshold and the streaming media data processing resource of the mobile terminal is sufficient, the mobile terminal 301 is allowed to connect and a response of allowing access is sent to the mobile terminal 301, and the set threshold is set according to an actual situation, for example, may be set to 10;
on the video conference terminal 302, the connection state with the mobile terminal 301 (name identifier of the mobile terminal 301) can also be displayed, and the user can also view the detailed information of the mobile terminal 301, including information such as the name, model and IP address of the mobile terminal 301;
after receiving the response allowing access, the App of the mobile terminal 301 may display information of the video conference terminal 302 connected to the mobile terminal 301, such as information of an IP address, a type of the video conference terminal 302, a name of the video conference terminal 302, and the like;
when the number of mobile terminals 301 connected to the video conference terminal 302 reaches a set threshold or the streaming media data processing resource of the mobile terminal is insufficient, the connection request is rejected, and a rejection response is sent to the mobile terminal 301, and after receiving the rejection response, the mobile terminal 301 ends the flow.
When there are a plurality of mobile terminals 301 that need to establish a communication connection with the video conference terminal 302, the plurality of mobile terminals 301 may send connection requests to the video conference terminal 302 one by one or simultaneously, and the video conference terminal 302 responds to the plurality of mobile terminals 301 one by one or simultaneously.
At least one mobile terminal 301, configured to invoke a camera of the mobile terminal 301 to collect video data corresponding to the mobile terminal, and a microphone to collect audio data corresponding to the mobile terminal, and send collected streaming media data to the video conference terminal 302, where the streaming media data includes video data and audio data;
after receiving the response allowing access, the App of the mobile terminal 301 may also call cameras and microphones of different operating systems in different mobile terminals 301, such as an Android system or an IOS system, to display a local camera picture to the user, and acquire video data and audio data of the current user;
calling cameras and microphone acquisition data of all mobile terminals 301 in communication connection with the video conference terminal 302; for example, a user a, a user B, a user C and a user D participating in the conference, where the user a corresponds to a mobile terminal a, the user B corresponds to a mobile terminal B, the user C corresponds to a mobile terminal C and the user D corresponds to a mobile terminal D, a camera of the mobile terminal a is called to acquire video data of the user a, and a microphone of the mobile terminal a is called to acquire audio data of the user a; calling a camera of the mobile terminal B to acquire video data of the user B, and calling a microphone of the mobile terminal B to acquire audio data of the user B; calling a camera of the mobile terminal C to acquire video data of the user C, and calling a microphone of the mobile terminal C to acquire audio data of the user C; calling a camera of the mobile terminal D to acquire video data of the user D, and calling a microphone of the mobile terminal D to acquire audio data of the user D;
after the video data and the audio data are collected, the video data and the audio data are transmitted to the video conference terminal 302 side through a wireless network, and the transmission of the video data and the audio data can adopt a UDP protocol to ensure real-time performance.
The video conference terminal 302 is configured to determine whether a currently set result is to play streaming media data of one mobile terminal 301 or to play streaming media data of a plurality of mobile terminals 301; determining a corresponding streaming media data processing rule according to a currently set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal 302 and processing according to a second preset rule in the MCU 303; processing the received streaming media data according to a first preset rule, and sending the processed streaming media data to the MCU 303;
the result of the setting may adopt default setting, and under the default condition, the streaming media data of the mobile terminal 301 connected with the video conference terminal 302 is configured to be played, or the setting may be performed by the user;
the user may set the operation interface of the video conference terminal 302;
when a plurality of mobile terminals 301 establish communication connection with the video conference terminal 302, a user may modify the settings, that is, may select whether to play streaming media data of one mobile terminal 301 (any connected mobile terminal 301) or play streaming media data of a plurality of mobile terminals 301.
The MCU303 is configured to process the streaming media data sent by the video conference terminal 302 according to a second preset rule, and send the processed streaming media data to other video conference terminals 302 and all the mobile terminals 301 connected to other video conference terminals 302;
the other video conference terminals 302 are used for playing the streaming media data sent from the MCU 303;
all mobile terminals 301 connected to other video conference terminals 302 are used for playing the streaming media data sent from the MCU 303.
The video conference terminal 302 is configured to, when it is determined that a currently set result is that streaming media data of one mobile terminal 301 is played, extract the streaming media data of one of the mobile terminals 301, re-decode and re-encode the streaming media data, and send the re-decoded streaming media data to the MCU 303; the MCU303 is configured to send the streaming media data sent by the video conference terminal 302 to other video conference terminals 302 and all the mobile terminals 301 connected to other video conference terminals 302 after performing encoding and decoding processing on the streaming media data;
when the video conference terminal 302 is used for judging that the current setting result is that streaming media data of a plurality of mobile terminals 301 are played, the video conference terminal 302 is used for performing picture synthesis processing on all video data according to a preset video data processing sub-rule, performing sound mixing processing on all audio data according to a preset audio data processing sub-rule, and sending the processed video data and audio data to the MCU 303; the MCU303 is configured to perform encoding and decoding processing on the video data and the audio data sent by the video conference terminal 302, and send the video data and the audio data to other video conference terminals 302 and all mobile terminals 301 connected to other video conference terminals 302.
The video conference terminal 302 performs picture synthesis processing on all video data according to the preset video data processing sub-rule, and performs audio mixing processing on all audio data according to the preset audio data processing sub-rule, including: the video conference terminal 302 decodes the video data of each mobile terminal 301, then codes the video data into a uniform media format, and then performs multi-picture synthesis to obtain the video data of the video conference terminal 302;
removing external noise and noise from the audio data of each mobile terminal 301, filtering out effective sound, performing audio mixing operation on audio data with a preset number of sound volume decibels and with a top ranking, and taking the audio data as the audio data of the video conference terminal 302; for example, three paths of audio data with the largest volume are selected for mixing operation.
Optionally, the video conference terminal 302 and all the mobile terminals 301 connected to the video conference terminal 302 are further configured to receive streaming media data from other video conference terminals 302 and play the streaming media data.
The scheme of the present embodiment is further described below by taking the example of collecting video data.
For example, a user a and a user B participate in the video conference at the video conference terminal 302 side, the user a corresponds to a mobile terminal a, the user B corresponds to a mobile terminal B, both the mobile terminal a and the mobile terminal B establish communication connection with the video conference terminal 302, and a camera of the mobile terminal a is called to acquire video data of the user a; calling a camera of the mobile terminal B to acquire video data of the user B;
after the video data are collected, the video data are transmitted to the video conference terminal 302 side through a wireless network, and if the current setting result is that the streaming media data of a plurality of mobile terminals are played, the video data of the user A and the video data of the user B are decoded, then the video data and the video data are coded into a uniform media format, and then the two pictures are synthesized and used as the video data of the video conference terminal 302; sending the processed video data to the MCU303, and sending the video data to other video conference terminals 302 and all mobile terminals 301 connected with other video conference terminals 302 for playing after the MCU303 performs coding and decoding processing on the streaming media data; referring to fig. 2, fig. 2 is a schematic diagram of a display on a mobile terminal according to this embodiment, in fig. 2, an upper half is video data of a user a, a lower half is video data of a user B, and fig. 2 shows a display screen at a certain time point and on one of the mobile terminals 301 connected to other video conference terminals 302.
The scheme of the embodiment can be applied to single-point and multipoint video conferences.
When the mobile terminal 301 needs to disconnect from the video conference terminal 302, the mobile terminal 301 sends a disconnection request to the video conference terminal 302; the present video conference terminal 302 receives the disconnection request; the present video conference terminal 302 responds to the disconnect request; the mobile terminal 301 updates the interface and displays the disconnection state of the video conference terminal 302; the mobile terminal 301 closes the camera and the microphone, and stops sending the streaming media data to the video conference terminal 302; the video conference terminal 302 updates the interface and displays the disconnection state with the mobile terminal 301.
Through the implementation of this embodiment, can be with the video data and the audio data of the corresponding user that at least one mobile terminal 301 that is connected with this video conference terminal 302 was gathered, play on other video conference terminals 302 and all mobile terminals 301 that are connected with other video conference terminals 302, the meeting participant need not move the position alright realize clear voice effect, and can obtain meeting room detail picture closely, the use of meeting participant has greatly been made things convenient for, the communication effect of meeting participant has been improved, user experience has been promoted, the demand in various nimble application scenes has been satisfied.
The video conference terminal 302 of the embodiment can utilize the camera and the microphone of the mobile terminal 301 of the conference room participant, so that the communication effect is improved; on one hand, the camera position of the mobile terminal 301 is completely controlled by the user, and can shoot detail images of any corner of the conference room at a short distance or a long distance, even text materials or articles placed on the conference table; on the other hand, the speaker can freely speak without taking a move to the position of the microphone on the conference table, and the microphone of the mobile terminal 301 is used as a voice collecting device, so that the sound quality effect is good. Meanwhile, the video conference terminal 302 can save the peripheral cost by fully utilizing the peripheral devices of the existing mobile terminal 301. In addition, the scheme of the embodiment also supports the access of a plurality of mobile terminals 301, and the conference capacity is expanded.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented in a general purpose computing system, centralized on a single computing system or distributed across a network of computing systems, or alternatively, in program code executable by a computing system, such that the steps shown and described may be executed by a computing system on storage media (ROM/RAM, magnetic disk, optical disk), and in some cases, in a different order than that shown and described herein, or as separate integrated circuit modules, or as a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The foregoing is a more detailed description of embodiments of the present invention, and the present invention is not to be considered limited to such descriptions. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (8)
1. A method of conducting a video conference using a terminal, comprising:
establishing communication connection between at least one terminal and the video conference terminal;
calling cameras of all terminals to acquire video data corresponding to the cameras, acquiring audio data corresponding to the microphones, and sending acquired streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data;
judging whether the current set result is to play the streaming media data of one terminal or play the streaming media data of a plurality of terminals;
determining a corresponding streaming media data processing rule according to the current set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal and processing according to a second preset rule in central control equipment of a video conference system;
processing the received streaming media data in the video conference terminal according to a first preset rule, and sending the processed streaming media data to the video conference system central control equipment; processing the data in the video conference system central control equipment according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all terminals connected with the other video conference terminals for playing;
when the current setting result is that the streaming media data of a plurality of terminals are played, the received streaming media data are processed in the video conference terminal according to a first preset rule and sent to the central control equipment of the video conference system; processing in the video conference system central control device according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all terminals connected with the other video conference terminals for playing, wherein the playing comprises:
performing picture synthesis processing on all video data according to a preset video data processing sub-rule, performing sound mixing processing on all audio data according to a preset audio data processing sub-rule, sending the processed video data and audio data to the video conference system central control equipment, and sending the video conference system central control equipment to other video conference terminals and all terminals connected with the other video conference terminals for playing after performing coding and decoding processing on streaming media data;
the picture synthesis processing of all the video data according to the preset video data processing sub-rule and the audio mixing processing of all the audio data according to the preset audio data processing sub-rule comprises the following steps:
decoding the video data of each terminal, then coding the video data into a uniform media format, then carrying out multi-picture synthesis, and taking the multi-picture synthesis as the video data of the video conference terminal;
and removing external noise and noise from the audio data of each terminal, filtering out effective sound, carrying out sound mixing operation on the audio data with the preset number of sound volume decibels and the ranking of the sound volume decibels before the sound volume decibels are used as the audio data of the video conference terminal.
2. The method according to claim 1, wherein when the current setting result is that streaming media data of a terminal is played, the received streaming media data is processed in the video conference terminal according to a first preset rule and sent to the central control device of the video conference system; processing in the video conference system central control device according to a second preset rule, and sending the processed streaming media data to other video conference terminals and all terminals connected with the other video conference terminals for playing, wherein the playing comprises:
and extracting the streaming media data of one terminal, re-decoding and re-encoding the streaming media data, sending the re-decoded streaming media data to the video conference system central control equipment, and sending the re-decoded streaming media data to other video conference terminals and all terminals connected with the other video conference terminals for playing after the video conference system central control equipment performs coding and decoding processing on the streaming media data.
3. The method for conducting a video conference using a terminal according to claim 1 or 2, further comprising: the video conference terminal and all terminals connected with the video conference terminal receive the streaming media data from other video conference terminals;
and playing the streaming media data sent by other video conference terminals on the video conference terminal and all terminals connected with the video conference terminal.
4. The method for video conferencing using terminals according to claim 1 or 2, wherein the establishing of the communication connection between at least one terminal and the terminal of the video conference comprises:
an application program used for establishing communication connection with the video conference terminal in the starting terminal searches for a connectable video conference terminal;
the video conference terminal replies the information of the video conference terminal under the condition of receiving the search request;
the application program displays a search result list, and if at least one video conference terminal is searched, a connection request is sent to the video conference terminal according to the video conference terminal selected by a user from the search result list;
after the video conference terminal receives the connection request, the video conference terminal is allowed to connect under the condition that the number of the terminals connected with the video conference terminal does not reach the set threshold value and the processing resources of the streaming media data of the video conference terminal are enough.
5. A system for conducting a video conference using terminals, comprising: the system comprises at least one terminal, at least two video conference terminals and video conference system central control equipment, wherein the at least one terminal is in communication connection with the video conference terminal, and the video conference terminals are in communication connection through the video conference system central control equipment;
the at least one terminal is used for calling a camera of the terminal to collect video data corresponding to the terminal, a microphone to collect audio data corresponding to the terminal, and sending collected streaming media data to the video conference terminal, wherein the streaming media data comprise the video data and the audio data;
the video conference terminal is used for judging whether the current set result is that the streaming media data of one terminal is played or the streaming media data of a plurality of terminals is played; determining a corresponding streaming media data processing rule according to the current set result, wherein the streaming media data processing rule comprises processing according to a first preset rule in the video conference terminal and processing according to a second preset rule in central control equipment of a video conference system; processing the received streaming media data according to a first preset rule, and sending the streaming media data to the central control equipment of the video conference system;
the video conference system central control equipment is used for processing the streaming media data sent by the video conference terminal according to a second preset rule and sending the processed streaming media data to other video conference terminals and all terminals connected with the other video conference terminals;
the other video conference terminals are used for playing the streaming media data sent from the video conference system central control equipment;
all the terminals connected with the other video conference terminals are used for playing the streaming media data sent from the video conference system central control equipment;
the video conference terminal is used for carrying out picture synthesis processing on all video data according to a preset video data processing sub-rule, carrying out sound mixing processing on all audio data according to a preset audio data processing sub-rule and sending the processed video data and audio data to the central control equipment of the video conference system when judging that the current set result is that the streaming media data of a plurality of terminals are played; the video conference system central control equipment is used for coding and decoding the video data and the audio data sent by the video conference terminal and sending the video data and the audio data to other video conference terminals and all terminals connected with the other video conference terminals;
the video conference terminal is used for decoding the video data of each terminal, then coding the video data into a uniform media format, then carrying out multi-picture synthesis, and taking the multi-picture synthesis as the video data of the video conference terminal;
and removing external noise and noise from the audio data of each terminal, filtering out effective sound, carrying out sound mixing operation on the audio data with the preset number of sound volume decibels and the ranking of the sound volume decibels before the sound volume decibels are used as the audio data of the video conference terminal.
6. The system of claim 5, wherein the video conference terminal is further configured to extract streaming media data of one of the terminals when it is determined that the current setting result is that streaming media data of one of the terminals is played, re-decode and re-encode the streaming media data, and send the re-decoded streaming media data to the central control device of the video conference system; and the video conference system central control equipment is used for coding and decoding the streaming media data sent by the video conference terminal, and then sending the coded and decoded streaming media data to other video conference terminals and all terminals connected with the other video conference terminals.
7. The system according to claim 5 or 6, wherein the terminal of the video conference and all terminals connected to the terminal of the video conference are further configured to receive streaming media data from the other video conference terminals and play the streaming media data.
8. The system for conducting video conference using terminal according to claim 5 or 6, wherein said at least one terminal for starting an application program for establishing communication connection with said own video conference terminal in a terminal searches connectable video conference terminals; the application program displays a search result list, and if at least one video conference terminal is searched, a connection request is sent to the video conference terminal according to the video conference terminal selected by a user from the search result list;
the video conference terminal is used for replying the information of the video conference terminal under the condition of receiving the search request; after the video conference terminal receives the connection request, the video conference terminal allows the connection of at least one terminal under the condition that the number of the terminals connected to the video conference terminal does not reach a set threshold value and the processing resources of the streaming media data of the video conference terminal are enough.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710375061.XA CN108933914B (en) | 2017-05-24 | 2017-05-24 | Method and system for carrying out video conference by using mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710375061.XA CN108933914B (en) | 2017-05-24 | 2017-05-24 | Method and system for carrying out video conference by using mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108933914A CN108933914A (en) | 2018-12-04 |
| CN108933914B true CN108933914B (en) | 2021-09-28 |
Family
ID=64450597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710375061.XA Active CN108933914B (en) | 2017-05-24 | 2017-05-24 | Method and system for carrying out video conference by using mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108933914B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109756695A (en) * | 2018-12-25 | 2019-05-14 | 视联动力信息技术股份有限公司 | A kind of method and apparatus that TV news stream is shown |
| CN112511782B (en) * | 2019-09-16 | 2024-05-07 | 中兴通讯股份有限公司 | Video conference method, first terminal, MCU, system and storage medium |
| CN110944144B (en) * | 2019-11-27 | 2021-09-14 | 深圳明心科技有限公司 | Method and system for quickly configuring video terminal to access video system |
| CN114697445B (en) * | 2020-12-31 | 2023-09-01 | 华为技术有限公司 | Volume adjusting method, terminal and readable storage medium |
| CN113542660A (en) * | 2021-07-20 | 2021-10-22 | 随锐科技集团股份有限公司 | Method, system and storage medium for realizing conference multi-picture high-definition display |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007174281A (en) * | 2005-12-22 | 2007-07-05 | Kyocera Corp | Tv-telephone system, communication terminal, and relaying apparatus |
| CN104135641A (en) * | 2014-07-15 | 2014-11-05 | 北京小鱼儿科技有限公司 | Tool of mobile terminal and server for intelligently integrating audio and video |
| CN105100676A (en) * | 2014-05-19 | 2015-11-25 | 中兴通讯股份有限公司 | Video conference terminal and working method thereof, and data transmission method and system |
| CN105472309A (en) * | 2015-12-31 | 2016-04-06 | 杭州华为数字技术有限公司 | Data transmission method, device and system |
| CN105959614A (en) * | 2016-06-21 | 2016-09-21 | 维沃移动通信有限公司 | Method and system for processing video conference |
-
2017
- 2017-05-24 CN CN201710375061.XA patent/CN108933914B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007174281A (en) * | 2005-12-22 | 2007-07-05 | Kyocera Corp | Tv-telephone system, communication terminal, and relaying apparatus |
| CN105100676A (en) * | 2014-05-19 | 2015-11-25 | 中兴通讯股份有限公司 | Video conference terminal and working method thereof, and data transmission method and system |
| CN104135641A (en) * | 2014-07-15 | 2014-11-05 | 北京小鱼儿科技有限公司 | Tool of mobile terminal and server for intelligently integrating audio and video |
| CN105472309A (en) * | 2015-12-31 | 2016-04-06 | 杭州华为数字技术有限公司 | Data transmission method, device and system |
| CN105959614A (en) * | 2016-06-21 | 2016-09-21 | 维沃移动通信有限公司 | Method and system for processing video conference |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108933914A (en) | 2018-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108933914B (en) | Method and system for carrying out video conference by using mobile terminal | |
| US9024997B2 (en) | Virtual presence via mobile | |
| US7508413B2 (en) | Video conference data transmission device and data transmission method adapted for small display of mobile terminals | |
| US20060001737A1 (en) | Video conference arrangement | |
| CN101645952A (en) | Conference telephone terminal, system and method for sharing data | |
| CN101478642A (en) | Multi-picture mixing method and apparatus for video meeting system | |
| WO2018006573A1 (en) | Device implementing video conference screen sharing | |
| CN104580995A (en) | Communication method and device of video conference | |
| CN103096020B (en) | video conference system, video conference device and method thereof | |
| CN103795964A (en) | Video conferencing method and device thereof | |
| CN105763832A (en) | Video interaction and control method and device | |
| KR20140098573A (en) | Apparatus and Methd for Providing Video Conference | |
| CN104184982A (en) | Audio and video communication method and system, terminal device, and audio and video communication service center | |
| KR100699253B1 (en) | Apparatus and method for posting video data and audio data to web in video telephony of mobile communication terminal | |
| WO2015003532A1 (en) | Multimedia conferencing establishment method, device and system | |
| CN104506955A (en) | Network video phone with built-in camera and connected with external HDMI (high definition multimedia interface) display device | |
| CN114531564B (en) | Processing method and electronic equipment | |
| WO2016206471A1 (en) | Multimedia service processing method, system and device | |
| US9013537B2 (en) | Method, device, and network systems for controlling multiple auxiliary streams | |
| US10244207B2 (en) | Videoconference communication device | |
| KR101918676B1 (en) | Videoconferencing Server for Providing Multi-Screen Videoconferencing by Using Plural Videoconferencing Terminals and Camera Tracking Method therefor | |
| CN112203126B (en) | Screen projection method, screen projection device and storage medium | |
| CN117176999A (en) | Multi-person wheat connecting method, device, computer equipment and storage medium | |
| KR20180100395A (en) | Multimedia information playing method and system, collecting device, standardization server | |
| JP2011029732A (en) | Video call system, calling-side terminal, receiving-side terminal, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |