KR101227625B1 - Conference terminal, conference server, conference system and data processing method - Google Patents

Abstract

A conference terminal, a conference server, a conference system, and a data processing method are provided. The conference terminal includes a mark acquisition module configured to acquire a mark data signal; An image acquisition module, configured to acquire an image data signal; An overlapping module configured to overlap the mark data signal with the image data signal to generate a hybrid image data signal; And an output module configured to output the hybrid image data signal. The present invention improves the interactivity and discussion efficiency between video conferencing users.

Description

CONFERENCE TERMINAL, CONFERENCE SERVER, CONFERENCE SYSTEM AND DATA PROCESSING METHOD}

The present invention relates to a conference terminal, a conference server, a conference system, and a data processing method.

In early video conferencing, people interacted with each other only with images and sounds. Later, the H.261 protocol enabled video conferencing participants to deliver 4CIF-sized static graphic files over a motion video channel, thereby allowing participants to share slides with each other. However, such sharing is realized only by competing for motion video channels. The resolution of the slides is not high, and the transmission of the slides used to cause motion video pauses.

Later, the H.239 protocol enabled participants to create H.239 secondary stream channels to transmit slides exclusively and to send motion video using mechanism channels. Therefore, there is no mutual impact between the transmission of motion video and the transmission of slides. A typical application scenario is shown in FIG. The conference terminal 12 simultaneously receives a conference image captured by the camera 11 and a file (for example, a PPT file or a word file) of the computer 10 and receives a video encoding for the conference image and the file (usually Based on H.263 and H.264). Then, via the channel provided by the H.239 protocol, the conference terminal 12 simultaneously sends two video streams to the Multipoint Control Unit (MCU) 13, which sends the video stream during the conference. To another terminal 14 that is present. After receiving these two video streams, the terminal 14 decodes these two video streams to obtain two different video image streams, and displays these video image streams on the projector 16 or the TV receiver 15. do. In this way, any participant can share files with all other participants, and simple data exchange is realized. However, this mode of sharing slides has a significant drawback: participants cannot interact with each other. When a speaker describes a slide, other participants can see the slide but cannot display it on the slide, which reduces discussion efficiency and interactivity.

In view of these shortcomings, another solution has been proposed in the prior art, which enables the display on the electronic board to enable interaction between participants in a video conference. As shown in Fig. 2, the copyboard server 20 is installed on the conference server side, and the copyboard client is provided for each video conference terminal, for example, the client 21, the client 22, and the client 23. , The client 24, and the client 25. These clients are generally personal computers (PCs), and some clients may be conference servers or conference clients in the conference subsystem. All clients are connected to the copyboard server 20 via a network. The copyboard server 20 discloses an copyboard, which may be a blank background. Alternatively, a picture or slide may be used as the background. The copyboard server 20 sends the content on the copyboard to all clients in the form of a picture. Therefore, each client can see the same content, and the participant can display on the copyboard through the client, for example, to draw a line or circle or enter text on the copyboard. After receiving the operation performed by the client, the copyboard server 20 updates the contents of the copyboard and sends the updated contents to all clients. In this way, all participants share the copyboard for discussion. However, this mode has a limited use category. For example, the user cannot display the slides directly, that is, the system cannot share the displayed slides. In addition, the system must create another copyboard system, and at least one computer must be configured at each meeting place, increasing system complexity and configuration cost.

In the prior art, another method, remote collaboration, is similar to the method of displaying the copyboard. As shown in FIG. 3, the system includes a remote cooperative server 30, with a computer, for example, computer 31, computer 32, computer 33, and computer 34 for each video conference location. ). These computers are connected to the remote cooperative server 30 via a network. In a meeting place where the desktop must be shared with other participants, the remote collaboration server is started and the computer desktop image (eg, PPT file or word file) of the meeting place is shared with all other participants. After connecting to the remote collaboration server 30 through a local computer, participants at other conference venues can also view and share the computer desktop provided by this remote collaboration server 30. At the same time, operations performed via the local mouse and keyboard are sent to the remote cooperative server 30. Therefore, all participants can collaborate on the same application, such as PPT or Word. However, this system also requires an extra remote cooperative system, which increases the system complexity and configuration cost since at least one computer must be configured at each meeting place. In addition, special remote collaboration parameters must be configured for each computer at each conference site, and the user must operate two systems, for example a shared remote system and a local system. Files such as shared PPT files or word files must be copied to the computer at the meeting place in advance, and the plug and play features available in notebook computers cannot be realized, which makes operation inconvenient.

Therefore, the prior art has at least the following problems:

In video conferencing systems, when participants exchange data, the interactive data provided by prior art video conferencing systems to participants is limited. For example, in a system with copyboards, the slides cannot be displayed. In addition, video conferencing systems such as electronic blackboards are complex and expensive.

Embodiments of the present invention provide a conference terminal, a conference server, a conference system, and a data processing method, which can exchange data between arbitrary conference terminals and increase the discussion efficiency of conference participants without complicating the conference system. do.

Embodiments of the present invention provide the following technical solutions:

An embodiment of the present invention provides a conference terminal, wherein the conference terminal,

A mark acquisition module configured to acquire a mark data signal;

An image acquisition module, configured to acquire an image data signal;

An overlapping module configured to overlap the mark data signal with the image data signal to generate a hybrid image data signal; And

An output module configured to output the hybrid image data signal

.

An embodiment of the present invention provides a conference system, wherein the conference system,

A first conference terminal, a conference server, and at least one second conference terminal.

The first conference terminal,

An image acquisition module configured to acquire an image data signal forwarded by the conference server; And

A mark transmission module configured to acquire a mark data signal and to transmit the mark data signal to the conference server

.

The conference server,

An image data receiving module, configured to receive the image data signal transmitted by any conference terminal in the conference system;

A mark receiving module configured to receive the mark data signal;

An overlapping module configured to overlap the mark data signal with the image data signal to generate hybrid image data; And

A transmission module configured to transmit the hybrid image data to the first conference terminal and the at least one second conference terminal.

.

The at least one second conference terminal is configured to receive and display the hybrid image data.

An embodiment of the present invention provides another conference system, wherein the other conference system,

A first conference terminal, a conference server, and at least one second conference terminal.

The first conference terminal,

An image acquisition module, configured to acquire an image data signal sent by the conference server; And

A mark transmission module configured to acquire a mark data signal and to transmit the mark data signal to the conference server

.

The conferencing server is configured to receive the image data signal and the mark data signal and to forward the image data signal and the mark data signal.

The at least one second conference terminal,

A first timer configured to generate a first time interval value;

An image acquisition module, configured to acquire the image data signal forwarded by the conference server;

A first external mark acquisition module, configured to acquire the mark data signal forwarded by the conference server, and output the mark data signal according to the first time interval value to generate a first mark data signal;

A first superposition module configured to overlap the first mark data signal with the image data signal to generate a first hybrid image data signal; And

Displaying module configured to display the first hybrid image data signal

.

An embodiment of the present invention provides a conference system, wherein the conference system,

A first conference terminal, a conference server, and at least one second conference terminal.

The first conference terminal,

A first timer configured to generate a first time interval value;

A second timer configured to generate a second time interval value;

A first internal mark acquisition module, configured to acquire a mark data signal sent by a local conference terminal, and output the mark data signal according to the first time interval value to generate a first mark data signal;

A second internal mark acquiring module, configured to acquire the mark data signal transmitted by the local conference terminal, and output the mark data signal according to the second time interval value to generate a second mark data signal;

A second superposition module configured to superimpose the mark data signal on the image data signal to generate second hybrid image data; And

A second transmission module, configured to transmit the first hybrid image data to a display screen of the local conference terminal or to transmit the second hybrid image data to the conference server through an auxiliary stream channel.

.

The conferencing server is configured to receive and forward the second hybrid image data.

The at least one second conference terminal is configured to receive and display the second hybrid image data.

An embodiment of the present invention provides a data processing method, the data processing method,

Obtaining an image data signal;

Obtaining a mark data signal;

Generating a hybrid image data signal by superimposing the mark data signal on the image data signal; And

Outputting the hybrid image data signal

It includes.

An embodiment of the present invention provides another data processing method, the other data processing method,

Receiving an image data signal and a mark data signal transmitted by the conference terminal;

Generating hybrid image data by superimposing the mark data signal on the image data signal; And

Transmitting the hybrid image data, the image data signal, or the mark data signal to the conference terminal.

It includes.

Embodiments of the present invention provide the following advantages:

In the above-described technical solution, the conference terminal superimposes the mark data signal on the image data signal to generate a hybrid image data signal (eg, the hybrid image data signal is a marked slide). In this way, the conference terminal allows the participant to add a mark to the slide while viewing the slide, improve interactivity between video conferencing users, and improve discussion efficiency without increasing the complexity of the conference system.

BRIEF DESCRIPTION OF DRAWINGS To make the technical solution under the present invention or in the prior art more clear, the following describes a detailed description of embodiments of the present invention or the accompanying drawings included in the prior art. Apparently, the accompanying drawings described below are merely illustrative and non-limiting, and those skilled in the art may derive other drawings from these accompanying drawings without any creative efforts.
1 is a schematic structural diagram of an application scenario of a conventional conference system.
2 is a schematic structural diagram of an application scenario using an electronic blackboard for interaction in a conference system of the prior art.
3 is a schematic structural diagram of an application scenario using a remote cooperative server for interaction in a prior art conference system.
4 is a schematic overall structural diagram of a conference terminal according to an embodiment of the present invention.
5 is a specific schematic structural diagram of the conference terminal shown in FIG.
FIG. 6 is a specific schematic structural diagram of a method in which the conference terminal shown in FIG. 5 receives a local mark data signal and locally overlaps the local mark data signal to generate hybrid image data.
FIG. 7 illustrates that when the conference terminal illustrated in FIG. 5 sends a local mark data signal and then records and acquires a token, and then sends a local mark data signal, the first token control module requests a token from an H.239 auxiliary stream channel. A schematic structural diagram of the method.
FIG. 8 is a specific schematic structural diagram of a method in which the conference terminal shown in FIG. 5 receives an external mark data signal and locally overlaps the external mark data signal to generate hybrid image data.
9 is a specific schematic structural diagram of the first external mark acquisition module shown in FIG. 8.
FIG. 10 illustrates that when the conference terminal illustrated in FIG. 5 performs a local overlap and then generates a hybrid image data after performing the overlap and obtaining a token, the second token control module acquires the token from the H.239 secondary stream channel. A schematic structural diagram of how to apply.
11 is another specific schematic structural diagram of the conference terminal shown in FIG.
12 is another specific schematic structural diagram of a conference terminal.
13 is a schematic diagram of a specific application scenario of a conference terminal.
14 is a schematic diagram of another particular application scenario of a conference terminal.
15 is a schematic diagram of another particular application scenario of a conference terminal.
16 is a schematic structural diagram of a conferencing server according to an embodiment of the present invention.
17 is a specific schematic structural diagram of the conference server shown in FIG.
18 is a schematic structural diagram of a conference system according to an embodiment of the present invention.
FIG. 19 is a schematic diagram of an application scenario of a plurality of MCUs connected to the conference system shown in FIG. 18.
20 is another schematic structural diagram of a conference system according to an embodiment of the present invention.
FIG. 21 is a schematic diagram of an application scenario of a plurality of MCUs connected to the conference system shown in FIG. 20.
22 is another schematic structural diagram of a conference system according to an embodiment of the present invention.
FIG. 23 is a schematic diagram of an application scenario of a plurality of MCUs connected to the conference system shown in FIG. 22.
24 is a schematic flowchart of a data processing method according to an embodiment of the present invention.
25 is another schematic flowchart of a data processing method according to an embodiment of the present invention.

BRIEF DESCRIPTION OF DRAWINGS To clarify the technical problems solved by embodiments of the present invention, the technical solutions and the advantages of the present invention, the present invention will be described in more detail with reference to the accompanying drawings and exemplary embodiments.

In the prior art, considering that the user's will cannot combine the dual-stream function of the H.239 standard with a marking function similar to that of the copyboard, and that sufficient interaction between users is not possible, An embodiment of the present invention provides a conference terminal, a conference server, a conference system, and a data processing method in the conference system.

The first of the conference terminals Example

4 is a schematic structural diagram of a conference terminal. The conference terminal 40 of the present embodiment,

A mark acquisition module 41 configured to acquire a mark data signal;

An image acquisition module 42 to acquire an image data signal;

An overlap module 43 configured to overlap the mark data signal with the image data signal to generate a hybrid image data signal;

An output module 44 configured to output the hybrid image data signal to a display screen of a conference server such as an MCU that forwards the signal to a local conference terminal or another conference terminal.

.

The conferencing terminal 40 may be a terminal at a meeting place of a video conference. It may be a conference terminal server responsible for managing signals of a plurality of input devices in a conference place. The input device may be an audio input device, a video input device, an electronic blackboard, or a recording screen. The meeting place further includes an image display device (e.g., a conventional display) that outputs a video signal, a sound box that outputs an audio signal, the image data signal may be a PPT file or a similar file, and the mark data signal may be It may be a circle or a comment. The conference terminal 40 generates a hybrid image data signal by superimposing the mark data signal on the received image data signal. For example, the hybrid image data signal is a marked slide. Without increasing the complexity of the conferencing system, the conferencing terminal can display slides while the participant is viewing the slides, thereby improving the interaction and discussion efficiency between video conferencing users.

Second of conference terminal Example

As shown in FIG. 5, based on the first embodiment of the conference terminal, the conference terminal 40 is configured to generate a first time interval 45, a second time interval value configured to generate a first time interval value. It may further include a second timer 46. In practice, the two timers can be the same timer, which outputs two time interval values.

Thus, the mark acquisition module 41 includes a first mark acquisition module 411 and a second mark acquisition module 412.

The first mark acquisition module 411 is configured to receive a mark data signal, output the mark data signal according to a first time interval value, and generate a first mark data signal. The first time interval value controls displaying the mark data signal on the display screen of the local conference terminal. The first time interval value is generally small, ie the first time interval is generally short. If the display screen 40 of the conference terminal is a TV receiver, the best value of the first time interval is equal to the frame interval of the TV receiver.

The second mark acquisition module 412 is configured to output a mark data signal, output a mark data signal according to a second time interval value, and generate a second mark data signal. Since the mark data signal must be transmitted on the H.239 auxiliary stream channel when transmitted to the teleconferencing terminal, it is preferable that the second time interval value is not too short. The second time interval value may be as large as one second so that the size of the transmitted data is reduced.

3rd conference terminal Example

As shown in FIG. 6, the first mark acquisition module 411 of the second embodiment of the conference terminal may be a first internal mark acquisition module 4110, obtains a mark data signal sent by the local conference terminal, The mark data signal is output in accordance with the one time interval value to generate the first mark data signal. For example, through the touch screen of the local conference terminal, the user adds a circle-like mark to the image data signal received by the image acquisition module 42. The image data signal may be a slide, which is a slide received from a local slide or a teleconferencing terminal. The first internal mark acquisition module 4110 generates a first mark data signal by outputting the original mark according to the first time interval value.

In addition, the second mark acquisition module 412 may be a second internal mark acquisition module 4120, receives the mark data signal transmitted by the local conference terminal, and outputs the mark data signal according to the second time interval value. To generate a second mark data signal.

The conference terminal includes a mark encoding module 47 and a mark transmitting module 48. The second mark data signal generated by the second internal mark acquisition module 4110 is transmitted to the mark encoding module 47, and the mark encoding module 47 encodes the second mark data signal to generate the second mark data signal. Create The third mark data signal is transmitted to the mark transmission module 48. The mark transmitting module 48 transmits a third mark data signal to the conference server in the conference system via the H.239 auxiliary screen channel, and the third mark data signal is forwarded to another conference terminal by the conference server.

4th conference terminal Example

As shown in FIG. 7, in a previous conference terminal, data is transmitted over an H.239 secondary stream channel, or a user-defined channel between the conference terminal and a conference server or other conference terminal, and the mainstream of the H.239 protocol. The channel is designed to transmit motion video. In this way, there is no conflict between the transmission of static image data and the transmission of motion video. However, when transmitting a hybrid image data or mark data signal generated by a local conference terminal using the H.239 secondary stream channel of the H.239 protocol, the conference terminals in the conference system are based on mutual exclusion of tokens. Prevent transmission collisions. First, a determination is made to check whether the H.239 auxiliary stream channel supports the mark auxiliary stream. If the H.239 auxiliary stream channel supports the mark auxiliary stream, the conferencing terminal 40

A first token control module 49 configured to apply for a venue token from the H.239 auxiliary stream channel and generate a first control signal after successfully obtaining the venue token;

As shown in FIG.

The mark transmitting module 48 transmits the third mark data signal to the conference server through the H.239 auxiliary stream channel according to the first control signal.

That is, before transmitting the mark data signal, the current conference terminal T1 applies the token. In this way, the mark data signal of T1 is allowed to be displayed locally and transmitted to the teleconferencing server, which is then forwarded by the conference server to another conference terminal. If another conference terminal T2 in the system wants to transmit its own mark data signal, then T2 must first apply the token and the current token holder changes from conference terminal T1 to conference terminal T2, thereby transmitting different conference terminals. The collision of mark data signals is prevented.

5th conference terminal Example

As shown in Figure 8, in all previous embodiments of the conference terminal, the conference terminal must provide a receiving function. Specifically, the first mark acquisition module 411,

Receives the mark data signal transmitted by at least one conference terminal except the local conference terminal and forwarded by the conference server, and outputs the mark data signal according to the first time interval value generated by the first timer 45 A first external mark acquisition module 4111 is configured to generate a first mark data signal.

The superimposition module 43 is a first superimposition module 430, and is configured to superimpose the first mark data signal on the image data signal to generate a first hybrid image data signal.

The output module 44 is the first transmission module 440, and is configured to send the first hybrid image data to the display screen of the local conference terminal so that the first hybrid image data is displayed on the display screen.

6th conference terminal Example

As shown in Fig. 9, conference terminals at multiple conference venues must make marks simultaneously during the discussion process. Mutual exclusion should be performed to prevent disorder of the mark data signals at the various meeting places. Based on the previous fifth embodiment of the conference terminal, when the first mark acquisition module 411 receives the mark data signals sent by the plurality of conference terminals, the first external mark acquisition module 4111 may:

A first external mark acquisition submodule 41110, configured to receive a mark data signal transmitted by at least one conference terminal except the local conference terminal and forwarded by the conference server;

At least one decoder corresponding to each meeting place, for example, decoder 1, decoder 2, and decoder 3, wherein the mark data signal transmitted by the meeting place (ie, another conference terminal) is received and decoded by the decoder, respectively, The at least one decoder for decoding the received mark data signal and generating a mark data signal queue; And

A mark output module 41111 configured to output a mark data signal in accordance with a first time interval value and an order of the mark data signal queue, and to generate at least one first mark data signal

.

Then, the first mark data signals are sequentially transmitted to and overlapped with the first overlapping module 430. In order to enable the receiver to reliably identify the mark data signal transmitted by each conference terminal in the overlapping, the mark data signal transmitted by each conference terminal is assigned a conference number, a terminal number, and the correspondence of the conference terminal. It should include the mark data signal content. The content of the mark data signal can be represented by a color bitmap such as a vector graph, a binary bitmap, a YUV monochrome bitmap, a red cyan (RGB) monochrome bitmap. When the content of the mark data signal is represented by a vector bitmap or a binary bitmap, the conferencing server (MCU) may randomly assign a filling color to each conference terminal. The overlapping module of the conference terminal uses the color assigned by the conference server to populate when overlapping tracks of several conference locations. When representing a track as a monochrome bitmap, such as a YUV or RGB bitmap, a transparent color must be consistently assigned to the system so that the receiver identifies the position of the track and superimposes the position on the slide, in which the mark data signal After is superimposed on the image data signal, the integrity of the hybrid image data of each meeting place is assured.

When the conference terminal receives the image data signal simultaneously with receiving the mark data signal transmitted by another conference terminal, the conference terminal overlaps the mark data signal with the received image data signal to form complete hybrid image data. In this way, the conference terminal knows the mark made by the remote party on the current image data.

7th conference terminal Example

As shown in Fig. 10, when the conferencing system does not support the mark auxiliary stream, the conferencing terminal can only send hybrid image data to other terminals. Hybrid image data is generated through superposition on the local conference terminal. In order to correctly order the hybrid image data sent by the various terminals, the conferencing terminal 40 applies for a meeting place token from another H.239 secondary stream channel of the H.239 protocol and successfully obtains the meeting place token. And further comprising a second token control module 50 configured to generate a second control signal. After acquiring the second control signal, the superimposition module 43 may begin recording the mark data signal and the image data signal, and generate the hybrid image data signal by superimposing the mark data signal on the image data signal.

That is, before adding a mark to the received slide, conference terminal T1 applies the control token of the H.239 secondary stream channel. After obtaining this control token, conference terminal T1 starts recording of the mark data signal added by the user, and superimposes this mark data signal on the slide to generate hybrid image data. The conference terminal T1 sends the encoded hybrid image data to the conference server via the H.239 auxiliary stream channel, which forwards the hybrid image data to another conference terminal. If a conference terminal in another conference wants to add a mark data signal, the conference terminal may take the same action. In this method, by continuously switching the transmission token of the H.239 auxiliary stream channel, the conference terminal of each conference place can add the mark to the slide.

Therefore, the superimposition module 43 is specifically the second superimposition module 43, and superimposes the first mark data signal on the image data signal according to the second control signal generated by the second token control module 50. And generate second hybrid image data, such as a marked slide.

The output module 44 is specifically a second transmission module 441, which sends the second hybrid image data to the display screen of the local conference terminal or sends the second hybrid image data to the conference server via the H.239 secondary stream channel. It is configured to send. In this method, the second hybrid image data can be displayed on the local conference terminal. After adding the mark to the touch screen, the user can see the track of the mark he drew in time. Alternatively, the second hybrid image data may be transmitted to the conference server via the H.239 secondary stream channel and forwarded by the conference server to another conference terminal. In this way, the other conference terminal can view the slide currently displayed by the user.

8th conference terminal Example

As shown in FIG. 11, the conference terminal 40 illustrated in the previous embodiment,

An image data storage module 51, configured to store hybrid image data in a local conference terminal, wherein each marked slide is stored in the local conference terminal so that the conference progress can be recorded while the conference is in progress;

Configured to delete the mark data signal in the hybrid image, wherein during addition of the mark data signal, the user can delete the added mark data signal, and the token holder can delete the added mark in the token control mode. A mark deletion module 52 capable of sending a command to command 43; And

Configured to transmit the image data signal received by the internal image acquisition module 420 from the local conference terminal to the conference server, wherein the image data signal is forwarded by the conference server to the other conference terminals so that the other conference terminals can receive the local conference. Image transmission module 53, which can view image data of the terminal

As shown in FIG.

In addition, the image acquisition module 42 may include an internal image acquisition module 420 configured to acquire an image data signal transmitted by a local conference terminal; And an internal image acquisition module 421 configured to acquire an image data signal transmitted by other conference terminals and forwarded by the conference server.

In all previous embodiments, the image data signal received by the image acquisition module 42 may be a slide or copyboard. The copyboard is generated by the input device of the conference terminal. For example, after a user creates a mark through the touch screen, the system automatically creates the copyboard. All of the mark data signals of the user are superimposed on the copyboard so that the user can easily and conveniently create a mark.

9th of the conference terminal Example

As shown in FIG. 12, in all previous embodiments, the conference location of the conference terminal 40 may include a plurality of input devices. The input device may be an audio input device, a video input device, an electronic blackboard, or a recording screen. Such an input device may be directly or indirectly connected to the conference terminal. To facilitate switching between multiple input devices at the conference location of the conference terminal 40, the conference terminal 40 can control a plurality of input devices at the conference location through mutual exclusion of tokens. Therefore, the conference terminal,

A token management module (54) configured to receive a subtoken application of an input device at a meeting location where a local conference terminal is located, and to assign a subtoken to the input device after confirming that the application of the subtoken was successful

As shown in FIG.

Therefore, the first internal mark acquisition module 4110 acquires the mark data signal of the input device at the conference location where the local terminal is located, outputs the mark data signal according to the first time interval value, and generates the first mark data signal. do.

The second internal mark acquisition module 4112 may acquire a mark data signal of an input device at a conference location where the local conference terminal is located, and output a mark data signal according to a second time interval value to generate a second mark data signal. do.

That is, regardless of which of the input devices connected to the conference terminal 40 at the conference place obtains the token, the conference terminal receives the mark data signal of the input device obtaining the token. Nevertheless, in the video / audio signal, the conference terminal can control the signal through mutual exclusion of tokens.

In addition, when the input device at the meeting place of the input terminal 40 needs to conduct a private session with the target input device, the conference terminal 40 may further include a secret token control module 55. The secret token control module 55 may receive a secret token request sent by an input device at a meeting place where a conference terminal is located to conduct a secret session with a target input device, and apply the secret token. After confirming that this is successful, it is configured to assign a secret token to the input device using an H.239 auxiliary stream channel or a user-defined channel. In this way, the input device may conduct a secret session with the target input device over an H.239 secondary stream channel or a user-defined channel.

The target input device may here be an input device at a different meeting place capable of obtaining the mark data signal in real time, rather than a meeting place with the meeting terminal 40, or on the meeting place with the meeting terminal 40. It may be an input device.

In addition, the mark acquiring module 41 of the conference terminal 40 may be a real time mark acquiring module capable of acquiring a mark data signal in real time, or if the manual receiving option is selected and the manual receiving option is selected, the mark data It may be a manual mark acquisition module for acquiring the signal.

The image acquisition module 42 in the previous conference terminal 40 may be a real-time image acquisition module capable of acquiring the image data signal in real time, or if the manual reception option is selected and the manual reception option is selected, the image data signal. The manual image acquisition module may be obtained.

10th conference terminal Example

Hereinafter, a specific application scenario of the conference location where the conference terminal 40 is located will be described with reference to FIG. 13. As shown in Fig. 13, the conference terminal 40 is connected to the MCU 130, and the video and audio signals and data signals (existing data signals and interactive copyboards) to the MCU 130 via a network on this conference location. And receive the video and audio signals and data signals of the other meeting place from the MCU 130, such that they are transmitted to the output device of the meeting place for display.

The meeting place includes at least one input device. For video and audio signals, the input device is

An audio input device 131 which is a general microphone for capturing a sound signal on a conference place and transmitting the sound signal to the conference terminal 40; And

Video input device 132, which is a general one or more cameras that captures a video signal on a conference location and transmits the video signal to conference terminal 40

It includes.

The conference terminal 40 encodes the received video and audio signals and sends these signals to the MCU 130 over the network, and then the MCU 130 sends the encoded signal on the conference location to another conference location for display. .

In the data signal:

A data server 135 connected to the conference terminal 40 is installed at the conference place, and as a sender, the user at the conference place edits the data on the copyboard 133 and the recording screen 134, eg For example, it adds a mark data signal sent to the conference terminal 40 through the data server 135, and when the signal is sent to the MCU 130 by the conference terminal 40, the MCU 130 sends this signal. It is simultaneously distributed to conference terminals 40 at different conference locations.

As a receiver, the conference terminal 40 transmits the data signal received from the MCU 130 to the data server 135, and the data server 135 transmits the data signal to the copyboard 133 and the conference location at the conference location. To one or more recording screens 134 in the.

The meeting place further includes some signal output devices. For video and audio signals, the audio output device 136 serving as a receiver is a general stereo speaker which provides a sound signal transmitted by the conference venue to a participant at the conference venue, and a video output device (not shown in the figure). ) Is one or more display screens or projectors in general that provide a video signal transmitted by conference terminal 40 to a participant at a conference venue.

In a conference involving cooperation, one participant must change from one role to another, for example from speaker to non-speaker, or from non-speaker to speaker. In a real-life meeting scenario, the speaker's role changes from one person to another only after the person in front of him completes the action. Therefore, embodiments of the present invention provide a method of operation as a preferred solution and use control tokens to switch roles:

(a) A main control token is established for the entire conference system, which includes at least two conference locations. The conference terminal 40 is set for each conference place. The meeting terminal 40 at one meeting place is connected to the meeting terminal at another meeting place through the MCU 130 and transmits a signal. The main control token is equivalent to the meeting place token. If the speaker at the conference site succeeds in applying for this conference site, it means that the conference site is now ready to speak.

When the first speaker at the current meeting place 1 has obtained the meeting place token, and if the second speaker at the meeting place 2 obtains the meeting place token and attempts to speak, the second speaker obtains the meeting place token and then the main speaker Changes to. The content spoken by the main speaker is sent to other terminals on this conference location or to conference terminals at another conference location.

Corresponding to this scenario, each user-machine interactive device (electronic blackboard or recording screen) may present a control token application. The device may actively apply for a token or actively transfer a control token to an interactive conference terminal at this conference location or other conference locations.

(b) A hierarchy of control tokens is established. That is, a main control token of the level of the conference system (the previous conference place token is equivalent) is set, and each participant at the conference place corresponds to the subtoken. This method proceeds in terms of the following application scenarios:

First, a solution for the local meeting place is created. After consensus on the meeting place, the final solution is distributed to other meeting places. There are two scenarios:

Scenario 1: The first and second speakers are at the same conference location, and the first speaker is speaking. In this scenario, after acquiring the subtoken of this meeting place, the second speaker may be turned into the main speaker. The first speaker can also actively deliver the subtoken to the second speaker at the local conference location.

Scenario 2: The first speaker and the second speaker are in different meeting places, and the first speaker is speaking. In this scenario, after presenting the application for the token, the second speaker first applies for the token of his meeting place. After the application succeeds, the second speaker applies for a subtoken within the conference venue. After acquiring the subtoken, the second speaker is turned into the main speaker. The first speaker can also actively pass the meeting place token to the meeting place of the second speaker, and the meeting place of the second speaker distributes the local subtoken to the second speaker.

There is a special application scenario: first, a solution for the local meeting place is created; After consensus on the meeting place, the final solution is distributed to other meeting places. In this scenario, the data server 135 on this meeting place does not send the incomplete mark information to the conference terminal 40 before the agreement for this meeting place is reached and the transmission function is enabled. To other participants at this meeting location. After the transmission function is enabled, the conference terminal 40 transmits the obtained mark signal to the MCU 130, and the MCU 130 transmits the mark signal to another conference location.

In this embodiment, data is transferred between the conference terminal 40 and the MCU 130 via the H.239 protocol. The H.239 protocol provides a control channel, mainstream channel and H.239 auxiliary stream channel. The mainstream channel transmits video signals between conference venues and the H.239 auxiliary stream channel transmits data signals such as PPT files, word files or marks. When transmitting data signals, the transmission method is the same as that provided in the first to tenth embodiments of the conference terminal 40, as illustrated below:

(a) Comment information is superimposed on the PPT file and encoded together, and the encoded information is transmitted on the same channel.

(b) PPT data and comment information are separately encoded and transmitted on several channels. Another suitable implementation is: Several marks are separately attached to the encoded PPT data and comment information, and the PPT data and comment information are transmitted on the same channel.

(c) Speakers generally require many tools, such as various colors, to speak their thoughts more accurately. Various actions on the copyboard are reflected in real time on display devices at different meeting locations.

When the conference terminal 40 at each conference location receives a video and audio signal or a data signal from the MCU 130, the application scenario of real time reception or manual reception is as follows:

(1) When the recording screen 134 is used, and a meeting place serving as a transmitter transmits a data signal such as a comment in real time or through manual operation: The person who uses the recording screen 34 is a speaker. During the talk, the speaker makes a mark on the PPT slide, for example, draws a straight line. At regular intervals, an image generated by the speaker's actions is sent to the data server 135, or the speaker starts the transfer function manually, and the data server 135 sends the image to the copyboard 133 and the local conference. It is sent to the other recording screen on the place and sent to the conference terminal 40. The meeting terminal 40 sends the image to the MCU 30, and the MCU 30 forwards the image to the meeting terminal 40 on another meeting place.

The conference terminal 40 of the conference place serving as a reception counterpart may receive a data signal forwarded by the MCU 130 in real time or manually.

(2) When the copyboard is used and the comment is transmitted in real time or through manual operation: The person using the copyboard 133 is a speaker. During the talk, the speaker makes a mark on the PPT slide, for example, draws a straight line. At regular intervals, an image generated by the speaker's actions is sent to the data server 135, or the speaker manually initiates the transfer function, and the data server 135 sends the image to another recording screen on the local meeting location ( 134 and the conference terminal 40. The meeting terminal 40 sends the image to the MCU 30, and the MCU 30 forwards the image to the meeting terminal 40 on another meeting place.

The conference terminal 40 of the conference site serving as a receiver may receive a data signal forwarded by the MCU 130 in real time or manually.

(3) In the above-described scenario, another participant corresponding to the copyboard and recording screen used by the non-speaker should make a comment on the lecture and make his comment on the reception screen.

Since the speaker's speech rate does not always match the rate at which other participants comment, two scenarios are possible:

The first page that other participants comment on is the same as the second page while the speaker is speaking.

The first page that other participants comment on is different from the second page while the speaker is speaking.

The solution is to provide a mode switch to the receiving terminal of the other participants. If passive reception is expected, the running mode switches to the real time update mode. In the real-time update mode, the content displayed by the receiving terminal is replaced with the speaker's lecture content, and other participants cannot make comments during the lecture. To be able to make comments, the mode must be switched to comment holding mode. In the comment holding mode, the content displayed by the terminal does not change with the speaker's lecture content, and other participants can make a comment during the lecture. After completing the comment, the mode is switched to the update mode.

(4) In scenario 1-2, the comment maker may expect the comment or sent page to be stored at any time in view of a summary or review after the meeting. A simple method is that after the storage function is enabled on the copyboard or recording screen operated by the comment maker, the comment maker stores the current comment in the data server 135 on the local meeting place.

(5) Sometimes two participants need to talk secretly during the meeting. In this case, the H.239 secondary stream channel shall be initiated for the participant in need of secret communication. A private token is generated from the H.239 secondary stream. After two secret talkers acquire a secret token, secret talker 1 sends a comment to secret talker 2, after receiving a comment sent by secret talker 1, secret talker 2 sends comment information to secret talker 1, Secret information is exchanged accordingly.

14 is a derivative view of FIG. 13. In Fig. 14, the copyboard 133 and the recording screen 134 are directly connected to the conference terminal 40 on the meeting place, and the data server 135 has existing data such as a PPT slide for lecture information and comment information. It is only configured to store the.

FIG. 15 is another derivative of FIG. In FIG. 15, the conference terminal 40 is a center, the copyboard 133 and the recording screen 134 are only connected to the conference terminal 40 on the conference venue, and the data server 135 is the conference terminal 40. In addition, the data server 135 is merely a storage device for storing data demonstrated on the copyboard 133 or existing data such as a PPT under lecture information or comment information.

The conference terminal 40 may include a storage module for storing data signals instead of the data server 135. Specifically, an input device such as the copyboard 133 may also store data signals generated by the copyboard itself, or may store data signals transmitted by other participants and forwarded by the conference terminal.

In the previous embodiment of the conference terminal, data collaboration between participants is performed based on the existing dual-stream functionality of the H.239 protocol, without involving any spare device, and the input source on the conference device. (E.g., electronic boards and recording screens) and input sources on other meeting venues are performed, thereby improving the interaction and discussion efficiency between users.

First Conferencing Server Example

As shown in FIG. 16, a conferencing server 60 is provided in an embodiment of the present invention. The conferencing server is typically an MCU applied to video conferencing, which provides data forwarding and services for each conferencing terminal. Conferencing server 60,

An image data receiving module 61 configured to receive an image data signal transmitted by the conference terminal;

A mark receiving module 62 configured to receive a mark data signal transmitted by the conference terminal;

An overlap module 63 configured to overlap the mark data signal with the image data signal to generate hybrid image data; And

A transmitting module 64 configured to transmit the hybrid image data, or the image data signal, or the mark data signal to the conference terminal

.

Secondary Conferencing Server Example

As shown in Fig. 17, when the superimposition module 63 superimposes the mark data signal on the image data signal, it is possible to receive the mark data signal transmitted by the plurality of conference terminals, and the superimposition module 63 is one It should be possible to distinguish the mark data signal transmitted by the conference terminal from the mark data signal transmitted by the other conference terminal. The overlap module 63 includes a filling module 631 and a processing module 632.

The filling module 631 is configured to assign a filler color or a transparent color to the mark data signal, process the mark data signal using the filling color or the transparent color, and generate a monochromatic mark data signal. The content of the mark data signal can be represented by a color bitmap such as a vector graph, a binary bitmap, a YUV monochrome bitmap, an RGB monochrome bitmap. When the content of the mark data signal is represented by a vector bitmap or a binary bitmap, the filling module may randomly assign a filling color to the mark data signal of each conference terminal. The overlap module 63 uses the filling color to fill the tracks of several conference locations when overlapping. When representing the content of the mark data signal in a color bitmap, such as a YUV or RGB bitmap, the MCU must specify a transparent color so that the receiving conference terminal can accurately identify the position of the track. In this method, after the mark data signal is superimposed on the image data signal, the integrity of the hybrid image data of each meeting place is assured.

The processing module 632 is configured to overlap the color mark data signal with the image data signal to generate hybrid image data.

When sending the hybrid image data or mark data signal to the conference terminal, the conference server 60 selects the H.239 secondary stream channel and controls the transmission module 64 to hybridize the image data via the H.239 secondary stream channel. Or a transmission control module 65 configured to transmit the mark data signal to the conference terminal.

In practice, the meeting server 60 receives the meeting place token request sent by the meeting terminal and returns the meeting place token to the meeting terminal after confirming that the application of the meeting place token was successful. It may further include. The transmission control module 65 selects the H.239 auxiliary stream channel according to the conference location token, and controls the transmission module 64 to transmit hybrid image data or mark data signals to the conference terminal via the H.239 auxiliary stream channel. do.

When a plurality of conference servers are in the system, the conference server may forward the superimposed hybrid image data or mark data signal sent by the conference terminal or the image data signal transmitted by the conference terminal to another conference server, Other conference servers send these signals to conference terminals connected to the conference servers.

If the conference terminal does not provide an overlapping function, the conference server 60 superimposes the mark data signal transmitted by each conference terminal on the image data signal (for example, slide) and then forwards this signal to other conference terminals. do. In this way, each conference terminal in the conference system can see the marks added by other conference terminals, cooperation between participants is performed, and user interactivity and discussion efficiency are improved.

Conference system Example

A conferencing system including the above-described conference terminal and the above-mentioned conference server will be described below.

The aforementioned conference terminal and the aforementioned conference server can produce the following three application scenarios:

(1) The conference terminal 1 sends its shared image data signal to the conference server, and the conference terminal 2 sends a local mark data signal to the conference server. The conference server superimposes the mark data signal on the image data signal to generate hybrid image data, and sends this hybrid image data to all conference terminals. After receiving the hybrid image data, the conference terminal decodes and displays the hybrid image data.

(2) The conference terminal 1 sends its shared image data signal to the conference server, and the conference terminal 2 sends a local mark data signal to the conference server. The conferencing server forwards these signals to all conference terminals. After receiving these signals, the conference terminal superimposes this mark data signal on the image data signal and displays the superimposed signal.

(3) The conference terminal 1 superimposes the local mark data signal on its shared image data signal (local image data or image data received from the remote side) to generate hybrid image data, and adds the hybrid image data to the H.239 auxiliary. Send to the conferencing server via the stream channel. The conference server forwards this hybrid image data to another conference terminal 2. After receiving the hybrid image data, the other conference terminal 2 decodes and displays this hybrid image data.

In all the above described application scenarios, all conference terminals are functionally identical. Conferencing terminal 1 is capable of local overlap, thus conferencing terminal 2, conferencing terminal 1 can transmit local image data and mark data signals, thus conferencing terminal 2.

First in a conference system Example

18 is a schematic structural diagram of a conference system according to the first application scenario described above. The conference system includes a terminal 71 and a notebook computer 72 connected to the terminal 71; Conference server 73; A display 75 having a conference terminal 74 and a touch screen and connected to the conference terminal 75; A conference terminal 76 and a conventional display 77 connected to the conference terminal 76. There is no limit to the number of conference terminals. Each conference terminal represents a conference location. The conference terminal 71 includes an image transmission module 53 configured to transmit the image data signal to the conference server 73.

The conferencing server 74 includes a mark transmitting module 48 configured to transmit a mark data signal to the conference server 73.

Similar to the conference server 60 described above, the conference server 73 is,

An image data receiving module 61 configured to receive a mark data signal transmitted by the image sending module 53;

A mark receiving module 62 configured to receive a mark data signal transmitted by the mark sending module 48;

An overlap module 63 configured to overlap the mark data signal with the image data signal to generate hybrid image data; And

Transmission module 64 configured to transmit hybrid image data to conference terminal 71, conference terminal 74, and conference terminal 76.

.

The conference terminal 71 and the conference terminal 76 are configured to receive and display hybrid image data.

Specifically, this application scenario includes the following process:

(1) The conference server 71 is a conference place where slides should be shared with other participants. The user can connect the notebook computer to the conference terminal 71 via a conventional Video Graphics Array (VGA) cable and send a slide to the conference server 73 via the H.239 secondary stream channel. The conference server 73 sends slides to the conference terminal 74 and the conference terminal 76.

(2) The conference terminal 74 is a conference place provided with the touch screen display 75. In the process of viewing the slide, the user of the meeting place has a query and needs to discuss it with others. Therefore, the users gather in a circle and discuss on the display 75. This prototype is recorded by the touchscreen on the display and sent to the conference server 73 via the conference terminal 74. In addition, if the conference server 73 does not have a slide viewed by the user of the conference terminal 74, the conference terminal 74 must send the slide to the conference server 73.

(3) After receiving the circular mark sent by the conference terminal 74, the conferencing server 73 decodes the video stream of the slide to obtain the slide image, and then overlaps the circular mark, The superimposed circular mark is encoded, and the encoded hybrid image stream is sent to all meeting places through the H.239 auxiliary stream channel.

(4) The conference terminal 76 is a conventional conference location connected to the conventional display 77. This conference location receives the hybrid video stream sent by the conference server 73 via the H.239 secondary stream channel. After being decoded, a slide image with its superimposed circular mark sent by conference terminal 74 can be viewed.

Similarly, conference terminal 71 and all other conference venues receive a slide image as received by conference terminal 76. If the other meeting place also has a mark input device such as the meeting terminal 74, this meeting terminal can also send the mark which can be seen at all meeting places.

Second of the conference system Example

19 illustrates an application scenario according to a plurality of connected conference servers of a conference system based on the first embodiment of the conference system described above. In FIG. 19, the double-headed arrow indicates transmission of the hybrid image data stream.

When there are a plurality of conference servers in the system, the plurality of conference servers are connected to each other. Each circuit server may be connected to one or more conference terminals. One of the conferencing servers is a master conference server and the other conferencing servers are slave conference servers. The master conference server is responsible for superimposing the added mark data signal on the shared image data signal transmitted by the conference terminal.

In this embodiment, it is assumed that two MCUs (conference servers) are connected. The system includes MCU1, MCU2, conference terminal T1, conference terminal T2, and conference terminal T3. MCU1 is the master MCU and MCU2 is the slave MCU. If more than two MCUs are connected, one of these MCUs is the master MCU and the other MCU is the slave MCU.

During the meeting, conference terminal T1 sends its shared image data signal to its connected MCU2, and conference terminal T2 sends its mark data signal to its connected MCU2. The slave MCU2 forwards the shared image data signal of T1 and the mark data signal of T2 in the conference system to the master MCU (ie, MCU1). The master MCU (MCU1) in the conference system superimposes the mark data signal on the image data signal and compresses the signal to generate hybrid image data, which is forwarded to all conference terminals in the conference system. In the conference terminal connected to the slave MCU MCU1, the hybrid image data is sent to each slave MCU by the master MCU in the conference, and then forwarded by the slave MCU. After receiving the hybrid image data, each terminal in the conference system decodes and displays this hybrid image data.

Even when more MCUs are connected, the operation is similar. The master MCU in the connected conference system is responsible for superimposing the mark data signal on the image data signal during the conference.

In the above scenario, a plurality of MCUs are connected in the system, one of which is the master MCU and the other being the slave MCU. In practice, it is possible for multiple MCUs in the system to be master MCUs. That is, each MCU is responsible for overlapping and forwarding. In order to perform coordination between the plurality of MCUs, the system may include a control unit configured to obtain a transmission time of the mark data signal and generate a control signal in order of the transmission time of the mark data signal.

Overlapping modules in at least two master conference servers superimpose the mark data signal on the image data signal according to the control signal to generate hybrid image data, and send the hybrid image data signal to each conference terminal in the system. The control unit may be a gatekeeper in the system. Gatekeeper is a softswitch based switch server that handles signal switching and control over Voice over Internet Protocol (VoIP) networks. The control unit may also be another control module of similar function.

Token control mode can be used instead of the gatekeeper to control collaboration between conference servers. In the system, the time for each conference terminal to send a mark data signal is notified to each master conference server. The master conference server actively acquires a control token in accordance with the transmission time of the mark signal. After this master conference server completes the nesting, the next master conference server acquires control tokens in the time sequence in which all conference terminals send mark data signals, and the nesting continues until the final master conference server completes the nesting. do. The final master conference server sends the final hybrid image data signal to each conference terminal in the system. Each master conference server actively acquires a control token or the gatekeeper assigns a control token to each master conference server.

Third of the conference system Example

20 is a schematic structural diagram of a conferencing system in the above-described second application scenario. The conference system includes a conference terminal 81, a notebook computer 82 connected to the conference terminal 81, and a conventional display 83; Conferencing server 84; A conference terminal 85, and a display 86 connected to the conference terminal 85 and having a touch screen; Conference terminal 87, and a conventional display 88 connected to the conference terminal 87; Each conference terminal represents a conference location. The number of conference terminals is not limited. The conference terminal 81 includes an image transmission module 53 configured to send an image data signal to the conference server 83.

The conference terminal 85 includes a mark transmission module 48 configured to transmit a mark data signal to the conference server 83.

The conference server 83 forwards the received image data signal and mark data signal to the conference terminal 87 and the conference terminal 81.

Both the conference terminal 87 and the conference terminal 81,

A first timer 45 configured to generate a first time interval value;

An image acquisition module 42 configured to receive an image data signal forwarded by the conferencing server;

A first external mark acquisition module 4111, configured to receive a mark data signal forwarded by the conferencing server 83, output the mark data signal according to a first time interval value, and generate a first mark data signal;

A second overlapping module 431, configured to overlap the first mark data signal with the image data signal to generate a second hybrid image data signal; And

Display module, such as display 88, configured to display the second hybrid image data signal.

.

The conference terminal 87 receives the mark data signals from the plurality of conference terminals, enables the receiver to identify the mark data signal transmitted by each conference terminal upon overlapping, and is transmitted by each conference terminal. The mark data signal should include the conference number, the terminal number and the corresponding mark data signal content of this conference terminal. The content of the mark data signal may be represented by a color bitmap such as a vector graph, a binary bitmap, a YUV monochrome bitmap, or an RGB monochrome bitmap. When the content of the mark data signal is represented by a vector bitmap or a binary bitmap, the conference server may randomly assign a filler color to each conference terminal. The overlapping module of the conference terminal uses the color assigned by the conference server to populate when overlapping tracks of several conference locations. When representing a track as a color bitmap, such as a YUV or RGB bitmap, a transparent color must be assigned to the system consistently so that the receiver identifies the position of the track and superimposes the position on the slide. Specifically, the first external mark acquisition module 4111,

Receives a mark data signal transmitted by at least one conference terminal except the local conference terminal and forwarded by the conference server, and from the mark data signal finds a corresponding decoder according to information such as conference number and terminal number; A first external mark acquisition submodule 41110, configured to transmit a mark data signal corresponding to the decoder;

At least one decoder configured to decode mark data signals of at least one terminal, respectively, and generate a mark data signal queue; And

A mark output module 41111 configured to output a mark data signal in accordance with a first time interval value and an order of the mark data signal queue, and to generate at least one first mark data signal

.

This application scenario includes the following process:

(1) The conference server 81 is a conference place where slides should be shared with other participants. The user connects the notebook computer to the conference terminal 81 (typically via a VGA cable) and sends a slide to all participants via the H.239 secondary stream channel.

(2) The conference terminal 85 is a conference location with a touch screen display 86. In the process of viewing the slide, the user of the meeting place has a query and needs to discuss it with others. Therefore, the users gather in a circle and discuss on the display 75. This prototype is recorded by the touchscreen on the display and sent to the conference server 84 via the conference terminal 85.

(3) The conference server 84 forwards the circular mark sent by the conference terminal 85 to the conference terminal 87 and the conference terminal 81.

(4) The conference terminal 87 is a conventional conference place, which receives a slide code stream from the conference terminal 81 and simultaneously receives a circular mark from the conference terminal 85. The conference terminal 87 decodes the slide code stream to obtain a slide image, and superimposes this circular mark on the slide image. In this way, one can see an image with its superimposed circular marks.

If the other meeting place also has a mark input device such as the conference terminal 85, this meeting place can also send the mark that all meeting places can see.

Fourth of the conference system Example

21 illustrates an application scenario according to a plurality of connected conference servers in a conference system based on the previous third embodiment of the conference system. In FIG. 21, the double-headed arrow indicates transmission of the hybrid image data stream.

It is assumed that two MCUs are connected in the system, MCU1 is the master MCU and MCU2 is the slave MCU. Even if more than two MCUs are connected, the scenario is similar. That is, one of these MCUs is a master MCU and the other MCU is a slave MCU.

In the conference system, conference terminal T1 sends its shared image data signal to its connected MCU2, and conference terminal T2 sends its local mark data signal to its connected MCU2. The slave MCU2 forwards its shared image data signal of T1 and the mark data signal of T2 to the master MCU (ie MCU1) in the conference system. The master MCU in the conference system forwards the shared received image data signal and mark data signal to each conference terminal in the conference system. In the conference terminal connected to the slave MCU (MCU1), the shared received image data signal and the mark data signal are sent by the master MCU to each slave MCU in the conference system and forwarded by the slave MCU. After receiving the shared received image data signal and the mark data signal, each terminal in the conference system superimposes this data signal on the image data signal and displays these signals.

Even when more MCUs are connected, the operation is similar. The master MCU in the connected conference system is responsible for collecting and forwarding the shared image data signal and mark data signal of all conference terminals in the conference system.

In the present embodiment, when the plurality of MCUs are master MCUs, the MCUs generate control signals in time order of adding mark data signals. Each master MCU sequentially forwards the received mark data signal to the receiving conference terminal according to the control signal. The receiving conference terminal overlaps and displays this signal. The control token is obtained according to the time sequence of adding the mark data signal. The control token can be actively obtained by the master MCU or can be delivered by a control unit in the system. Each master MCU forwards the received mark data signal in accordance with a control token, and each receiving conference terminal receives and overlaps the signal.

5th conference system Example

22 is a schematic structural diagram of a conferencing system in the above-described second application scenario. The conference system includes a conference terminal 91 and a notebook computer 92 connected to the conference terminal 91; Conference server 93; A conference terminal 94 and a display 95 connected to the conference terminal 94 and having a touch screen; A conference terminal 96 and a conventional display 97 connected to the conference terminal 96. The number of conference terminals is not limited, and each conference terminal represents a conference place. The conference terminal 91 is,

A second timer 46 configured to generate a second time interval value;

A first internal mark receiving module 4110, configured to receive a mark data signal transmitted by the local conference terminal, output the mark data signal according to a first time interval value, and generate a first mark data signal;

A second internal mark receiving module 4120, configured to receive a mark data signal transmitted by the local conference terminal, output the mark data signal according to a second time interval value, and generate a second mark data signal;

A first superimposition module 430, configured to superimpose the first mark data signal on the image data signal to generate a first hybrid image data signal; And

A first sending module 440 configured to send the second hybrid image data to a display screen of a local conference terminal or to send a first hybrid image data to a conference server via an H.239 auxiliary stream channel

.

The conference terminal 93 is configured to receive first hybrid image data transmitted by the at least one conference terminal 94 and to forward the first hybrid image data.

The conference terminal 96 and the conference terminal 91 are configured to receive, decode and display the first hybrid image data.

Specifically, this application scenario includes the following process:

(1) In the initial state, conference terminal 91 is connected to a notebook computer, obtains an H.239 secondary stream token, and shares a slide with other conference venues via an H.239 secondary stream channel (this is H. 239 is the same as sending the shared slide to other terminals via an auxiliary stream channel). Therefore, both the conference terminal 94 and the conference terminal 96 can view the slide of the conference terminal 91.

(2) When viewing the slide, the user of the conference terminal 94 may want to make a mark on the slide. To send this mark to others, conference terminal 94 performs the following process:

(2.1) Apply the control token of the H.239 secondary stream channel.

(2.2) Determine whether the conferencing controller (current token holder) approves the application. When the conferencing controller approves the application, it acquires a token, starts recording the mark data signal of the touch screen 95 of the conference terminal 94, and displays the mark data signal on the conference terminal 94. Nested in

(2.3) Encode the slide on which the mark data signal overlaps.

(2.4) Send the encoded slide stream to the conferencing server 93 via the H.239 secondary stream channel.

In this case, the conferencing server 94 is the transmitter of the H.239 secondary stream. If a conference terminal at another conference location wants to add a mark, the conference terminal takes the same action. In this way, by continuously switching the transmission tokens of the H.239 auxiliary stream, the conference terminal of each conference site can add the mark to the slide.

In this system, however, the conference terminal 94 may individually transmit the added mark via the touchscreen 95 using the token control mode and use this mark as the second H.239 auxiliary stream (slide). Is the first H.239 auxiliary stream). Two parameters must be added to enable switching when convening a meeting. One parameter indicates whether the second H.239 auxiliary stream is supported and the other parameter indicates whether the second H.239 auxiliary stream is mark information.

If the second H.239 auxiliary stream is mark information, the token control mode is used as the default. That is, the mark is also used in the mode for requesting a token. In this solution, "second H.239 auxiliary stream" is the opposite term for the H.239 auxiliary stream of the slide and may actually be any H.239 auxiliary stream.

The procedure for sending a mark in token control mode is as follows:

(3.1) Apply the control token of the mark as an H.239 auxiliary stream.

(3.2) Acquire the token successfully and start recording the mark data signal added via the touchscreen 95.

(3.3) Send the user's mark data signal (mark track) as a separate H.239 auxiliary stream to the conference server 93, which forwards the H.239 auxiliary stream to another conference terminal.

In practice, the mode of using the token to control the transmission of the mark data signal can be combined with the mode of using the token to control the hybrid slide.

6th conference system Example

FIG. 23 illustrates an application scenario according to a plurality of connected conference servers of a conference system based on the fifth embodiment of the conference system described above. In FIG. 23, the double-headed arrows indicate the transmission of the hybrid image data stream.

It is assumed that two MCUs are connected in the system, MCU1 is the master MCU and MCU2 is the slave MCU. Even if more than two MCUs are connected, the scenario is similar. That is, one of these MCUs is a master MCU and the other MCU is a slave MCU.

The conferencing terminal T1 locally generates a hybrid image data by superimposing a local mark data signal on its shared image data signal and then sends the hybrid image data to MCU2 via an H.239 auxiliary stream channel. MCU2 forwards this hybrid image data to the master MCU (MCU1) at the conference. The master MCU MCU1 sends the received hybrid image data to each conference terminal in the conference system. The conference terminal connected to the slave MCU in the conference system receives the hybrid image data forwarded by the slave MCU.

Even when more MCUs are connected, the operation is similar. The master MCU in the connected conference system constantly forwards the hybrid image data sent to all conference terminals in the conference system. Alternatively, every MCU in the system is a master MCU, the hybrid image data is forwarded by the MCU in a particular order, and one of the MCUs can be finally forwarded to avoid network congestion.

In all embodiments of the conferencing system, if network congestion or packet loss occurs when data is transferred between the conference terminal and the conferencing server, network congestion or packet loss is resolved in the following manner, so that users in the system have a better experience. Will be:

(1) Reliable transmission: For example, enlarged data such as image data signals and hybrid image data are transmitted via User Datagram Protocol (UDP) to prevent network congestion.

(2) Packet loss prevention technology: Redundant data is transmitted using packet loss recovery technology. If a small packet is lost, for example if the mark data signal is lost, the size of the lost data is small, so the receiver uses redundant data along with the received data to recover the lost data. Data is transmitted using a reliable transmission protocol, such as Transmission Control Protocol (TCP), enabling reliable transmission of mark data signals between devices. Mark data signals can also be transmitted via other reliable protocols such as UDP.

All packet loss prevention in existing systems can be applied to these interactive applications, improving the application experience in case of packet loss or network congestion.

When the mark data signal is transmitted by the conference terminal to the conference server and forwarded by the conference server to another conference terminal, the mark data signal is transmitted via the TDP protocol or UDP protocol of the network, or HMLP (HMLP) of the H.320 system. Speed Multilayer Protocol). To ensure real-time effects, mark data signals are generally transmitted over UDP in an IP network.

Alternatively, on the MCU side or the receiving conference terminal side, the time of receiving data is recorded, and the mark data signal is superimposed on the image data signal in the order of the receiving time. In this way, the mark data signal added by each conference terminal can be superimposed exactly, and the user experience is improved. Alternatively, the time stamp is added to record the transmission time when the conference terminal transmits the image data signal and the mark data signal. The MCU may obtain a time stamp when receiving the image data signal or the mark data signal. In particular, the mark data signal is superimposed on the image data in accordance with the time for transmitting the mark data signal. In this way, the mark data signal of each conference terminal can be superimposed exactly on the image data signal, improving user experience and interactivity.

In the above-mentioned conference system, it is not necessary to configure an extra data conference system or device in the process of interactive cooperation between users; Data collaboration between participants is carried out through the existing dual-stream capabilities of the H.239 standard, which improves user interactivity and discussion efficiency.

Way Example :

As shown in Fig. 24, a data processing method is provided in this embodiment. Way,

Step S160: Acquire an image data signal.

Step S161: Acquire a mark data signal.

Step S162: The mark data signal is superimposed on the image data signal to generate a hybrid image data signal.

Step S163: Output the hybrid image data signal. The hybrid image data signal may be output to the local conference terminal and displayed, or may be output to a conference server that forwards the hybrid image data to another conference terminal.

In the above method, the mark data signal is superimposed on the obtained image data signal to generate a hybrid image data signal (for example, the hybrid image data signal is a slide with an added mark). In this way, participants can add marks to the slide while viewing the slide, improve interactivity between video conferencing users, and improve discussion efficiency.

The following describes the process of executing the method shown in FIG. 24 in detail.

Step S170: Acquire an image data signal.

Step S171: Acquire a mark data signal input by the local conference terminal.

Step S172: Generate a first time interval value.

Step S173: Generate a second time interval value.

Step S174: Output the mark data signal according to the first time interval value to generate a first mark data signal.

Step S175: Output a mark data signal according to a second time interval value to generate a second mark data signal, and encode a second mark data signal to generate a third mark data signal.

Step S176: Determine whether the mark auxiliary stream is supported. If the mark auxiliary stream is supported, step S177 is performed. If the mark auxiliary stream is not supported, step S178 is performed or a third mark data signal is sent through a user-defined channel.

Step S177: Apply the meeting place in the H.239 secondary stream channel. After the conference location token is successfully obtained, generate a first control signal and send a third mark data signal to the conference server via the H.239 secondary stream channel in accordance with the first control signal.

Step S178: Apply the meeting place token in another H.239 secondary stream channel. After successfully obtaining the meeting place token, a second control signal is generated, and according to the second control signal, the first mark data signal is superimposed on the mark data signal to generate first hybrid image data.

Step S179: send the first hybrid image data to the display screen of the local conference terminal, or send the first hybrid image data to the conference server via the H.239 secondary stream channel, wherein the first hybrid image data is sent to the other conference by the conference server. Forwarded to the terminal.

In the executing process, step S171 may precede this step: receiving the subtoken application of the input device at the conference location where the local conference terminal is located, confirming that the application of the subtoken was successful, and then subtoking the input device to the input device. Allocate In this case, step S171 may be: acquiring the mark data signal of the input device on the conference location where the local conference terminal is located according to the subtoken.

In addition, when the received mark data signal is transmitted by another terminal, another process of executing the method shown in FIG. 24 will be described in detail below.

Step S180: Acquire an image data signal.

Step S181: Acquire a mark data signal transmitted by at least one conference terminal except the local conference terminal and forwarded by the conference server. Specifically, such mark data signal can be manually received in real time.

Step S182: Generate a first time interval value.

Step S183: Decode mark data signals of at least one conference terminal, respectively, and generate a mark data signal queue.

Step S184: Output the mark data signal according to the order of the first time interval value and the mark data signal queue, and generate at least one first mark data signal.

Step S185: The first mark data signal is superimposed on the image data signal to generate a second hybrid image data signal.

Step S186: Send the second hybrid image data to the display screen of the local conference terminal.

Each mark data signal includes a conference number, a conference terminal number, and the contents of the mark data signal. The conference number and conference terminal number are designed so that the receiver can reliably identify the mark data signal transmitted by each conference terminal in the overlapping process, and the content of the mark data signal is a vector graph, a binary bitmap, a YUV monochrome. It can be represented by a color bitmap such as a bitmap, an RGB monochrome bitmap. If the content of the mark data signal is represented by a vector bitmap or a binary bitmap, the conferencing server can randomly assign a filling color to each conference terminal, and the overlapping module of the conference terminal provides multiple conference locations. Use the color assigned by the conferencing server to fill when overlapping tracks. When representing the content of the mark data signal in a monochrome bitmap such as a YUV or RGB bitmap, a transparent color must be constantly assigned to the system so that the receiver correctly identifies the position of the mark data signal and superimposes this position on the slide.

other way Example :

As shown in Fig. 25, another data processing method is provided in this embodiment. In this method,

Step S190: Receive an image data signal and a mark data signal sent by the conference terminal.

Step S191: Generate the hybrid image data by superimposing the mark data signal on the image data signal.

Step S192: Send hybrid image data, or image data signal, mark data signal to the conference terminal.

Step S191,

Assigning a filling color or transparent color to the mark data signal and using the filling color or transparent color to process the mark data signal to generate a monochromatic mark data signal, wherein the content of the mark data signal is a vector graph, a binary bitmap, Generating the YUY monochrome bitmap, or an RGB monochrome bitmap; And

Generating hybrid image data by superimposing the monochromatic mark data signal on an image data signal;

It includes.

When representing the contents of the mark data signal as a vector graph or a binary bitmap, the MCU can randomly assign the filling color or mark data signal of each conference terminal, and the conference can be filled to fill when overlapping tracks of multiple conference venues. The nested module of the terminal uses its assigned filling color. When representing the mark data signal as a monochrome bitmap such as YUV or RGB, the MCU allows the receiving conference terminal to accurately identify the position of the track and hybridize each conference location after the mark data signal is superimposed on the image data signal. Transparent colors must be specified to ensure the preservation of image data.

Before step S192, the following steps may be included: after receiving the meeting place token request sent by the meeting terminal and confirming that the application of the meeting place token was successful, the meeting place token is returned to the meeting terminal. In this case, step S192 is as follows: When sending the hybrid image data or mark data signal to the conference terminal according to the conference location token, the conference server selects the H.239 secondary stream channel and sends the hybrid image data or mark data signal. It can be sent to the conference terminal via the H.239 secondary stream channel. The system supports the H.239 protocol, the structure of the system is simple, and all conference terminals in the conference system can see the marks added by other conference terminals. Collaboration between participants is performed, improving user interactivity and discussion efficiency.

Those skilled in the art will appreciate that some or all of the steps of the method provided in any of the above-described embodiments may be realized by a program instructing relevant hardware. Such programs may be stored in a computer readable storage medium. When the program is executed, the steps in the foregoing method embodiments are performed.

delete

Claims (48)

In a conference terminal,
A mark acquisition module configured to acquire a mark data signal;
An image acquisition module, configured to acquire an image data signal;
An overlapping module configured to overlap the mark data signal with the image data signal to generate a hybrid image data signal;
An output module configured to output the hybrid image data signal;
A first timer configured to generate a first time interval value; And
A second timer configured to generate a second time interval value
Including,
The mark acquisition module,
A first mark acquisition module configured to acquire the mark data signal and to output the mark data signal according to the first time interval value to generate a first mark data signal; And
A second mark acquisition module, configured to acquire the mark data signal and to output the mark data signal according to the second time interval value to generate a second mark data signal
Containing, conference terminal. The method of claim 1,
The first mark acquiring module is configured to acquire a mark data signal transmitted by a local conference terminal and to output the mark data signal according to the first time value to generate the first mark data signal. Acquisition module,
The second mark acquiring module is configured to acquire a mark data signal transmitted by the local conference terminal, and output the mark data signal according to the second time value to generate the second mark data signal. A conference terminal, which is a mark acquisition module. The method of claim 2,
A mark encoding module configured to encode the second mark data signal to generate a third mark data signal; And
A mark transmitting module, configured to transmit the third mark data signal to a conference server through an auxiliary stream channel or a user-defined channel
It further comprises a conference terminal. The method of claim 3,
A first token control module configured to apply for a conference site token in the auxiliary stream channel to generate a first control signal after successfully obtaining the conference site token,
And the mark transmitting module transmits a third mark data signal to the conference server through the auxiliary stream channel according to the first control signal. The method of claim 1,
The first mark acquisition module,
Obtain the mark data signal transmitted by at least one conference terminal except the local conference terminal and forwarded by a conference server, and output the mark data signal according to the first time interval value to output the first mark data signal. A first external mark acquisition module, adapted to generate. The method of claim 5,
The first external mark acquisition module,
A first external mark acquisition submodule, configured to obtain a mark data signal transmitted by the at least one conference terminal and forwarded by the conference server;
At least one decoder configured to decode mark data signals of the at least one conference terminal, respectively, and generate a mark data signal queue; And
A mark output module configured to output the mark data signal according to the order of the mark data signal queue to generate at least one first mark data signal
Containing, conference terminal. The method according to claim 6,
The superimposition module is a first superimposition module configured to overlap the first mark data signal with the image data signal to generate a first hybrid image data signal,
And the output module is a first transmission module configured to transmit the first hybrid image data to a display screen of the local conference terminal. The method of claim 2,
And a second token control module configured to apply for a meeting place token in an auxiliary stream channel to generate a second control signal after successfully obtaining the meeting place token. 9. The method of claim 8,
The superimposition module is a second superimposition module configured to superimpose the first mark data signal on the image data signal to generate second hybrid image data according to the second control signal,
And the output module is a second transmitting module configured to transmit the first hybrid image data to a display screen of the local conference terminal or to transmit the second hybrid image data to a conference server through the auxiliary stream channel. . The method of claim 2,
And a token management module configured to receive a sub-token request of an input device at a meeting place where the local conference terminal is located, to allocate a sub-token to the input device after confirming that the sub-token application is successful,
The first internal mark acquiring module acquires a mark data signal of an input device at the conference location where the local conference terminal is located, outputs the mark data signal according to the first time interval value, and outputs the first mark. Generates a data signal, and
The second internal mark acquisition module acquires a mark data signal of an input device at the conference location where the local conference terminal is located according to the sub-token, and obtains the mark data signal according to the second time interval value. Outputting to generate the second mark data signal. The method of claim 1,
The mark acquisition module is a real-time mark acquisition module configured to acquire the mark data signal in real time,
And the image acquisition module is a real time image acquisition module configured to acquire the image data signal in real time. The method of claim 1,
The mark acquiring module is configured to provide a manual receiving option and to manually acquire the mark data signal when the manual receiving option is selected,
And the image acquisition module is configured to provide a manual reception option and to manually acquire the image data signal when the manual reception option is selected. 13. The method according to any one of claims 1 to 12,
After receiving a secret token request sent by the input device at the conference location where the local conference terminal is located, conducting a secret meeting with a target input device, after confirming that the secret token application is successful, the auxiliary stream And a secret token control module configured to assign a secret token to the input device using a channel or a user-defined channel. In the conferencing server,
An image data receiving module configured to receive an image data signal sent by the conference terminal;
A mark receiving module configured to receive a mark data signal by the conference terminal;
An overlapping module configured to overlap the mark data signal with the image data signal to generate hybrid image data; And
A transmission module configured to transmit the hybrid image data, or the image data signal, or the mark data signal
Including,
The overlap module,
A filling module configured to assign a filler color or a transparent color to the mark data signal, process the mark data signal using the filler color or the transparent color, and generate a color mark data signal; And
A processing module configured to overlap the color mark data signal with the image data signal to generate hybrid image data
Containing a conference server. 15. The method of claim 14,
A transmission control module configured to select an auxiliary stream channel and control the transmission module to transmit the hybrid image data, or the image data signal or the mark data signal, to the conference terminal or another conference server via the auxiliary stream channel; Conferencing server further includes. 16. The method of claim 15,
A token management module configured to receive a meeting place token request sent by the meeting terminal, and to return the meeting place token to the meeting terminal after confirming that the application of the meeting place token was successful,
The transmission control module selects the auxiliary stream channel according to the conference location token, and controls the transmission module to exchange the hybrid image data, the image data signal, or the mark data signal through the auxiliary stream channel. Or a conferencing server, which sends to another conferencing server. In the conference system,
A first conference terminal, a conference server, and at least one second conference terminal,
The first conference terminal,
An image acquisition module configured to acquire an image data signal forwarded by the conference server; And
A mark transmission module configured to acquire a mark data signal and to transmit the mark data signal to the conference server
/ RTI >
The conference server,
An image data receiving module, configured to receive the image data signal transmitted by any conference terminal in the conference system;
A mark receiving module configured to receive the mark data signal;
An overlapping module configured to overlap the mark data signal with the image data signal to generate hybrid image data; And
A transmission module configured to transmit the hybrid image data to the first conference terminal and the at least one second conference terminal.
/ RTI >
The at least one second conference terminal is configured to receive and display the hybrid image data,
The conference servers are at least two connected conference servers,
The at least two connected conference servers include one master conference server and at least one slave conference server,
The master conference server transmits the hybrid image data to the at least one slave conference server and a conference terminal connected to the master conference server,
The slave conference server transmits the received hybrid image data to a conference terminal coupled to the slave conference server. 18. The method of claim 17,
The at least two conference linked servers are both master conference servers,
The conference system,
A control unit configured to obtain a transmission time of the mark data signal, and generate a control signal according to the order of the transmission time of the mark data signal,
The superimposition module at each of the at least two master conference servers superimposes the mark data signal on the image data signal according to the control signal to generate the hybrid image data. 19. The method of claim 18,
The at least two master conference servers obtain a control token according to the control signal,
The superimposition module superimposes the mark data signal on the image data signal according to the control token to generate the hybrid image data. In the conference system,
A first conference terminal, a conference server, and at least one second conference terminal,
The first conference terminal,
An image acquisition module, configured to acquire an image data signal sent by the conference server; And
A mark transmission module configured to acquire a mark data signal and to transmit the mark data signal to the conference server
/ RTI >
The conferencing server is configured to receive the image data signal and the mark data signal and to forward the image data signal and the mark data signal,
The at least one second conference terminal,
A first timer configured to generate a first time interval value;
An image acquisition module, configured to acquire the image data signal forwarded by the conference server;
A first external mark acquisition module, configured to acquire the mark data signal forwarded by the conference server, and output the mark data signal according to the first time interval value to generate a first mark data signal;
A first superposition module configured to overlap the first mark data signal with the image data signal to generate a first hybrid image data signal; And
Displaying module configured to display the first hybrid image data signal
Containing, conference system. 21. The method of claim 20,
The first external mark acquisition module,
A first external mark acquisition submodule, configured to obtain mark data signals transmitted by at least one other conference terminal and forwarded by the conference server;
At least one decoder configured to decode mark data signals of the at least one other conference terminal, respectively, and generate a mark data signal queue; And
A mark output module configured to output the mark data signals according to the order of the first time interval value and the mark data signal queue to generate at least one first mark data signal
Containing, conference system. 22. The method according to claim 20 or 21,
The conferencing servers are at least two connected conferencing servers. The method of claim 22,
The at least two connected conference servers include one master conference server and at least one slave conference server,
The master conference server transmits the image data signal and the mark data signal to the at least one slave conference server and a conference terminal connected to the master conference server,
And the slave conference server transmits the image data signal and the mark data signal to a conference terminal connected to the slave conference server. The method of claim 22,
The at least two conference linked servers are master conference servers,
The conference system,
A control unit configured to obtain a transmission time of the mark data signal, and generate a control signal according to the order of the transmission time of the mark data signal,
The at least two master conference servers forward the image data signal and the mark data signal in accordance with the control signal. 25. The method of claim 24,
The at least two master conference servers acquire a control token in accordance with the control signal and forward the image data signal and the mark data signal in accordance with the control token. In the conference system,
A first conference terminal, a conference server, and at least one second conference terminal,
The first conference terminal,
A first timer configured to generate a first time interval value;
A second timer configured to generate a second time interval value;
A first internal mark acquisition module, configured to acquire a mark data signal sent by a local conference terminal, and output the mark data signal according to the first time interval value to generate a first mark data signal;
A second internal mark acquiring module, configured to acquire the mark data signal transmitted by the local conference terminal, and output the mark data signal according to the second time interval value to generate a second mark data signal;
A second superposition module configured to superimpose the mark data signal on an image data signal to generate second hybrid image data; And
A second transmission module, configured to transmit the first hybrid image data to a display screen of the local conference terminal or to transmit the second hybrid image data to the conference server through an auxiliary stream channel.
/ RTI >
The conferencing server is configured to receive and forward the second hybrid image data,
And the at least one second conference terminal is configured to receive and display the second hybrid image data. The method of claim 26,
The first conference terminal,
A second token control module, configured to apply for a venue token in another auxiliary stream channel to generate a second control signal after successfully obtaining the venue token;
And the second superimposition module superimposes the first mark data signal on the image data signal according to the second control signal to generate second hybrid image data. The method of claim 26 or 27,
The conferencing servers are at least two connected conferencing servers. 29. The method of claim 28,
The at least two connected conference servers include one master conference server and at least one slave conference server,
The master conference server transmits the second hybrid image data to the at least one slave conference server and a conference terminal connected to the master conference server,
And the slave conference server transmits the received second hybrid image data to a conference terminal coupled to the slave conference server. A data processing method for implementing data exchange between conference terminals,
Obtaining an image data signal;
Obtaining a mark data signal;
Generating a hybrid image data signal by superimposing the mark data signal on the image data signal; And
Outputting the hybrid image data signal
Including,
After acquiring the mark data signal,
Generating a first time interval value;
Generating a second time interval value;
Generating a first mark data signal by outputting the mark data signal according to the first time interval value; And
Outputting the mark data signal according to the second time interval value to generate a second mark data signal
More,
Acquiring the mark data signal,
Acquiring the mark data signal transmitted by a local conference terminal. 31. The method of claim 30,
After outputting the mark data signal according to the second time interval value and generating the second mark data signal,
Encoding the second mark data signal and generating a third mark data signal; And
Transmitting the third mark data signal to a conferencing server via an auxiliary stream channel;
Data processing method further comprising. 32. The method of claim 31,
Transmitting the third mark data signal to the conference server through an auxiliary stream channel,
Requesting a token on the auxiliary stream channel to generate a first control signal after successfully obtaining the token; And
And transmitting the third mark data signal to the conference server through the auxiliary stream channel in accordance with the first control signal. 31. The method of claim 30,
Acquiring the mark data signal,
Acquiring the mark data signals transmitted by at least one conference terminal other than a local conference terminal and forwarded by a conference server. 34. The method of claim 33,
After acquiring the mark data signals transmitted by at least one conference terminal except the local conference terminal and forwarded by a conference server,
Decoding said mark data signals of said at least one conference terminal, respectively; And
Outputting the mark data signals according to the order of the first time interval value and the mark data signal queue, and generating at least one first mark data signal
Data processing method further comprising. 35. The method of claim 34,
Superimposing the mark data signal with the image data signal to generate a hybrid image data signal,
Generating a first hybrid image data signal by overlapping the at least one first mark data signal with the image data signal,
The outputting of the hybrid image data signal may include:
And transmitting the first hybrid image data to a display screen of the local conference terminal. 31. The method of claim 30,
After generating the first mark data signal by outputting the mark data signal according to the first time interval value,
Applying for a token in an auxiliary stream channel to generate a second control signal after successfully obtaining the token,
Generating the hybrid image data signal by superimposing the mark data signal on the image data signal,
Generating second hybrid image data by superimposing the first mark data signal on the image data signal according to the second control signal;
The outputting of the hybrid image data signal may include:
Transmitting the second hybrid image data to a display screen of the local conference terminal, or transmitting the second hybrid image data to a conference server through the auxiliary stream channel. 31. The method of claim 30,
Prior to acquiring the mark data signal transmitted by the local conference terminal,
Receiving a sub-token request of an input device on a meeting place where the local conference terminal is located, and after confirming that the sub-token application is successful, allocating a sub token to the input device;
More,
Acquiring the mark data signal transmitted by the local conference terminal,
And acquiring, according to the sub-token, the mark data signal input by an input device on a conference location where the local conference terminal is located. 34. The method of claim 33,
Acquiring the mark data signals transmitted by the at least one conference terminal except the local conference terminal and forwarded by the conference server,
Receiving in real time or manually the mark data signal transmitted by the at least one conference terminal except the local conference terminal and forwarded by the conference server. A data processing method for implementing data exchange between conference terminals,
Receiving an image data signal and a mark data signal transmitted by the conference terminal;
Generating hybrid image data by superimposing the mark data signal on the image data signal; And
Transmitting the hybrid image data, the image data signal, or the mark data signal to the conference terminal.
Including,
Generating the hybrid image data by superimposing the mark data signal on the image data signal,
Assigning a filling color or a transparent color to the mark data signal, processing the mark data signal using the filling color or the transparent color, and generating a monochromatic mark data signal; And
Generating the hybrid image data by superimposing the monochromatic mark data signal on the image data signal
Comprising a data processing method. 40. The method of claim 39,
The step of transmitting the hybrid image data, the image data signal, or the mark data signal to the conference terminal,
Selecting an auxiliary stream channel and transmitting the hybrid image data or the mark data signal to the conference terminal via the auxiliary stream channel. 41. The method of claim 40,
Prior to the step of transmitting the hybrid image data, or the image data signal, or the mark data signal, to the conference terminal,
Receiving a meeting place token request sent by the meeting terminal and returning the meeting place token to the meeting terminal after confirming that the application of the meeting place token was successful.
More,
The step of transmitting the hybrid image data, the image data signal, or the mark data signal to the conference terminal,
And transmitting the hybrid image data, or the image data signal, or the mark data signal to the conference terminal at another conference venue in accordance with the conference venue token.

delete delete delete delete delete delete delete

Images