KR20160061773A - Method for generating conference video screen, server and system - Google Patents

Abstract

A method for configuring a conference screen between different kinds of conference video systems, may include a step of receiving a first conference screen from a first conference video system, a step of receiving a second conference screen from a second conference video system, a step of detecting at least one speaker from the second conference video system, a step of generating a third conference screen by reconfiguring the second conference screen based on the detected speaker, and a step of generating a fourth conference screen by making the first conference screen and the third conference screen merged. So, a multilateral video conference between the different kinds of conference video systems can be implemented.

Description

METHOD FOR GENERATING CONFERENCE VIDEO SCREEN, SERVER AND SYSTEM,

The present invention relates to a method, a server, and a system for configuring a conference screen.

The video conferencing system combines multiparty voice and images, and mixes the data and sends it to participants of each conference. Each participant of the conference can participate in a conference using a device such as a mobile communication terminal and a video terminal.

On the other hand, conventional video conferencing devices were compatible only between products manufactured by the same manufacturer. For this reason, consumers are dependent on specific manufacturers, and the choice of videoconferencing devices has been limited. In addition, interworking between the video conferencing device and the software video conferencing solution can not be performed, and it has been difficult to popularize the remote video conferencing.

In Korean Patent Laid-Open Publication No. 2011-0112686, priority is assigned to a camera for each microphone to set a microphone and camera environment information, a speaker who speaks through a microphone is taken through a camera in consideration of a microphone environment setting, On a video conference screen.

It is desired to enable a multi-party video conference between heterogeneous video conference systems by detecting a speaker from the video conference system and dynamically reconfiguring the multi-party conference screen based on the detected speaker. It is to be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to a first aspect of the present invention, there is provided a method of configuring a conference screen between heterogeneous video conferencing systems, comprising: receiving a first conference screen from a first video conferencing system; Receiving a second screen from the second video conference system, detecting one or more speakers from the second video conference system, reconstructing the second conference screen based on the detected speaker to generate a third conference screen, And generating a fourth screen by merging the first screen and the third screen.

 The integrated platform server of the heterogeneous video conferencing system according to the second aspect of the present invention includes a receiving unit that receives a first screen from the first video conferencing system and receives a second screen from the second video conferencing system, A second detecting unit for detecting one or more speakers from the two-party video conference system; and a second speaker for reproducing the second speaker screen based on the detected speaker and generating a third speaker screen, And a screen reconstruction unit for generating a fourth screen.

The heterogeneous video conferencing system according to the third aspect of the present invention includes a first video conferencing system for transmitting a first conference screen to an integrated platform server and receiving a fourth conference screen from the integrated platform server, A second video conference system for transmitting the first video conference system to the integrated platform server and receiving the fourth video conference screen from the integrated platform server and a second video conference system for receiving the first video conference system from the first video conference system, Receiving a second conference screen, detecting one or more speakers from the second conference system, reconstructing the second conference screen based on the detected speaker, generating a third conference screen, And an integrated platform server for merging the screen and the third screen to generate the fourth screen.

The above-described task solution is merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and the detailed description of the invention.

According to any one of the above-mentioned objects of the present invention, the integrated platform server can detect the speaker from the video conference system and dynamically reconfigure the multi-party conference screen based on the detected speaker. In addition, the integrated platform server can provide multipoint videoconferencing between heterogeneous videoconferencing systems with minimal computational cost through software videoconferencing solutions. In addition, the integrated platform server enables protocol interworking between video conferencing equipment and software video conferencing solution made by different manufacturers, so that it is possible to provide video conferencing service anytime and anywhere without being restricted by the type of video conferencing equipment owned by the user .

1 is a configuration diagram of a heterogeneous video conferencing system according to an embodiment of the present invention.
FIG. 2 is a detailed configuration diagram of the integrated platform server shown in FIG. 1 according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating a problem of a conference screen generated between existing heterogeneous video conferencing systems.
4A and 4B are views showing a conference screen between heterogeneous video conferencing systems that solve the problem of FIG. 3, according to an embodiment of the present invention.
5 is a flowchart illustrating a method of configuring a conference screen among heterogeneous video conferencing systems according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

In this specification, the term " part " includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be implemented using two or more hardware, or two or more units may be implemented by one hardware.

In this specification, some of the operations or functions described as being performed by the terminal or the device may be performed in the server connected to the terminal or the device instead. Similarly, some of the operations or functions described as being performed by the server may also be performed on a terminal or device connected to the server.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

1 is a configuration diagram of a heterogeneous video conferencing system according to an embodiment of the present invention.

Referring to FIG. 1, the heterogeneous video conferencing system may include a first video conferencing system 100, a second video conferencing system 110, and an integrated platform server 120. However, since the heterogeneous video conferencing system of FIG. 1 is only one embodiment of the present invention, the present invention is not limited to FIG. 1 and may be configured differently from FIG. 1 according to various embodiments of the present invention . Here, it is assumed that the first video conferencing system 100 and the second video conferencing system 110 are different types of video conferencing systems.

The first video conferencing system 100 may receive a conference call from one or more video conferencing devices (e.g., a smartphone, tablet PC, desktop PC, dedicated video conferencing device, etc.) Lt; / RTI > In addition, the first video conferencing system 100 may generate a first conference screen based on the received video of the conference person, and may transmit the first conference screen to the unified platform server 120.

The second videoconference system 110 may receive images for the delegate from one or more videoconference devices belonging to the second videoconference system 110. [ In addition, the second video conference system 110 can generate a second conference screen based on the received video image of the conference person, and transmit the second conference screen to the unified platform server 120.

Here, the first video conferencing system 100 and the second video conferencing system 110 may be systems in which two or more MCUs (Multipoint Control Units) exist. An MCU is a device or software solution that acts as a bridge between video conferencing systems and is responsible for the mixing and distribution of video and audio.

The integrated platform server 120 may receive the first conference screen from the first video conferencing system 100 and the second conference screen from the second video conferencing system 110. [

If the conference originating the conference between the first video conferencing system 100 and the second video conferencing system 110 belongs to the first video conferencing system 100, Can operate. Thereafter, in merging the received conference screen, the integrated platform server 120 merges the reconfigured second conference screen based on the first conference screen received from the first video conferencing system 100, which is the reference system .

The integrated platform server 120 may detect one or more speakers from the second videoconference system 110 that is a non-reference system and reconstruct the second conference screen based on the detected speaker to generate the third conference screen.

In addition, the integrated platform server 120 may combine the first screen and the third screen to generate a fourth screen and transmit the screen to the first and second video conferencing systems 100 and 110 .

In general, the components of the heterogeneous video conferencing system of FIG. 1 are connected through a network 130. The network refers to a connection structure in which information can be exchanged between each node such as terminals and servers. Examples of such a network include a 3rd Generation Partnership Project (3GPP), a Long Term Evolution (LTE), a WIMAX World Interoperability for Microwave Access), Wi-Fi, 3G, 4G, and the like.

Hereinafter, the operation of each component of the heterogeneous video conferencing system of FIG. 1 will be described in more detail.

2 is a detailed configuration diagram of the integrated platform server 120 shown in FIG. 1 according to an embodiment of the present invention.

2, the integrated platform server 120 includes a receiving unit 200, a detecting unit 210, a screen reconstructing unit 220, an examining unit 230, a text converting unit 240, a transmitting unit 250, Not shown). However, the integrated platform server 120 shown in FIG. 2 is only one embodiment of the present invention, and various modifications are possible based on the components shown in FIG.

The receiving unit 200 can receive the first screen from the first video conferencing system 100 and receive the second screen from the second video conferencing system 110. [ Here, the first screen includes an image for one or more conference persons belonging to the first video conferencing system 100. The second conference screen includes an image of one or more conference participants belonging to the second video conference system 110.

On the other hand, it is assumed that a conference between the first video conferencing system 100 and the second video conferencing system 110 is established by a conference participant belonging to the first video conferencing system 100, Can operate as a reference system.

The detection unit 210 may detect one or more speakers from the second videoconference system 110, which is a non-reference system. In an embodiment of the present invention, the detecting unit 210 can detect a speaking person through a voice input to a plurality of microphone devices belonging to the second video conferencing system 110. For example, the detection unit 210 can convert the voice input to the microphone device into text, and recognize only the voice that has been successfully converted as a speaking person.

In addition, the detection unit 210 can detect a change of a speaking person from the second video conference system 110. [ Specifically, the detection unit 210 can confirm that the speaker has been changed each time the microphone device to which a voice is input is changed.

The screen reconstruction unit 220 can reconstruct the second screen based on the speaker detected from the second video conferencing system 110 to generate the third screen. Here, the third screen may be a screen reconstructed only with respect to the detected speaker. In an embodiment of the present invention, the screen reconstructing unit 220 may add a filter on the second screen image received from the second video conferencing system 110, and may use a focusing and zooming method Two screens can be reconstructed into an image for the detected speaker. Specifically, the screen reconstruction unit 220 can generate a layout as many as the number of detected speakers. Also, the screen reconstructing unit 220 can generate a third screen by disposing an image of the detected speaker in the generated layout. If two or more speakers are detected for a predetermined period of time, the screen reconstructing unit 220 may use the layout of the first detected speaker as the root, and may merge the layout of the detected speakers afterwards . For example, when two or more speakers simultaneously input a voice to a microphone device (a microphone device belonging to the second video conference system 110), the screen reconstructing unit 220 generates layouts for the number of utterances, You can merge them into one layout.

In another embodiment of the present invention, the screen reconstruction unit 220 can reconstruct a second screen by creating a plurality of layouts of the same size and overlapping the generated plurality of layouts with predetermined portions. For example, the screen reconstructing unit 220 can reconstruct the second screen by two-person layout, three-person layout, four-person layout, and the like.

In addition, when two speakers are detected, the screen reconstructing unit 220 inserts the video of the corresponding speaker into the layout for two players, and inserts the video of the speaker into the layout for three players when three speakers are detected.

On the other hand, the screen reconstructing unit 220 may generate the fourth screen by merging the first screen and the third screen.

When the change of the speaker is detected from the second video conferencing system 110, the screen reconstructing unit 220 reconstructs the second screen based on the changed speaker, generates the third screen again, The fourth screen can be generated again by merging the conference screen and the first conference screen.

In an embodiment of the present invention, the screen reconstructing unit 220 generates a layout every time a speaker changes, enlarges the image for the speaker about the center, and inserts an image for the speaker into the generated layout , The third screen can be generated.

The inspector 230 may check the activation status of a plurality of microphone devices belonging to the second video conferencing system 110. [ In addition, the inspector 230 may monitor the input of voice to a plurality of microphone devices belonging to the second video conferencing system 110.

The text conversion unit 240 can confirm whether the audio signals from the plurality of microphone devices belonging to the second video conferencing system 110 can be converted into text. At this time, the audio signal successfully converted into text can be recognized as the voice of the speaker.

The transmission unit 250 can transmit the fourth screen (a screen in which the first screen and the third screen are merged) to the first video conferencing system 100 and the second video conferencing system 110, respectively.

A DB (not shown) stores data input and output between the respective components in the integrated platform server 120, and inputs and outputs data between the components outside the integrated platform server 120 and the integrated platform server 120 Quot; For example, the DB (not shown) may store a first conference screen received from the first video conferencing system 100 and a second conference screen received from the second video conferencing system 110. One example of such a DB (not shown) includes a hard disk drive, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and a memory card existing in the integrated platform server 120 or the like.

Those skilled in the art will recognize that the receiving unit 200, the detecting unit 210, the screen reconstructing unit 220, the examining unit 230, the text converting unit 240, the transmitting unit 250, and the DB (not shown) Or one or more of these may be implemented in an integrated fashion.

FIG. 3 is a diagram illustrating a problem of a conference screen generated between existing heterogeneous video conferencing systems.

3, when the conference screens (the first screen 300 and the second conference screen 301) received from a plurality of video conferencing systems of different models are merged into one using the existing conference screen construction method, Can be complicated.

Suppose, for example, that three confer- ences from company A and four confer- ence from company B participate in the video conference. A second conference screen 300 composed of images of three conference persons 303 to 305 at company A and an image of four conference persons 306 to 309 at company B 301 is merged into one screen, any one of the first screen 300 or the second screen 301 becomes the reference screen, and the remaining conference screens are merged. If the first screen 300 is the reference and the second screen 301 is merged into the first screen 300, four (306 to 309) ) Is reduced to an image size of 1 person, and merged. In order to prevent this, each of the three delegates 303 to 305 in the first screen 300 can be merged into the second screen 301, but in this case, the preparation time for the conference becomes long.

In order to solve this problem, the present invention considers the second screen 301 as a single logistically, and the layout of the second screen 301 as a user basis. At this time, instead of displaying all of the conference participants (306 to 309) in the second conference screen 301 on one screen, only the screen of the talking person is displayed, and the screen of the remaining conference person is displayed in a hidden state Make it invisible. That is, only the screen of the conference person who inputs voice to the microphone device is displayed among the conference participants (306 to 309).

4A and 4B are views showing a conference screen between heterogeneous video conferencing systems that solve the problem of FIG. 3, according to an embodiment of the present invention.

Referring to FIGS. 4A and 4B, the integrated platform server 120 may set a first screen 400 received from the first video conferencing system 100 as a reference screen.

The integrated platform server 120 receives the second conference screen 401 from the second video conference system 110 and detects a speaker who inputs a voice to a plurality of microphone devices belonging to the second video conference system 110 .

The integrated platform server 120 may use a focusing and zoom in method to configure an image of the detected speaker as a third screen (not shown). For example, as shown in FIG. 4A, when one detected speaker 408 among the conference persons belonging to the second conference screen 401 is one, the third conference screen (not shown) is displayed on one detected speaker 408 Can be reconstructed into a Korean image. As another example, if there are at least two detected speakers 408 and 409 out of the speakers belonging to the second conference screen 401 as shown in FIG. 4B, the third screen (not shown) 408 and 409). ≪ / RTI >

The integrated platform server 120 generates a fourth screen 402 by merging the first screen 400 and the third screen (not shown), and the first video conference system 100 and the second video conference To the system 110.

5 is a flowchart illustrating a method of configuring a conference screen among heterogeneous video conferencing systems according to an exemplary embodiment of the present invention.

The method of configuring a conference screen between heterogeneous video conferencing systems according to the embodiment shown in FIG. 5 includes a first video conferencing system 100, a second video conferencing system 110, and a second video conferencing system 100 according to the embodiment shown in FIGS. And processed in a time-series manner in the integrated platform server 120. [ Therefore, the contents described with respect to the first video conferencing system 100, the second video conferencing system 110, and the integrated platform server 120 of FIGS. 1 through 4B, The present invention can also be applied to a method of constructing a conference screen between heterogeneous video conferencing systems according to an embodiment.

Referring to FIG. 5, in step S501, the integrated platform server 120 may receive a first conference screen from the first video conferencing system 100. FIG.

In step S503, the integrated platform server 120 may receive a second conference screen from the second video conferencing system 110. [

In step S505, the integrated platform server 120 may detect one or more speakers from the second videoconference system 110. [

In step S507, the integrated platform server 120 can reconstruct the second conference screen based on the speaker detected in step S505 to generate the third conference screen. In this case, the third screen may be a screen composed of images of the speakers detected in step S505.

In step S509, the integrated platform server 120 may generate the fourth screen by merging the first and third screens.

Although not shown in FIG. 5, in step S505, the integrated platform server 120 may check the activation state of a plurality of microphone devices belonging to the second video conferencing system 110. FIG.

Although not shown in FIG. 5, in step S505, the integrated platform server 120 may monitor input of voice to a plurality of microphone devices belonging to the second video conferencing system 110. FIG.

Although not shown in FIG. 5, in step S505, the unified platform server 120 can confirm whether an audio signal from a plurality of microphone devices belonging to the second video conferencing system 110 can be converted into text.

Although not shown in FIG. 5, in step S507, the integrated platform server 120 may generate layouts as many as the number of the speakers detected in step S505.

Although not shown in FIG. 5, the method of configuring the conference screen may further include a step (not shown) of detecting a change of a speaker from the second video conference system 110 after step S509.

Although not shown in FIG. 5, the method of configuring a conference screen may further include a step (not shown) of reconstructing a second conference screen based on the changed speaker after step S509 to regenerate a third conference screen .

Although not shown in FIG. 5, the method of configuring a conference screen includes a step (not shown) of transmitting a fourth conference screen to the first video conference system 100 and the second video conference system 110 after step S509 .

In the above description, steps S501 to S509 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed.

One embodiment of the present invention may also be embodied in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium can include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

It is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents. .

100: First video conference system
110: Second video conference system
120: Integrated Platform Server

Claims (20)

A method of configuring a conference screen between heterogeneous video conference systems,
Receiving a first conference screen from a first video conference system;
Receiving a second conference screen from a second video conference system;
Detecting one or more speakers from the second videoconference system;
Reconstructing the second conference screen based on the detected speaker to generate a third conference screen; And
Generating a fourth conference screen by merging the first conference screen and the third conference screen
The method comprising the steps of:
The method according to claim 1,
The method of claim 1,
And checking the activation status of the plurality of microphone devices belonging to the second video conferencing system.
3. The method of claim 2,
The method of claim 1,
And monitoring voice input to a plurality of microphone devices belonging to the second video conferencing system.
The method according to claim 1,
The method of claim 1,
And confirming whether or not audio signals from a plurality of microphone devices belonging to the second video conferencing system can be converted into text.
The method according to claim 1,
Detecting a change of a speaking person from the second videoconference system;
Reconstructing the second conference screen based on the changed speaker and regenerating the third conference screen
Further comprising the steps of:
The method according to claim 1,
Wherein a conference between the first video conferencing system and the second video conferencing system is established by a conference member belonging to the first video conferencing system
Wherein the first video conferencing system operates as a reference system.
The method according to claim 1,
Transmitting the fourth screen to the first video conferencing system and the second video conferencing system
Further comprising the steps of:
The method according to claim 1,
Wherein the generating the third screen comprises:
And generating a layout for the number of detected speakers.
The method according to claim 1,
Wherein the third screen is a screen composed of an image of the detected speaker.
In an integrated platform server between heterogeneous video conferencing systems,
A receiving unit that receives a first screen from the first video conferencing system and receives a second screen from the second video conferencing system;
A detector for detecting one or more speakers from the second video conference system; And
A screen reconstruction unit for reconstructing the second conference screen based on the detected speaker to generate a third conference screen and generating a fourth conference screen by merging the first conference screen and the third conference screen,
And an integrated platform server.
11. The method of claim 10,
Further comprising: an inspection unit for checking an activation state of a plurality of microphone devices belonging to said second video conferencing system.
12. The method of claim 11,
Wherein,
And monitors the input of voice to the plurality of microphone devices.
11. The method of claim 10,
Further comprising: a text conversion unit for confirming whether or not audio signals from a plurality of microphone devices belonging to the second video conferencing system can be converted into text.
11. The method of claim 10,
Wherein the detecting unit detects a change of a speaking person from the second video conference system,
Wherein the screen reconstruction unit reconstructs the second conference screen based on the changed speaker and regenerates the third conference screen.
11. The method of claim 10,
And a transmitting unit for transmitting the fourth screen to the first video conferencing system and the second video conferencing system.
In a heterogeneous video conferencing system,
A first video conference system for transmitting a first conference screen to an integrated platform server and receiving a fourth conference screen from the integrated platform server;
A second video conference system for transmitting a second conference screen to the integrated platform server and receiving the fourth conference screen from the integrated platform server; And
Receiving the first conference screen from the first conference system, receiving the second conference screen from the second conference system, detecting one or more speakers from the second conference system, And the second conference screen is reconstructed to generate a third conference screen, and the fourth conference screen is generated by merging the first conference screen and the third conference screen,
Wherein the video conference system comprises:
17. The method of claim 16,
The integrated platform server comprises:
And checks the activation state of the plurality of microphone devices belonging to the second video conferencing system.
17. The method of claim 16,
The integrated platform server comprises:
And verifies whether or not an audio signal from a plurality of microphone devices belonging to the second video conferencing system can be converted into text.
19. The method of claim 18,
The integrated platform server comprises:
And detects the change of the speaker from the second video conference system and reconstructs the second conference screen based on the changed speaker to regenerate the third conference screen.
17. The method of claim 16,
The integrated platform server comprises:
And transmits the fourth screen to the first video conferencing system and the second video conferencing system.

Images