JP2003134487A - Multi-spot conference system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To dynamically change the layout of a combined image to be displayed on each conference terminal. SOLUTION: A video combining means 106 of a server device 100 holds an XML file that defines the picture-combined layout of combined video to be distributed to each terminal 200, and the picture-combined layout of the XML file can be selected at an optional time such as a time during a conference and a time before the start of a conference in the each terminal. The video combining means follows the defined file to combine the videos of conference participants with a layout set in the each terminal, and distributes the video to the each terminal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multipoint conferencing system for transmitting video, audio, etc. for conducting a conference between multipoints in a remote place, and particularly in the case of laying out and synthesizing images of conference participants. Regarding layout.

[0002]

2. Description of the Related Art Generally, in this type of conference system, when the controller side synthesizes each video sent from a plurality of conference terminals on one screen and distributes it to each conference terminal, for example, The image is 2 in the first window
The video of the second conference terminal is laid out in the second window as in, and distributed to each conference terminal. In addition, each conference terminal displays the composite image sent from the controller side in the same layout.

[0003]

However, in the above-mentioned conventional conference system, since the layout of the composite image displayed on each conference terminal is the same and cannot be dynamically changed, each conference participant can obtain a desired image. There is a problem that the composite image cannot be seen in the layout.

In view of the above-mentioned problems of the conventional example, the present invention can dynamically change the layout of the composite image displayed on each conference terminal, so that each conference participant can see the composite image in the desired layout. It is an object of the present invention to provide a multipoint conference system capable of performing.

[0005]

In order to achieve the above object, the present invention is configured such that a plurality of conference terminals and a conference server device are connected via a network, and the conference server device receives from the plurality of conference terminals. A multipoint conferencing system for synthesizing video and audio to deliver to the plurality of conference terminals, wherein the conference server device stores layout information of a composite video for each of the plurality of conference terminals, and the layout information. The composite video of the layout is generated for each conference terminal based on the above, and is delivered to each of the plurality of conference terminals.

With the above configuration, the layout of the composite image displayed on each conference terminal can be dynamically changed, and each conference participant can view the composite image in a desired layout.

According to the present invention, each of the plurality of conference terminals sends a request for changing the layout information of the composite video to the conference server device, and the conference server device performs the plurality of requests based on the change request. The layout information of the composite video defined for each of the conference terminals is changed.

With the above arrangement, the layout of the composite image displayed on each conference terminal can be dynamically changed, and each conference participant can see the composite image in the desired layout.

According to the present invention, a plurality of conference terminals and a conference server device are connected via a network, and the conference server device synthesizes the video and audio of the conference participants received from the plurality of conference terminals to synthesize the plurality of conference terminals. Is a multipoint conference system for delivering to a conference terminal of
The composite video having the same layout is generated and delivered to the plurality of conference terminals, the plurality of conference terminals store layout information of the composite video that can be defined for each, and the conference based on the layout information. The layout of the composite video distributed from the server device is changed.

With the above configuration, the layout of the composite image displayed on each conference terminal can be dynamically changed, and each conference participant can view the composite image in a desired layout.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a multipoint conference system according to the present invention.

In the system shown in FIG. 1, as an example, a maximum of eight multipoint conference client terminals (hereinafter also simply referred to as "terminals") with respect to a multipoint conference server apparatus (also simply referred to as a conference server apparatus or server apparatus) 100. 200 is LA
It is connected via the N / WAN 300. Server device 10
0 is a hard disk (HD) as a mass storage device 1
20 are provided.

The LAN / WAN 300 can be connected to WWW servers 400 around the world, and the conference server device 100 is also a WWW server.

FIG. 2 shows an operation in which the terminal 200 accesses the WEB material on the WWW server 400 (and further on the conference server device 100) in the system shown in FIG.
The terminal 200 has a browser. Then, the conference server device 100 displays the WEB material name and the UR on the WWW server 400.
It has a WEB material database for correspondingly registering L addresses.

The WEB material database further includes
Your WEB material is registered. The WEB material is a text file or a file including images such as JPEG, GIF, and BMP.

The procedure (1) in FIG. 2 is the WWW server 4
The procedure to "see" the WEB material on 00 is shown, and the procedure (2)
Indicates the procedure of "showing". (1) When the client of the terminal 200a on the viewing side selects the WEB material name on the WWW server 400 and operates the "view" button to send the WEB material name to the conference server device 100, the conference server Device 100 responds by sending its URL address. As a result, the terminal 200a can access the WEB material on the WWW server 400.

(2) The client of the terminal 200a on the showing side selects the URL address of the WEB material on the WWW server 400 and operates the "I want to show" button to the client of the terminal 200b on the other side to be shown, and
Sends the URL address to the conference server device 100, the conference server device 100 displays the terminal 2 on the visible side.
The URL address is sent to 00b. As a result, the terminal 200b on the visible side is displayed on the WWW server 400.
Since the above WEB material can be accessed, the client of the terminal 200a can show the WEB material to be displayed to the client of the other terminal 200b. In this way, the conference server device 100 is not burdened while viewing (accessing) the WEB material on the WWW server 400.

The server device 100 will be described in detail with reference to FIG. This server device 100 is a communication control means 10 composed of software and hardware.
1, user authentication means 102, reservation management means 103, server control means 104, character synthesis means 105, video synthesis means 106, audio format conversion means 107, speech synthesis means 108, WWW server means 111, storage means 113,
And a load monitoring control means 115, a user management database 109, a reservation management database 110, a WEB material database 112, a video database 114, and a screen synthesis definition database 116 which are configured on the HD 120.

The communication control means 101 is a client terminal 2
As data / control command 1 received from 00, audio / video data, user authentication data, reservation registration information, terminal information, video material data, WEB material data, URL data, enlarged video request command and other control data are received, As the data / control command 2 to be transmitted to the client terminal 200, audio / video data, HT
It sends ML data, conference information data, user authentication result notification, and other control data.

Communication control means 101 and user authentication means 10
User authentication data and user authentication result notification between 2
Are exchanged, the communication control means 101 and the reservation management means 1
Reservation information data 4 is exchanged with 03. The control data 5 is exchanged between the communication control means 101 and the server control means 104, and the character information data 6 is sent from the communication control means 101 to the character synthesis means 105.
Video data and composite video data 7 are exchanged between the communication control means 101 and the video composition means 106, and the character video data 10 is sent from the character composition means 105 to the video composition means 106. The communication control means 101 sends the GSM voice data 8 to the voice format conversion means 107, and the voice synthesis means 108 sends the synthesized GSM voice data 9 to the communication control means 101. The PCM synthesized voice data and the GSM synthesized voice data 11 are exchanged between the voice format conversion means 107 and the voice synthesis means 108.

User authentication data 12 is exchanged between the user authentication means 102 and the user management database 109, and the reservation management means 103 and the reservation management database 1 are exchanged.
Reservation information data 13 is exchanged with 10.
HTTP request data and HTTP response data 14 are exchanged between the server control means 104 and the WWW server means 111, and voice synthesis control data 15 is exchanged between the server control means 104 and the voice synthesis means 108. The video synthesis control data 16 is sent from the server control means 104 to the video synthesis means 106. The storage control data and the read control data 17 are exchanged between the server control means 104 and the storage means 113, and the stored video data and the read video data 18 are exchanged between the video composition means 106 and the storage means 113. The accumulated voice data and the read voice data 19 are exchanged between the voice synthesis unit 108 and the storage unit 113.

WEB material and WEB material data 20 are exchanged between the WWW server means 111 and the WEB material database 112, and video / audio data and video material data 21 are exchanged between the accumulating means 113 and the video database 114. It Further, WEB materials and WEB material data 20 are exchanged between the storage means 113 and the WEB material database 112. The screen synthesis definition data 24 is exchanged between the video synthesis means 106 and the screen synthesis definition database 116. The load monitoring control means 115 exchanges load monitoring packet data and load monitoring response packet data 2 with the LAN / WAN 300.
2 is exchanged to send the rate control instruction data 23 to the communication control means 101.

The basic functions of the server device 100 will be described. -Proxy function: WWW server function. LAN / WAN3
It is determined whether the request from 00 is an HTTP request for a multipoint conference or a normal Internet access request, and in the latter case, the request is delivered to the Internet proxy. -User authentication function: Judges whether the user is registered in advance. -Video composition / distribution function: The screen composition definition database 11 stores the images of the conference participants received from all terminals 200.
6 are combined into one sheet with a layout defined for each terminal 200 and distributed to each terminal 200. Each terminal 20
The layout information defined for each 0 is used for each terminal 2
It can be changed by a change request from 00. -Voice synthesis / delivery function: The voices from each terminal 200 are mixed in a designated combination and delivered to all terminals 200. Character synthesizing function: Slice data is synthesized in the lower part of the image from each terminal 200 to synthesize one frame.

Recording function: Video / audio from each terminal 200 is recorded in the HD 120. Recording / reproducing function: Reproduces video / audio data recorded in the HD 120 and distributes it to each terminal 200. If it is a download, it is converted into the MPEG-1 format and downloaded. Rate control function: investigates the network bandwidth to which the terminal 200 is connected, and adopts a video bit rate and an audio encoding method according to the bandwidth.

Next, detailed functions will be described. <Meeting reservation> -Meeting registration: A function to reserve and register a meeting to be held. The contents registered at the time of reservation are date / time / held conference name / comment / participating user authentication setting. -Delete meeting: A function to delete the contents of a meeting reservation that has already been registered for reservation. -Meeting registration content list display: A function to display the already registered meeting content as a content list for the specified date. -Conference participation authentication: A function of determining whether or not a user is registered in the conference when the user participates in the conference, and rejecting the conference participation when the user is not registered.

<WEB material> Web material registration: A function of registering a name associated with a URL address used as a WEB material as a "WEB material name".・ WEB material deletion: A function to delete the already registered WEB material name. -WEB material list display: a function of displaying a list of WEB material names registered in the server apparatus 100.・ WEB material display: Select the registered WEB material name and WEB of the URL address associated with the WEB material name
A function to display page information. If it is during a meeting, the image display area is reduced and displayed. -Reflect WEB material: A function to reflect the content of the WEB material currently displayed to other conference participants.

<Video material> -Video material registration: A video recorded in a DV (abbreviation of digital video: the same hereinafter) camera is converted into MPEG-1 and registered on the server 100. -Delete video material: Delete the registered video material.・ Video material preview (Play / Pause / Fast forward / Rewind): Plays / Pauses / plays back the already registered video material.
Fast forward and rewind function. -Video material list display: A function to display a list of already registered video materials. -View video material (play / pause / fast forward / rewind): A function that allows registered video material to be composited during a meeting and distributed to be viewed by one person. -Show video material: A function to combine registered video materials during the conference and distribute them to everyone.

<Minutes> ・ Recording minutes: video / audio, video materials, W during the meeting
A function of recording the EB material on the server device 100. -Minutes download: A function of downloading the minutes registered in the server device 100 in the HTML format onto the recording medium of the terminal 200. -Browsing minutes: A function of distributing minutes registered on the server device 100 while receiving rate control. -Meeting list display / deletion: A function of displaying a list of registered minutes on the server 100 and deleting them. -Record minutes reproduction (WEB material reproduction): A function to refer to the downloaded minutes on the WEB browser.

<Display> • Conference participant name display: A function for composing the participant name in a part of the video display area. -Meeting information display: A function to display the information of the currently held meeting (meeting name / date / time / host name / participant name).・ Image enlargement / reduction

Next, the terminal 200 will be described in detail with reference to FIG. This terminal 200 has a voice input / output means 20.
1, video input means 202, WWW display means 203, operation input means 204, audio format conversion means 205, video format conversion means 206, WWW control means 20
7, audio codec control means 208, video codec control means 209, terminal control means 210, storage means 211,
It has a recording medium 212 and a communication control means 213, and the communication control means 213 is connected to the LAN / WAN 300 (see FIGS. 1 and 3).

The audio input / output unit 201 sends the PCM audio input signal 31 to the audio format conversion unit 205, and the audio codec control unit 208 sends the audio output signal 32 to the audio input / output unit 201. The DV video signal 33 is sent from the video input unit 202 to the video format conversion unit 206. WWW control means 207 to WW
The HTML file control information 34 is sent to the W display means 203, and the WWW display means 203 sends the WWW control means 20.
The CGI form information 35a is sent to 7. The operation event data 35b is sent from the operation input means 204 to the WWW display means 203.

The audio format conversion means 205 sends the PCM down-sampled audio data 36 to the audio codec control means 208, and the video format conversion means 206 sends the DV video data 37 to the video codec control means 209. Video codec control means 209
From the WWW control means 207, MPEG-1 video data 38 is sent. The WWW control means 207 sends the control data 39 to the terminal control means 210, and the control data 40 is exchanged between the terminal control means 210 and the storage means 211. Video / audio data, video material, and WEB material data (HTML) 41 are exchanged between the storage means 211 and the recording medium 212.

The voice codec control means 208 sends the GSM voice data 42 to the communication control means 213, and the communication control means 213 sends the GSM voice data 43 to the voice codec control means 208. The MPEG-1 video data 44 is sent from the video codec control means 209 to the communication control means 213, and the MPEG-1 video data 4 is sent from the communication control means 213 to the video codec control means 209.
5 is sent. Terminal control means 210 and communication control means 21
The control data 46 is exchanged with the device 3.

Next, the main functions of the terminal 200 will be described. -Voice data distribution: A function of capturing sound from a DV camera / microphone input and distributing the sound data to the server device.
(Audio input PCM data from DV camera / microphone input is downsampled, converted to GSM, packetized in GSM frame units, and delivered by HTTP protocol.) Audio playback: -Video data delivery: DV camera / USB camera A function to capture video from and distribute the video data to the server device.
(The DV format video data from the DV camera / USB camera is converted to MPEG-1, packetized in MPEG-1 frame units, and delivered by the HTTP protocol.)

Next, the display screen of the terminal 200 will be described with reference to FIGS. <Menu screen> On the menu screen, as shown in FIG.
"Meeting reservation screen display button", "Video material registration screen display button", "WEB material registration screen display button", "Meeting minutes browsing download screen display button", "Button to join the currently held meeting", "Current meeting information display area", "Current meeting status display area", "Current meeting information request button", "Today's scheduled meeting information" Area for displaying "is displayed. <Reservation screen> When the "conference reservation screen display button" on the menu screen shown in FIG. 5 is operated, a conference reservation screen as shown in FIG. 6 is displayed, and a conference can be reserved from this conference reservation screen. .

<Conference screen> When the "meeting start button" on the menu screen shown in FIG. 5 is operated, a conference screen as shown in FIG. 7 is displayed. In this meeting screen, "Meeting information display area", "Video material display area", "View" button and "Show" button of the selected video material, "Registered WEB material list display area", selected WEB material A “view” button, a “recording minutes start / stop button”, a “menu” button for returning to the menu screen, and a “meeting image display area” are provided. As shown in detail in FIG. 8, the “meeting image display area” is 4 × 2 in the lower half.
= 8 windows are divided, and each divided window displays the video of each of the eight conference participants. In addition, the upper half is divided into 2 × 1 = 2 windows, the right side window displays the video material that you want to see or is shown to other participants, and the left side window displays the image. An enlarged image of the material is displayed.

<Conference screen (display of video material)> When the "view" button and the "show" button of the selected video material on the meeting screen shown in FIG. 7 are operated, the video material is displayed as shown in FIG. The meeting screen of is displayed. On this conference screen, the video material is displayed in the "conference video display area", and in addition to the display of the conference screen shown in FIG. 7, "Play / pause / stop / slider / end video material" is displayed. Each button of is displayed. <Conference screen (WEB display)> On the conference screen shown in FIG.
When the "View" button of the selected WEB material is operated,
A WEB display conference screen as shown in FIG. 10 is displayed. In this conference screen, in addition to the display of the conference screen shown in FIG. 7, a “WEB page display area”, a reduced “conference image display area”, a “URL input field”, a “WEB page operation button”, and a display A "show" button for displaying the inside WEB page to another terminal and a "end WEB material display" button are displayed.

<Video material registration screen (during preview)> On the video material registration screen during preview, as shown in FIG. 11, the video material list registered on the server 100, the video material list on the terminal itself, and external recording. List of video materials in the device, move / delete button of selected video material, "to menu"
Buttons, video material playback display area, slider bar, video material playback / pause / stop button, video material name input field, etc. are displayed.

<WEB material registration screen> On the WEB material registration screen, as shown in FIG. 12, a WEB material name input field to be registered, an inputted WEB material name registration button, a WEB material name display list already registered, WEB material Display / delete button from list, “to menu” button, URL address input field, WEB page display area, WEB operation button and the like are displayed.

<Meeting minutes browsing screen (download)>
Figure 1 is displayed on the conference minutes browsing screen for downloading.
As shown in 3, a “menu” button, a minutes list display area, a download / view / delete button from the minutes list, a download destination designation area, and the like are displayed. <Meeting minutes browsing screen> Figure 14 on the meeting minutes browsing screen
As shown in, the meeting information display area, video / WE
B Material display mode switching button, slider bar, minutes play / pause / stop button, button to browse and download screen, minutes video display area, etc. are displayed.

<Meeting minutes browsing screen (WEB display)> W
As shown in FIG. 15, in the conference minutes browsing screen of the EB display, in addition to the display shown in FIG. 14, a URL address input field, a WEB page display area (instead of the minutes video display area), and a reduced minutes A video display area, WEB page operation buttons, etc. are displayed.

<Minutes> Next, with reference to FIG. 16, a process of starting the minutes recording of the server device 100 will be described. First, the terminal 200 transmits a minutes record recording start command to the server device 100 when the “minutes record start” button is operated.
In the server device 100, the communication control unit 101 has the terminal 20.
When a command (minutes record start command) is received from 0, the server controller 104 is instructed to analyze the command (step S1), and when the server controller 104 analyzes the received command as the minutes record start command, the minutes are recorded. A recording instruction is issued to the storage means 113 (step S2).
Then, the accumulating means 113 starts recording the video / audio during the conference in the video database 114 on the HD 120 as the minutes (step S3). Here, the video / audio recorded as the minutes is not all the video / audio received from the terminal 200 in order to save the capacity of the HD 120, but only the video / audio distributed to the terminal 200.

<Image Material Registration> Next, with reference to FIG. 17, the image material registration process of the server device 100 will be described. First, the terminal 200 transmits a video material registration command to the server device 100 when the “video material registration” button is operated. In the server device 100, when the communication control means 101 receives a command (video material registration command) from the terminal 200, it instructs the server control means 104 to analyze the command (step S21), and the server control means 104 images the received command. When analyzed with the material registration command, an instruction to write the video material is issued to the storage means 113 (step S22). Then, the storage unit 113 records the video material designated by the video material registration command in the video database 114 on the HD 120, and records the management data thereof in the user management database 109 (step S23).

<Video Material Reproduction> Next, with reference to FIG. 18, the processing of the video material reproduction of the server apparatus 100 will be described. First, when the “view” button is operated, the terminal 200 transmits a video material reproduction command to the server device 100. In the server device 100, when the communication control means 101 receives a command (video material reproduction command) from the terminal 200, it instructs the server control means 104 to analyze the command (step S31), and the server control means 104 images the received command. When analyzed with the material reproduction command, an access right judgment instruction is issued to the server control means 104 (step S32),
The server control means 104 determines the access right of the user (step S33).

Then, in the case of "access right",
The accumulation means 113 reproduces the video material designated by the video material reproduction start command from the video database 114 with reference to the user management database 109 (step S3).
4 → S35), and then the reproduced video / audio is synthesized by the video synthesizing means 106 and the voice synthesizing means 108, respectively (step S36), and then the communication control means 101
Is delivered to the terminal 200 (step S37). On the other hand, in the case of “no access right”, the server control means 1
04 generates a transmission command of “no access right” (steps S34 → S38), and then transmits this to the communication control means 1
01 is delivered to the terminal 200 (step S39).

<WEB Material Registration> Next, referring to FIG. 19, the web material registration processing of the server 100 will be described.
First, when the "WEB material registration" button is operated, the terminal 200 transmits a WEB material registration command to the server device 100. In the server device 100, when the communication control means 101 receives a command (WEB material registration command) from the terminal 200, it instructs the server control means 104 to analyze the command (step S71), and the server control means 104.
When the received command is analyzed by the WEB material registration command, the storing means 11 stores the write instruction of the WEB material management table.
3 (step S72). And storage means 1
13 creates a WEB material management table and records it in the WEB material database 112 on the HD 120 (step S73).

<WEB Material Event Registration> Next, referring to FIG. 20, the recording processing of the WEB material event of the server 100 will be described. First, when the “WEB material recording” button is operated, the terminal 200 transmits a WEB material recording control command to the server device 100. In the server device 100, the communication control unit 101 sends a command (W
When receiving the EB material recording control command), the server control unit 104 is instructed to analyze the command (step S41), and the server control unit 104 sends the received command to the WE.
When analyzed with the B material recording control command, a recording instruction of the URL of the video material being displayed is issued to the accumulating means 113 (step S42). Then, the storage unit 113 stores the meeting information (meeting name / holding date / organizer name / participant name) as a WEB material event of the video material, the URL address of the displayed video material, and the elapsed time from the start of the meeting. HD12
0 is recorded in the WEB material database 114 (step S43).

<Showing WEB Material> Next, the WEB material event notification process of the server 100 will be described with reference to FIG. First, when the “show” button is operated, the terminal 200 transmits a WEB material URL notification control command to the server device 100. In the server device 100, the communication control means 101 sends a command (WEB data U
When receiving the RL notification control command), it instructs the server control means 104 to analyze the command (step S5).
1) When the server control unit 104 analyzes the received command as the WEB material URL notification control command, it issues a URL notification command generation instruction (step S52).

Then, the server control means 104 generates a URL notification command and sends the transmission instruction to the communication control means 10.
1 (step S53), and then communication control means 1
01 transmits this URL notification command to another terminal 200 (step S54). As a result, the UR of the WEB material being displayed on the terminal 200 whose “show” button has been operated
Since the L address is displayed on the other terminal 200, the other terminal 200 uses this URL address for the WWW server 400.
(In addition, the WEB material of the server device 100) can be accessed and displayed.

<Reproduction of Minutes> Next, the minutes reproduction processing of the server device 100 will be described with reference to FIG. First, when the “minutes reproduction” button is operated, the terminal 200 transmits a minutes reproduction command to the server device 100. In the server device 100, when the communication control means 101 receives a command (minutes reproduction command) from the terminal 200, it instructs the server control means 104 to analyze the command (step S
61), when the server control means 104 analyzes the received command with the minutes reproduction command, it issues an instruction to read the minutes and WEB material event to the storage means 113 (step S62).

Then, the storage means 113 reads the minutes and the WEB material event from the HD 120 (step S6).
3) Then, the server control means 104 determines whether or not the elapsed time (reference time) of the WEB material event is reached (step S64). Then, in the case of the elapsed time (reference time), the server control means 104 generates a URL notification command and issues the distribution instruction to the communication control means 101 (step S65), and then the communication control means 101 sends this URL notification command. Is transmitted to the terminal 200 (step S66), and then the process proceeds to step S67. If it is not the elapsed time (reference time) in step S64, the process directly proceeds to step S67.
Proceed to. In step S67, the video and audio of the minutes are synthesized by the video synthesizing means 106 and the voice synthesizing means 107, respectively, and the distribution instruction is issued to the communication control means 101, and then the synthesis is performed while the communication control means 101 performs rate control. The selected video and audio are transmitted to the terminal 200 (step S68).

Therefore, when a conference participant reproduces the minutes after the conference has ended, the URL address is notified when the video material is displayed during the conference and registered as the WEB material. The WEB material can be reproduced and displayed by instructing reproduction by the address. Note that instead of transmitting the URL notification command to the terminal 200 in steps S65 and S66, the WEB
The material may be automatically reproduced and displayed based on the URL address.

Here, the screen compositing definition database 116 of the server device 100 holds an XML file that defines the screen compositing layout of the composite video distributed to each terminal 200, and the screen compositing layout of this XML file is the terminal 200. It can be selected at any time, such as during the meeting or before the start of the meeting. Then, the image synthesizing unit 106 synthesizes the images of the conference participants in the layout set for each terminal 200 according to the definition file.

The layout process will be described with reference to FIG. First, when the “start conference” button is operated, the terminal 200 transmits a conference start command to the server device 100.
In the server device 100, the communication control unit 101 has the terminal 20.
When a command (conference start command) is received from 0, the server control unit 104 is instructed to analyze the command (step S91). It is issued to the communication control means 101 (step S92).

When the communication control means 101 receives the instruction to receive the image for synthesis, it instructs the image synthesis means 106 to read the image synthesis layout data (step S9).
3) Next, the video synthesizing means 106 reads the video synthesizing layout data for each terminal 200 and synthesizes the images for synthesizing with the layout (steps S94, S95), and then the communication control means 101 sends the synthesized image to each terminal 200. It is transmitted (step S96).

Therefore, according to the layout method described above, the video synthesizing means 106 of the server apparatus 100 causes each terminal 2 to operate.
Since the XML file defining the screen synthesis layout of the synthesized video to be distributed to 00 is held and the screen synthesis layout of this XML file can be selected for each terminal 200, the layout of the synthesized image displayed on each terminal 200. Can be dynamically changed so that each conference participant can see the composite image in a desired layout.

It should be noted that instead of the server device 100 holding the above-mentioned screen synthesis layout XML file, each terminal 200 side (WWW control means 207) holds an XML file defining the screen synthesis layout of its own composite video. By doing so, the server device 100 may deliver the composite video of the same layout to all the terminals 200, and each terminal 200 side may change the layout of the composite video based on its own XML file.

Here, in FIG. 1, eight terminals 200 and 1 are connected.
Although only one server device 100 is shown in FIG.
As shown in FIG. 4, one conference can be held using any eight N terminals 200 and any one M server apparatus 100.

[0059]

As described above, according to the present invention, the layout of the composite image displayed on each conference terminal can be dynamically changed, and each conference participant can view the composite image in the desired layout. It is possible to provide a multipoint conference system capable of performing the above.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a multipoint conference system according to the present invention.

FIG. 2 is an explanatory diagram showing a procedure for accessing WEB materials in the system of FIG.

3 is a block diagram showing the server device of FIG. 1 in detail.

FIG. 4 is a block diagram showing the client terminal of FIG. 1 in detail.

5 is an explanatory diagram showing a menu screen of the terminal of FIG.

6 is an explanatory diagram showing a conference reservation screen of the terminal of FIG.

7 is an explanatory diagram showing a conference screen of the terminal of FIG.

FIG. 8 is an explanatory diagram showing in detail the conference video display area of FIG.

FIG. 9 is an explanatory diagram showing a conference screen (video material display) of the terminal of FIG.

FIG. 10 is an explanatory diagram showing a conference screen (WEB display) of the terminal of FIG.

11 is an explanatory diagram showing a video material registration screen (during preview) of the terminal of FIG.

FIG. 12 is an explanatory diagram showing a WEB material registration screen of the terminal of FIG.

FIG. 13 is an explanatory diagram showing a conference minutes browsing screen (download) of the terminal of FIG.

FIG. 14 is an explanatory diagram showing a conference minutes browsing screen of the terminal of FIG. 1.

15 is an explanatory diagram showing a conference minutes browsing screen (WEB display) of the terminal of FIG.

16 is a flowchart showing a process of recording minutes of the server device of FIG.

17 is a flowchart showing a process of registering video material in the server device of FIG.

FIG. 18 is a flowchart showing a process of reproducing video material by the server device of FIG.

19 is a flowchart showing a process of WEB material registration of the server device of FIG.

20 is a flowchart showing a process of recording a WEB material event in the server device of FIG.

FIG. 21 is a flowchart showing the processing of the WEB material event notification of the server device of FIG.

FIG. 22 is a flowchart showing a process of playing the minutes of the server device of FIG.

FIG. 23 is a flowchart showing a composition process of the server device of FIG.

24 is a configuration diagram showing an application example of the multipoint conference system in FIG.

[Explanation of symbols]

100 server device (multipoint conference server device) (conference server device) 112 WEB material database 114 video material database 120 hard disk (HD) 200 client terminal (multipoint conference client terminal) (conference terminal or simply terminal) 300 LAN / WAN 400 WWW server

Continued front page    (72) Inventor Yasuo Homoda             3-1, Tsunashima-Higashi 4-chome, Kohoku-ku, Yokohama-shi, Kanagawa             Matsushita Communication Industry Co., Ltd. F-term (reference) 5C064 AA02 AB03 AB04 AC04 AC14                       AC22 AD08                 5K015 AA10 AB01 AF05 JA03                 5K024 AA52 BB04 BB07 CC10 CC14                       DD01 DD04 FF04 GG01                 5K030 GA18 HB01 HB02 LD08                 5K101 KK07 LL02 MM06 MM07 NN06                       NN07 NN18 TT02 UU19

Claims (3)

[Claims] 1. A plurality of conference terminals and a conference server device are connected via a network, and the conference server device synthesizes video and audio received from the plurality of conference terminals and delivers them to the plurality of conference terminals. A multipoint conference system, wherein the conference server device stores layout information of a composite video for each of the plurality of conference terminals, and generates a composite video of a layout for each conference terminal based on the layout information. A multipoint conference system configured to deliver to each of the plurality of conference terminals. 2. A request for changing the layout information of the composite video is sent from each of the plurality of conference terminals to the conference server apparatus, and the conference server apparatus operates based on the change request. The multipoint conference system according to claim 1, wherein the layout information of the composite video defined for each is changed. 3. A plurality of conference terminals and a conference server device are connected via a network, and the conference server device synthesizes the video and audio of the conference participants received from the plurality of conference terminals to synthesize the plurality of conference terminals. Is a multipoint conference system, wherein the conference server device generates and distributes a composite video of the same layout to the plurality of conference terminals, and the plurality of conference terminals can be defined for each. A multipoint conference system configured to store layout information of a composite video and change the layout of the composite video distributed from the conference server device based on the layout information.