JP2001313915A - Video conference equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correspond to various uses by displaying to use an accumulated information video, the video in the past, etc., in a desired style with the video from a terminal, accumulating meeting histories, and displaying to read in the desired style. SOLUTION: A server 101 receives the video from the terminal 102, and an image synthesizing means 105 synthesizes the video from each terminal to return to the terminal 102 to display the synthesized video on a display means 113. The information video used for meeting is accumulated in a storing means 103. When the reproduction control command of the information video is received from the terminal 102, the command interpretation means 107 interprets it and sends a readout instruction to a readout control means 104. The image synthesizing means 105 synthesizes the information video read from the storing means 103 and the video from the terminal in one video and the synthesized video is sent out to the terminal 102.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference apparatus for transmitting a video signal and performing communication such as a video conference and a remote lecture.

[0002]

2. Description of the Related Art A conventional video conference system captures an image from a camera connected to each terminal, and transfers it to an MCU.
(Relay device), an image of each terminal is synthesized by the MCU, and the synthesized image is sent back to each terminal again. In such a system, a camera is used to take a picture of a user in front of each terminal, and each user has a meeting while watching a composite video including a video from another terminal returned from the MCU. Materials required for the meeting, topics of interest, etc.
An object is directly photographed by a camera called a shelf camera and transmitted to the MCU instead of the image of the user, or transmitted to the MCU as another image in parallel with the image of the user. Then, they discuss while watching the video of the camera that shows the material.

As a method of recording a conference using a video conference system in the minutes of a meeting, there is a method of recording events occurring during the conference in chronological order and reproducing the events. Here, the event is, for example, image information such as character input or paint generated on a whiteboard that can be written by the participant at the same time, provided for cooperative work.

[0004]

In the conventional video conference system, the processing means for synthesizing the video and the like is fixed, so that each terminal displays the synthesized video in a desired form or records the synthesized video in advance. It was not possible to use the video material that was placed and display it together with the current live video. As an indirect method when using video materials, there is a method of reproducing video materials on a terminal prepared separately from a terminal for a conference, and projecting a display of the terminal with a camera. However, in such a method, the image quality of the image is degraded, so that it is sometimes difficult to view the image material. In addition, video playback control such as video playback start and stop cannot be performed by a participant at another point. Also, in this case, a network bandwidth for transmitting the material video from the terminal to the server is required. Therefore, it is necessary to increase the line capacity or reduce the frame rate of other video, thereby increasing the scale of the apparatus and the quality of the moving image. Will be invited.

[0005] Conventionally, the history of a meeting is recorded as minutes as a video, and the minutes are then viewed or played back on the same device with the same operation specifications as the meeting without imposing a burden on the user. Could not. When recording the history of a meeting, it is necessary to synchronize and record the video between a plurality of points so that the user's remark does not come and go, but such recording and reproduction processing has been difficult. Further, in the reproduction of the minutes, there is a demand that the viewer wants to enlarge the display of the speaker, the person of interest, or the video of the material, or change the display layout arbitrarily. This is particularly necessary for use in distance lectures as well as video conferences.

[0006] Further, in the use as a video conference or a remote lecture, there is a demand that while viewing live images transmitted from terminals of speakers or instructors, it is desired to simultaneously arrange and view past images of the live images. For example, in a video conference, it is necessary to look back and confirm the assertion of the speaker, or in a remote lecture, to return to review the lecturer's explanation and its blackboard or OHP screen image several minutes ago. Conventionally, it has not been possible to simultaneously refer to the past video of the live video while watching the current live video. Also, in this case, since the demands such as the simultaneous reference of the live video and the past video and the manner of displaying the live video differ depending on the user, it is necessary to generate various forms of composite video corresponding to each user's request individually. However, such image processing was difficult.

[0007] The present invention has been made in view of the above circumstances, and has as its object to display and use stored material images, past images, and the like in a desired form together with images of other users. It is an object of the present invention to provide a video conference apparatus capable of coping with various usage situations, such as recording a video of a conference or a lecture and displaying and viewing the video in a desired form later.

[0008]

SUMMARY OF THE INVENTION The present invention is, first, a video conference apparatus for transmitting a video between one or more terminals and a server, wherein the server transmits the reference information including the material video to the server. Storage means for storing video data, read control means for reading video data stored in the storage means, reception means for receiving video transmitted from the terminal, and command interpretation means for interpreting commands from the terminal And video synthesizing means for synthesizing a plurality of video data including the video from the terminal into one video, and transmitting means for transmitting the synthesized video to the terminal. Thereby, in response to a command from the terminal, the video material stored in the storage means provided in the server in advance is read,
By synthesizing one image with the image synthesizing means together with the image from each terminal, it is possible to obtain a video conference device that can use the image as a material in a conference, a lecture, or the like.

Second, the terminal is provided with operation input means for issuing a command for synthesizing the video data of the material video only to the composite video intended for the terminal, and the video synthesizing means comprises a command from the terminal. Is received by the command interpreter, the video data of the material video read from the storage by the read controller is combined with only the video for the corresponding terminal.

Thirdly, the terminal is provided with operation input means for issuing a command for synthesizing the video data of the material video to the synthesized video for each terminal, and the video synthesizing means receives the command from the terminal. When the command data is received by the command interpreting means, the video data of the material video read from the storage means by the read control means is combined with the video for each terminal.

Fourth, a video conference apparatus for transmitting video between one or more terminals and a server, wherein the storage means for storing video data in the server; Receiving means for receiving, command interpreting means for interpreting a command from the terminal, video synthesizing means for synthesizing a plurality of video data including video from the terminal into one video, and video synthesized by the video synthesizing means And writing means for writing the video to the storage means, and transmitting means for transmitting the video synthesized by the video synthesis means to the terminal. Thus, in response to a command from the terminal, the video from each terminal is synthesized into one video by the synthesizing means, and this is stored in the storage means, so that the video from each terminal can be easily synchronized with each other. This has the effect of realizing a video conference device that can record the progress of a conference or lecture as a video of the conference history.

Fifthly, the terminal is provided with operation input means for issuing a command for recording a video from each terminal, and the writing control means receives the command from the terminal by the command interpreting means. In this case, the video data in which a plurality of videos including the video from the terminal are synthesized by the video synthesizing unit is written to the storage unit.

Sixth, a video conference apparatus for transmitting video between one or more terminals and a server, wherein the storage means for storing video data in the server, and a video transmitted from the terminal. Receiving means for receiving, command interpreting means for interpreting a command from the terminal, write control means for writing each video from the terminal to the storage means, and read control for reading video data stored in the storage means Means, video synthesizing means for synthesizing a plurality of video data including video from the terminal read from the storage means into one video, and sending means for transmitting the synthesized video to the terminal. And Thus, the video from each terminal is stored in the storage unit, and each stored video is read out according to a command from the terminal, and is combined into one video by the video synthesis unit. As a result, it is possible to realize a video conference apparatus capable of enlarging and displaying a specified video, changing a display position in a composite video, and recording and reproducing various conference history videos.

Seventh, there is provided a video conference apparatus for transmitting video between one or more terminals and a server, wherein the server stores storage means for storing video data, and stores the video transmitted from the terminal. Receiving means for receiving, command interpreting means for interpreting a command from the terminal, write control means for writing each video from the terminal to the storage means, and read control for reading video data stored in the storage means Means, video synthesizing means for synthesizing a plurality of video data including the video read from the storage means and the video from the terminal received by the receiving means into one video, and sending the synthesized video to the terminal Means. In this way, while the video from each terminal is stored in the storage means, the past video from the stored terminal is read in response to a command from the terminal, and the stored video and the current raw image received by the receiving means are read out. Of the current image from the terminal while the specified image is enlarged or displayed at the command of the user by changing the display position in the synthesized image, etc. The video conference apparatus capable of recording and reproducing the conference history video, which can simultaneously refer to the video and the past video of the video, is obtained.

Eighthly, the terminal is provided with operation input means for issuing a command for recording a video, and the writing control means, when a command from the terminal is received by the command interpreting means, The video data from each terminal received by the receiving means is written into the storage means.

Ninth, the terminal is provided with operation input means for issuing a command for reproducing the stored video,
The video synthesizing unit, when a command from the terminal is received by the command interpreting unit, converts a plurality of video data including the terminal video read from the storage unit by the read control unit as a video for the terminal. It is to be synthesized.

Tenth, the terminal is provided with operation input means for issuing a command for reproducing the stored video, and the video synthesizing means receives a command from the terminal by the command interpreting means. A plurality of video data including a past terminal video read from the storage unit by the read control unit and a current terminal video received by the receiving unit, as a video for the terminal. . Eleventh, the operation input means includes:
It has a slider for indicating the time of the video to be reproduced.

Twelfth, in the composite video displayed on the terminal, there is provided a graphical user interface for displaying a plurality of videos side by side and enlarging and displaying a specific video among them in an enlarged display area. It is a thing. Thirteenth, in the composite video displayed on the terminal, a plurality of videos are displayed side by side, and a specific video among them is enlarged and displayed in an enlarged display area. It has a graphical user interface to be displayed. Fourteenth, the operation input means is constituted by a graphical user interface represented on a display screen of a terminal.

Fifteenth, a video conference apparatus for transmitting a video between one or more terminals and a server, wherein a current video from the terminal, a stored material video, A graphical user interface for displaying a plurality of videos including at least one of the past videos recorded from the terminal side by side and enlarging and displaying a specific video among these videos in an enlarged display area It is characterized by having. For example, the current video is a video from all terminals participating in the conference, or any terminal, or a specific terminal, and the material video is used for the conference as reference information stored in, for example, storage means of a server. The past video is, for example, a recorded video of a predetermined terminal stored in the storage unit of the server. Sixteenth
A video conference apparatus for transmitting video between one or more terminals and a server, wherein a display screen on the terminal includes a current video from the terminal, a stored material video, and a recorded video from the recorded terminal. In addition to displaying a plurality of images including at least one of the past images side by side, a specific image in these images is enlarged and displayed in an enlarged display area, and the enlarged original image is highlighted and displayed. Graphical user
An interface is provided. Examples of the highlight display include a display example in which the original video is not displayed, a thick frame is added, and coloring is performed. Seventeenth,
A video conference device for transmitting video between one or more terminals and a server, wherein at least a past video from the recorded terminal is displayed on a display screen of the terminal,
It is characterized by having a graphical user interface displaying a slider for instructing the reproduction time of the video.

Eighteenth, the video data has been subjected to a compression process, and the compression format thereof is MPEG. Nineteenth, the video data has been subjected to compression processing, and the compression format is Motion.
It uses JPEG. Twentieth, the command interpreting means validates only a command from a predetermined terminal. Twenty-first, the command interpreting means validates commands from all terminals.

With these configurations, the operability is good for the user, and the video in a desired form can be easily displayed and viewed.

[0022]

Embodiments of the present invention will be described below with reference to the drawings. The video conference apparatus of the present invention is a video conference system in which a user in a remote place where a person cannot directly meet each other transmits a video and an audio using a terminal, and holds a conference among a plurality of participants. The present invention can be applied to various video transmission devices such as a remote lecture system for performing a remote lecture between them.

[First Embodiment] FIG. 1 is a functional block diagram showing a configuration of a video conference apparatus according to a first embodiment of the present invention. This teleconference device includes a server 101 and a terminal 1
02, and exchanges video signals, commands, and the like between the devices, and displays video and the like from another terminal on the terminal 102. The server 101 receives the video data captured and transmitted by the terminal 102, synthesizes the video,
Processing such as writing and reading is performed.

The server 101 includes a storage unit 103 for storing material video data or the like obtained by visualizing a material used for a conference or the like as reference information; a reading control unit 104 for reading data such as the material video data from the storage unit 103;
02, video receiving means 106 for receiving video data from the terminal 102, command interpreting means 107 for interpreting commands representing various operation instructions from the terminal 102, The transmission unit 108 is configured to transmit the video synthesized by the video synthesis unit 105 to the terminal 102.

The terminal 102 is obtained by a video camera 109 which is a photographing means for photographing the state of the terminal user, a command generating means 110 for issuing commands for various operation instructions to be transmitted to the server 101, and a video camera 109. A transmitting unit 111 for transmitting a video or a command from the command generating unit 110 to the server 101; a receiving unit 112 for receiving a composite video transmitted from the server 101; a CRT or a liquid crystal display (LCD) for displaying a video from the server 101; And a display means 113 composed of

In the present embodiment, one or more (especially two or more when used in a conference or the like) terminal 102 is connected to one server 1
01. In addition, it is also possible to provide two or more servers 101 to distribute processing. The server 101 or the terminal 102 may be a device having a dedicated processor for performing video signal processing or the like, or may be a device using a computer having a general-purpose processor. Read control means 104, video synthesizing means 1
05, the command interpreting means 107, and the like can also be realized by a software program. In this case, various operation modes can be executed according to operation setting changes, program rewriting, and the like.

In the terminal 102, the user is captured by the video camera 109, and the captured video is transmitted to the server 101 using the transmission unit 111. In the present embodiment, the sending unit 111 digitizes and compresses the video signal, and transmits the compressed video data to the server 101 via the transmission unit. MPEG, Motion JPEG, or the like is used as a compression format for video data. The server 101 and the terminal 102 are connected via a network using a public line or a dedicated line. The transmission of video data can be performed using a video conference protocol or a general-purpose network protocol (TCP / IP).
Etc.).

The command generating means 110 generates a command relating to the operation of the apparatus and the like,
To send to. The command generation unit 110 is, for example, the terminal 1
02, a material video selection button, a reproduction start button on a graphical user interface (hereinafter, referred to as a GUI) displayed on the screen of the display unit 113,
It corresponds to a playback stop button or the like. Command generation means 11
0 generates a command that is paired with each button when these buttons are pressed.

The operation input means using the GUI includes, for example, a means for providing a touch panel on the screen of the display means 113 and inputting an instruction by touching a portion corresponding to a button or the like displayed on the screen. A push button switch provided near the screen or a device for inputting an instruction by pressing a button corresponding to the screen display with a separate keyboard or remote control, and a screen displayed by providing a pointing device such as a mouse or a touch pad. A button or the like for inputting an instruction by instructing a button or the like with a pointer or a cursor can be used.

On the other hand, the server 101 receives the video data and the command transmitted from each terminal 102 by the receiving means 106.
To receive. Then, the received video from each terminal is synthesized into one video by the video synthesis means 105. In this video synthesis, a plurality of videos are combined into a single video by appropriately using various video signal processing techniques relating to digital video processing.

The method of synthesizing the video, that is, the selection of the video of the terminal to be synthesized, the display position, the display size, and the like may be fixed in the entire system, or may be determined by a command received from the terminal. Therefore, the combining method may be different for each terminal. At this time, the commands sent from the terminal 102 to the server 101 include, in addition to the above-mentioned commands, commands for specifying the video selection, display position, and display size of each terminal. Then, in the server 101, the video synthesizing unit 105
The sending means 108 sends back the video synthesized in step (1) to the terminal 102. Note that, in the server 101, before and after the process of synthesizing the video data, a process of restoring the compressed video data, a process of recompressing the synthesized video data, and the like are also performed. In the terminal 102, the video data of the composite video from the server 101 is received by the receiving unit 112 and displayed by the display unit 113.

Further, in the server 101, the terminal 102 which has received the command
Interpret commands from. At this time, if the command is to reproduce the material video stored in the storage unit 103, a read instruction is issued to the read control unit 104. The read control unit 104 that has received the read instruction reads the video data of the corresponding material video from the storage unit 103 and passes the video data to the video synthesis unit 105. The video synthesizing unit 105 synthesizes the material video read by the read control unit 104 together with the video from each terminal. The material video is video data used by the user as reference information at the time of a meeting or the like, and any video of a moving image and a still image can be used.

FIG. 2 shows a GUI displayed on the terminal 102.
FIG. 3 is a conceptual diagram showing a first example of FIG. At the left end of the display screen, a material video list 20 from which the user selects a material video is displayed.
1 and a participant list 202 in which the participants of the conference are displayed, and at the lower end, a reproduction control command such as "reproduction start", "pause", or "stop" for controlling reproduction of the material video. Control button 2 for performing operation input corresponding to
03 is displayed. Note that, in FIG. 2, a portion surrounded by a bold frame represents a combined image 204 combined by the server 101. Here, terminal images 205A, 205B, and 205C photographed at three terminals A, B, and C are arranged at a lower portion, and a material image 20 with a large display area is arranged at the upper portion.
6 shows an example in which 6 is arranged and synthesized and displayed.

As a method for the user to use the material video, the user views the material video only by himself / herself, and browses the material video by himself so that other users can simultaneously browse and discuss the material video. In some cases, the material video is "shown" to other users. The “see” button 207 and the “show” button 208 shown in FIG. 2 realize the above two methods. When the “view” button 207 is pressed, the reproduction of the material video is enabled only for the command issuing terminal itself.
A material video corresponding to only the composite video sent to the terminal is synthesized. "Show" button 20
When the button 8 is pressed, the material video corresponding to the composite video for each terminal of all the participants is synthesized so that all the participants can simultaneously browse and share the same material video. Then, by pressing the playback control button 203, each terminal can play or stop the material video.

It should be noted that among the commands issued by each terminal, only one of the conference participants can issue a playback control command such as selection of a material video to be played, playback start, stop, etc. Alternatively, all the participants of each terminal may issue the commands, and the commands may be made effective in the order in which they arrive at the server. In addition to the material video, the display / non-display of the terminal video of each terminal, the layout of a plurality of videos, the size of each video, and the like may be arbitrarily set and changed in each terminal.
In this case, the display mode of the composite video may be set so that the display area of each video can be indicated by, for example, a pointing device.

The video from each terminal to be displayed may be appropriately selected from all terminals participating in the conference, an arbitrary terminal specified on the display side, or a specific terminal set in advance. If the number of terminals participating in the conference is different from the number of images that can be displayed, especially if the number of images that can be displayed is smaller than the number of terminals, any terminal selected on the display side, or a specific Display video from the terminal.

FIG. 3 shows a GUI displayed on the terminal 102.
FIG. 5 is a conceptual diagram showing a second example of FIG. In this example, in addition to the first example shown in FIG. 2, an enlarged image 301 is arranged in a display area on the left side of the material image 206. The enlarged image 301 is obtained by enlarging the size of the display area of a specific image so that the image can be viewed in an enlarged manner.
For example, a specific terminal image 20 of multi-point terminal images
When the user wants to view the 5X in a greatly enlarged manner, the user selects and instructs the image using a pointing device or the like, and the target image is displayed in the area of the enlarged image 301 in FIG. In teleconferences, etc., the effect is that each user can refer to the video in a form that suits their needs at different times, such as viewing the speaker's expression enlarged, or viewing the video on the blackboard in a distance lecture. is there. The video synthesizing unit 105 may be configured to process and generate a composite video for each terminal in a time-division manner, or may be configured to provide a video synthesizing unit for each terminal and perform parallel processing.

When there are a large number of connected terminals, it is difficult to determine which terminal image is being enlarged and displayed. Therefore, when an image to be enlarged and displayed is instructed to be selected, for example, a cursor frame for selection is displayed around the terminal image, or the terminal image is enlarged and displayed, and the normal size of the previously displayed image is displayed. The terminal image is not displayed and a black screen or a blue screen is displayed. With this method, as the number of connected terminals (connection points) increases and the number of displayed images increases, the original display position of the enlarged image becomes easier to understand, and it is easier to identify which image is being enlarged It has the effect of becoming.

As described above, in the present embodiment, upon receiving a command from the terminal, the video material stored in the storage means in advance is read in response to the command, and the video from each terminal and the material video are combined by the video synthesizing means. By synthesizing a single video, it is possible to obtain a video conference apparatus that can easily use a video recorded and stored in a remote conference or a lecture as a material.

In addition, by providing operation input buttons on the GUI of the video display screen for realizing a function such that only the user himself / herself views the material video to be viewed or all participants, the use of the material can be reduced. The effect that can be more easily obtained is obtained. In addition, by providing an enlarged display area for an image, displaying a specific image in an enlarged manner, and hiding the image at its original display position, the user can better select the image to be focused on as desired. This makes it possible to see the image and to easily recognize which image is enlarged from among many images.

[Second Embodiment] FIG. 4 is a functional block diagram showing a configuration of a video conference apparatus according to a second embodiment of the present invention. The video conference device according to the second embodiment includes a server 401 and a terminal 402 as in the first embodiment.

The server 401 includes a storage unit 403 for storing terminal video data and the like from each terminal, a write control unit 404 for writing data to the storage unit 403, a terminal 4
02, a video synthesizing unit 405 for synthesizing video data transmitted from the terminal 402 into one video, a receiving unit 406 for receiving video data from the terminal 402, a command interpreting unit 407 for interpreting commands representing various operation instructions from the terminal 402, It has a transmitting unit 408 for transmitting the video synthesized by the video synthesizing unit 405 to the terminal 402.

The terminal 402 includes a video camera 409 for photographing the state of the terminal user, a command generating means 410 for generating commands for various operation instructions to be transmitted to the server 401, and a video or command generated by the video camera 409. It comprises a sending means 411 for sending a command from the means 410 to the server 401, a receiving means 412 for receiving the composite video sent from the server 401, and a display means 413 for displaying the video from the server 401.

Also in the present embodiment, one or more (especially two or more when used in a conference or the like) terminal 402 is connected to one server 401.

The operation of the terminal 402 is substantially the same as that of the first embodiment described above, and the second embodiment differs from the first embodiment in the command generated by the command generation means 410 and transmitted to the server 401. The command generation unit 410 corresponds to, for example, a conference history recording start button, a recording stop button, and the like in a GUI displayed on the screen of the display unit 413 of the terminal 402. Note that a selection button, a reproduction start button, a reproduction stop button, and the like of the above-described material video may be provided. The command generation means 410 generates a command paired with each button when these buttons are pressed.

On the other hand, in the server 401, the video data and the command transmitted from each terminal 402 are received by the receiving means 406.
To receive. Then, the received video from each terminal is synthesized into one video by the video synthesis means 405. The method of synthesizing the video is the same as that described in the first embodiment. Then
The image synthesized by the image synthesizing unit 405 is sent back to the terminal 402 by the sending unit 408. In the terminal 402, the receiving unit 412 receives the video data of the composite video from the server 401, and displays the video data on the display unit 413.

In the server 401, the terminal 402 which the command interpreting means 407 receives at the receiving means 406
Interpret commands from. At this time, if the command is the recording of the conference history, a write instruction is issued to the write control unit 404. Write control means 4 receiving a write instruction
04 receives the synthesized video data from the video synthesis means 405 and writes the data into the storage means 403. In addition,
The video recording of each terminal is performed not only when the recording command is received by the server 401 but also when the video from the terminal is transmitted to the server 4.
Recording may be automatically started when 01 is received.

The composite video stored here may be a video transmitted to the terminal that has issued the conference history recording command, or may be a video for a virtual user for storing the conference history. good. In a general usage mode, the images and layouts displayed on each terminal are different. Therefore, in order to store a composite image including all the images displayed on each terminal, a virtual user for image storage is required. It is desirable to perform synthesis by setting. Alternatively, the layout of the composite video may be determined in advance by the system, and the video synthesized using the layout may be created and stored as a composite video of the conference history. The video to be recorded is video data of a moving image in the present embodiment, and is recorded together with audio information.

FIG. 5 shows a GUI displayed on the terminal 402.
FIG. 3 is a conceptual diagram showing an example of the above. In addition to the GUI shown in FIG. 2, in the present embodiment, an operation input corresponding to a conference history recording command such as “recording start” or “stop” for controlling recording of the conference history is provided at the lower right of the display screen. A recording control button 501 to be performed is displayed. This recording control button 501
By pressing, an instruction to start or stop recording the conference history can be made from the terminal.

As in the first embodiment, of the commands issued by each terminal, only one of the conference participants can issue a conference history recording command such as recording start and stop, and this is used as the chairperson authority. Alternatively, all the participants of each terminal may issue the commands, and the commands may be made effective in the order in which they arrive at the server.

In the present embodiment, the composite video itself, which is used as a means of communication in a video conference, is stored, so that it is possible to reproduce the conference video itself that the user is watching at each terminal. Also speaking at meetings,
Since the images from each terminal showing the speech and the like are synthesized and stored in real time, the synchronization between the images does not shift.

As described above, in this embodiment, upon receiving a command from a terminal, the video from each terminal is synthesized into one video by the video synthesizing means in accordance with the command, and this is stored in the storage means. Thus, an effect is obtained that a video conference apparatus capable of easily recording the progress of a conference as a video of a conference history while easily synchronizing the video from each terminal is achieved.

[Third Embodiment] FIG. 6 is a functional block diagram showing a configuration of a video conference apparatus according to a third embodiment of the present invention. The video conference device according to the third embodiment includes a server 601 and a terminal 602 as in the first and second embodiments.

The server 601 includes a storage means 603 for storing terminal video data and material video data from each terminal, a writing control means 604 for writing data to the storage means 603, and a storage means 603 for storing the terminal video data and data from the terminal. Read control means 6 for reading data such as video data
14. Video synthesizing means 605 for synthesizing video data sent from the terminal 602 or video data read from the storage means 603 into one video, receiving means 606 for receiving video data from the terminal 602, and various operations from the terminal 602 Command interpreting means 60 for interpreting a command representing an instruction
7. It has a transmitting means 608 for transmitting the video synthesized by the video synthesizing means 605 to the terminal 602.

The terminal 602 includes a video camera 609 for photographing the user of the terminal, a command generation unit 610 for generating commands for various operation instructions to be transmitted to the server 601, and a video or command generated by the video camera 609. It comprises a sending means 611 for sending a command from the means 610 to the server 601, a receiving means 612 for receiving the composite video sent from the server 601, and a display means 613 for displaying the video from the server 601.

In this embodiment, one or more terminals 602 (especially one terminal 602 may be used when a conference history is reproduced) are connected to one server 601.

The operation of the terminal 602 is almost the same as that of the above-described second embodiment. In the third embodiment, the command generated by the command generation means 610 and transmitted to the server 601 is different. The command generation unit 610 corresponds to, for example, a conference history recording start button, a recording stop button, a conference history reproduction start button, a reproduction stop button, and the like in a GUI displayed on the screen of the display unit 613 of the terminal 602. Note that a selection button, a reproduction start button, a reproduction stop button, and the like of the above-described material video may be provided. Command generation means 610
Generates a command paired with each button when these buttons are pressed.

On the other hand, the server 601 receives the video data and command transmitted from each terminal
To receive. Then, the received video from each terminal is synthesized into one video by the video synthesis means 605. The method of synthesizing the video is the same as that described in the first embodiment. Then
The image synthesized by the image synthesizing unit 605 is sent back to the terminal 602 by the sending unit 608. In the terminal 602, the video data of the composite video from the server 601 is received by the receiving unit 612 and displayed by the display unit 613.

In the server 601, the command interpreting means 607 receives the information from the terminal 602 received by the receiving means 606.
Interpret commands from. At this time, if the command is the recording of the conference history, a write instruction is issued to the write control unit 604. Write control means 6 receiving a write instruction
04 receives video data of each terminal before combination from the receiving unit 606 and writes the data individually into the storage unit 603. The video of each terminal is recorded on the server 60.
1, the recording may be started automatically when the server 601 receives the video from the terminal. In the present embodiment, the video to be recorded is video data of a moving image, and is recorded together with audio information.

Further, when the command sent from the terminal 602 to the server 601 is a conference history reproduction command, the command interpreting means 607 interprets the command and issues a read instruction to the read control means 614. The read control unit 614 reads the video data of the individual terminal required for reproducing the conference history from the storage unit 603 and passes the video data to the video synthesis unit 605. The video synthesizing unit 605 synthesizes the read video of each terminal to make a video of the conference history, and transmits it to the terminal 602 by the transmitting unit 608. In the initial state, the display area of each video is determined by the preset selection and layout of the terminal video, and a composite video of the conference history is generated while synchronizing each video.

During the reproduction of the conference history, the terminal 60
When the server 602 receives a command specifying the display position and display size of each video from the server 2, the server 602 synthesizes the video according to the command. In other words, despite the past conference history video being recorded once, the viewer of the conference history can enlarge the video to his or her preference and pay attention to the speaker, or display the keyman at the center. The history can be reproduced in various forms, such as making it easier to see.

FIG. 7 shows a GUI displayed on the terminal 602.
FIG. 3 is a conceptual diagram showing an example of the above. In the present embodiment, after recording the conference history by using the same GUI as that shown in FIG. 5, when reproducing the conference history, the GUI shown in FIG.
I is used.

At the left end of the display screen, a conference history list 701 for selecting a conference history to be reproduced by the user and a "view" button 702 for giving an instruction to browse the conference history are displayed, and at the lower end. Is a reproduction control button 7 for inputting an operation corresponding to a conference history reproduction command such as "reproduction start", "pause", "stop" for controlling reproduction of the conference history.
03 is displayed. By pressing the playback control button 703, each terminal can play or stop the conference history. In this example, the terminal image 7 of each terminal is displayed at the bottom as a composite image 704 synthesized by the server 601.
05 are arranged, and a material image 706 and an enlarged image 707 are combined and displayed side by side in a large display area on the upper side.

Similarly to the reproduction of the material video in the first embodiment, the reproduction of the conference history is valid only for the command issuing terminal, and the conference history video corresponding to only the composite video sent to the terminal is synthesized. There is a method and a method of synthesizing a conference history video corresponding to the composite video for each terminal of all the participants, and simultaneously browsing and sharing the same conference history by all the participants.

As in the first and second embodiments, among the commands issued by each terminal, a conference history reproduction command such as reproduction start and stop and a conference history recording command such as recording start and stop are issued. Only one of the conference participants may be allowed to use this as the chairperson authority, or all participants at each terminal may be issued, and commands may be made effective in the order in which they arrive at the server.

As described above, in the present embodiment, when recording the conference history, the video from each terminal is individually stored in the storage means, and when the conference history is reproduced, each video included in the conference history is read out. By synthesizing the video by determining the layout at the time of synthesis according to the command from the terminal, the conference history can be arbitrarily recorded as a video, and the viewer of the conference history can be displayed according to his preference An effect is obtained that a video conference apparatus capable of reproducing a history in a form can be realized.

[Fourth Embodiment] FIG. 8 is a functional block diagram showing the configuration of a video conference apparatus according to a fourth embodiment of the present invention. The video conference device according to the fourth embodiment includes first to third
As in the embodiment, the configuration includes a server 801 and a terminal 802.

The server 801 includes a storage unit 803 for storing terminal video data and material video data from each terminal, a write control unit 804 for writing data to the storage unit 803, and Read control means 8 for reading data such as video data
14. Image synthesizing means 805 for synthesizing the image data transmitted from the terminal 802 and the image data read from the storage means 803 into one image, receiving means 806 for receiving the image data from the terminal 802, and various types of signals from the terminal 802. Command interpreting means 80 for interpreting a command representing an operation instruction
7. A transmission unit 808 for transmitting the video synthesized by the video synthesis unit 805 to the terminal 802.

The terminal 802 includes a video camera 809 for photographing the state of the terminal user, a command generating means 810 for generating various operation instruction commands transmitted to the server 801, and a video or command generated by the video camera 809. It comprises a sending means 811 for sending a command from the means 810 to the server 801, a receiving means 812 for receiving the composite video sent from the server 801, and a display means 813 for displaying the video from the server 801.

Also in the present embodiment, one or more (especially two or more when used in a conference or the like) terminal 802 is connected to one server 801.

The operation of the terminal 802 is almost the same as that of the third embodiment described above, and the fourth embodiment differs in the command generated by the command generation means 810 and transmitted to the server 801. The command generation unit 810 corresponds to, for example, a conference history recording start button, a recording stop button, a conference history reproduction start button, a reproduction stop button, a slider, and the like on a GUI displayed on the screen of the display unit 813 of the terminal 802. Note that a selection button, a reproduction start button, a reproduction stop button, and the like of the above-described material video may be provided.

The record button records the current video (live video) from each terminal at the server. The play button plays back a past video (recorded video) from each recorded terminal. The slider specifies the playback time of the video to be played. The command generation means 810 generates a command paired with each button when these buttons are pressed. The recording of the live video of each terminal may be automatically started when the server 801 receives the live video instead of when the server 801 receives the recording command.

On the other hand, the server 801 receives the video data and the command sent from each terminal 802 and
To receive. Then, the received video from each terminal is synthesized into one video by the video synthesis means 805. The method of synthesizing the video is the same as that described in the first embodiment. Then
The image synthesized by the image synthesizing unit 805 is sent back to the terminal 802 by the sending unit 808. In the terminal 802, the video data of the composite video from the server 801 is received by the receiving unit 812 and displayed by the display unit 813.

In the server 801, the command interpreting means 807 receives the information from the terminal 802 received by the receiving means 806.
Interpret commands from. At this time, if the command is recording of a live video, a write instruction is issued to the write control unit 804. Upon receiving the write instruction, the writing control unit 804 receives the video data from the terminal before combining from the receiving unit 806, and writes the data to the storage unit 803. In the present embodiment, the video to be recorded is video data of a moving image.

Further, when the command transmitted from the terminal 802 to the server 801 is a command for reproducing a recorded video, the command interpreting means 807 interprets the command,
A read instruction is issued to the read control unit 814. The read control unit 814 reads the video data of the video from the storage unit 803 and passes the video data to the video synthesis unit 805. The video synthesizing unit 805 synthesizes the recorded video of the terminal read from the storage unit 803 with the live video from the terminal received by the receiving unit 806, and transmits the synthesized video.
To the terminal 802.

Note that, during this conference, when a command for specifying the display position and display size of each video is received from the terminal 802, the server 802 synthesizes the video according to the command. In other words, users participating in the meeting can view live images in various forms, such as enlarging the image to their liking and paying attention to the speaker, or displaying the keyman at the center for easy viewing. The playback can be performed by comparing the live video with the past recorded video.

FIG. 9 shows a GUI displayed on the terminal 802.
FIG. 3 is a conceptual diagram showing an example of the above. In the present embodiment, when a live video of a terminal is recorded by using a GUI similar to that shown in FIG. 5, and then a comparison is made between the current live video and the past recorded video, as shown in FIG. Use the GUI.

At the left end of the display screen, a material video list 901 for the user to select a material video, a participant list 902 for displaying the participants of the conference, and an instruction to browse the material video only by himself. A “view” button 903 to be performed and a “show” button 904 for allowing other users to view the material video are displayed. At the lower end, a reproduction control slider 905 for performing an operation input corresponding to a recorded video reproduction control command for controlling the time of the video to be reproduced is displayed.

In this example, the terminal image 90 of each terminal is displayed at the bottom as a composite image 906 synthesized by the server 801.
7 are arranged, and a material image 9 is displayed in a large display area on the upper side.
08 and the enlarged image 909 are combined and displayed side by side.
Instead of the material video 908, the enlarged video 909 of the selected video 910 selected from the terminal video 907 and the past recorded video of the video may be displayed side by side and large.
By sliding the reproduction control slider 905 right and left, the user can reproduce a past video at a desired time of the designated video.

Similar to the reproduction of the material video in the first embodiment, the reproduction of the video from the terminal to the past is effective only for the command issuing terminal and corresponds only to the composite video sent to the terminal. There are a method of synthesizing the video and a method of synthesizing a video corresponding to the synthesized video for each terminal of all the participants and simultaneously viewing and sharing the same past video by all the participants.

As in the first and second embodiments, among the commands issued by each terminal, the issuance of a recorded video playback control command such as playback start / stop and the video recording control command such as recording start / stop are described. Alternatively, only one of the conference participants may be set as the chairperson authority, or all the participants of each terminal may issue the command, and the commands may be made effective in the order of arrival at the server.

As described above, in the present embodiment, while the video from each terminal is stored in the storage means, the stored past video from the terminal is read out by the command from the terminal.
By synthesizing the stored recorded video and the current live video received by the receiving unit into one video by the video synthesizing unit, the designated video can be enlarged and displayed according to a command from the user, or can be displayed within the synthesized video. It is possible to obtain a teleconferencing device capable of recording a conference history video, which can simultaneously refer to a live video from a terminal and a past video of the live video while changing the display position of the video. .

[0083]

As described above, according to the present invention, in accordance with a command from a terminal, a video material previously stored in a storage means is read out and combined with a video from each terminal into one video by a video synthesis means. By doing so, there is an effect that it is possible to provide a video conference apparatus that can use a video as a material in a conference or a lecture.

Further, in response to a command from the terminal, the images from each terminal are combined into one image by the combining means, and this is stored in the storage means, so that synchronization between the images from each terminal can be easily achieved. However, there is an effect that a video conference device capable of recording the progress of a conference or a lecture as a video of a conference history can be provided.

When recording a conference history during a conference or a lecture, the video from each terminal is individually stored in the storage means, and when the conference history is reproduced, each video included in the conference history is read out. By determining and synthesizing the layout at the time of synthesis in accordance with the command from the user, a viewer of the conference history can use a video conference apparatus that can reproduce the video of the history in a display form according to his / her preference. There are effects that can be provided.

In addition, while storing the video from each terminal during a conference or a lecture in the storage means, the stored past video from the terminal is read in response to a command from the terminal, and the stored past video is read. By combining the video and the current video received by the receiving means into one video by the video combining means, the specified video can be enlarged and displayed according to the command from the user, or the display position in the composite video can be changed. While
There is an effect that it is possible to provide a video conference apparatus capable of recording a conference history video that can simultaneously refer to a current video from a terminal and a past video of the video.

[Brief description of the drawings]

FIG. 1 is a functional block diagram illustrating a configuration of a video conference device according to a first embodiment of the present invention.

FIG. 2 is a conceptual diagram showing a first example of a GUI displayed on the terminal according to the first embodiment.

FIG. 3 is a conceptual diagram showing a second example of a GUI displayed on the terminal according to the first embodiment.

FIG. 4 is a functional block diagram illustrating a configuration of a video conference device according to a second embodiment of the present invention.

FIG. 5 is a conceptual diagram illustrating an example of a GUI displayed on the terminal according to the second embodiment.

FIG. 6 is a functional block diagram illustrating a configuration of a video conference device according to a third embodiment of the present invention.

FIG. 7 is a conceptual diagram illustrating an example of a GUI displayed on a terminal according to a third embodiment.

FIG. 8 is a functional block diagram illustrating a configuration of a video conference device according to a fourth embodiment of the present invention.

FIG. 9 is a conceptual diagram illustrating an example of a GUI displayed on a terminal according to a fourth embodiment.

[Explanation of symbols]

 101, 401, 601, 801 server 102, 402, 602, 802 terminal 103, 403, 603, 803 storage means 104, 614, 814 read control means 105, 405, 605, 805 video synthesis means 106, 406, 606, 806 Receiving means 107, 407, 607, 807 Command interpreting means 108, 408, 608, 808 Sending means 109, 409, 609, 809 Video camera 110, 410, 610, 810 Command generating means 111, 411, 611, 811 Sending means 112 , 412, 612, 812 Receiving means 113, 413, 613, 813 Display means 404, 604, 804 Writing control means

 ────────────────────────────────────────────────── ─── Continued on the front page F term (reference) 5C023 AA02 AA14 AA37 AA38 BA11 CA01 DA01 EA03 5C064 AA02 AC02 AC14 AC18 AC22 AD02

Claims (21)

[Claims] 1. A video conference apparatus for transmitting video between at least one terminal and a server, wherein the server stores video data of reference information including a material video in the server; Read control means for reading video data stored in the terminal; receiving means for receiving a video sent from the terminal; command interpreting means for interpreting a command from the terminal; and a plurality of videos including a video from the terminal A video conference apparatus comprising: video synthesizing means for synthesizing data into one video; and transmitting means for transmitting the synthesized video to a terminal. 2. The terminal, further comprising: an operation input unit for issuing a command for synthesizing video data of the material video only to the synthesized video for the terminal, wherein the video synthesizing unit transmits a command from the terminal to the terminal. 2. The television according to claim 1, wherein when received by the command interpreting means, the read control means combines the video data of the material video read from the storage means with only the video for the corresponding terminal. Conference equipment. 3. An operation input means for issuing a command for synthesizing video data of a material video to a synthesized video for each terminal, wherein said video synthesizing means transmits a command from said terminal to said command. 2. The video conference apparatus according to claim 1, wherein when received by the interpretation means, the read control means combines the video data of the material video read from the storage means with the video for each terminal. 4. A video conference apparatus for transmitting video between one or more terminals and a server, wherein the server stores video data, and receives a video transmitted from the terminal. Receiving means; command interpreting means for interpreting a command from the terminal; video synthesizing means for synthesizing a plurality of video data including video from the terminal into one video; A video conference apparatus comprising: a writing control unit that writes data into a storage unit; and a transmission unit that transmits a video synthesized by the video synthesis unit to a terminal. 5. An operation input unit for issuing a command for recording a video from each terminal to the terminal, wherein the write control unit receives a command from the terminal by the command interpretation unit. 5. The video conference apparatus according to claim 4, wherein the video synthesizing unit writes video data obtained by synthesizing a plurality of videos including a video from a terminal into the storage unit. 6. A video conference apparatus for transmitting video between one or more terminals and a server, wherein the server stores video data, and receives the video transmitted from the terminal. Receiving means, command interpreting means for interpreting a command from the terminal, write control means for writing each video from the terminal to the storage means, read control means for reading video data stored in the storage means, A video synthesizing unit that synthesizes a plurality of video data including a video from the terminal read from the storage unit into one video; and a sending unit that transmits the synthesized video to the terminal. Video conference equipment. 7. A video conference apparatus for transmitting video between one or more terminals and a server, wherein the server stores video data, and receives a video transmitted from the terminal. Receiving means, command interpreting means for interpreting a command from the terminal, write control means for writing each video from the terminal to the storage means, read control means for reading video data stored in the storage means, Image synthesizing means for synthesizing a plurality of video data including the video read from the storage means and the video from the terminal received by the receiving means into one video, and sending means for transmitting the synthesized video to the terminal; A video conference device comprising: 8. An operation input unit for issuing a command for recording a video to the terminal, wherein the write control unit receives the command from the terminal by the command interpreting unit, and receives the command from the terminal. 8. The video conference apparatus according to claim 6, wherein the video data received from each terminal is written in the storage unit. 9. The terminal, further comprising: an operation input unit for issuing a command for reproducing a stored video, wherein the video synthesizing unit receives a command from the terminal by the command interpreting unit, 8. The video conference apparatus according to claim 6, wherein a plurality of video data including a terminal video read from the storage unit by the read control unit is combined as a video for the terminal. 10. The terminal, further comprising: an operation input unit for issuing a command for reproducing a stored video, wherein the video synthesizing unit receives a command from the terminal by the command interpreting unit. A plurality of video data including a past terminal video read from the storage unit by the read control unit and a current terminal video received by the receiving unit are combined as a video for the terminal. The video conference device according to claim 7. 11. The video conference apparatus according to claim 10, wherein said operation input means has a slider for indicating a time of a video to be reproduced. 12. A graphical user interface for displaying a plurality of videos side by side in a composite video displayed on the terminal, and displaying a specific video among them in an enlarged display area. The video conference apparatus according to any one of claims 1, 4, 6, and 7, wherein 13. A composite video displayed on the terminal, a plurality of videos are displayed side by side, and a specific video among them is enlarged and displayed in an enlarged display area, and the enlarged original video is displayed. Graphical user to hide
An interface is provided.
The video conference device according to any one of 4, 6, and 7. 14. The apparatus according to claim 2, wherein said operation input means is constituted by a graphical user interface expressed on a display screen of a terminal.
The video conference device according to any one of 5, 8, 9, 10, and 11. 15. A video conference apparatus for transmitting a video between one or more terminals and a server, wherein a current video from the terminal, a stored material video, and a recorded video are recorded on a display screen of the terminal. And a graphical user interface for displaying a plurality of videos including at least one of the past videos from the terminal and displaying a specific video among these videos in an enlarged display area. A video conference device. 16. A video conference apparatus for transmitting a video between one or more terminals and a server, wherein a current video from the terminal, a stored material video, and a recorded video are recorded on a display screen of the terminal. Along with displaying a plurality of videos including at least one of the past videos from the terminal that has been displayed, a specific video among these videos is enlarged and displayed in an enlarged display area, and the enlarged original video is displayed. A video conference apparatus comprising a graphical user interface for highlighting. 17. A video conference apparatus for transmitting video between one or more terminals and a server, wherein at least a past video recorded from the terminal is displayed on a display screen of the terminal, Graphical user with a slider to indicate the playback time of this video
A video conference device comprising an interface. 18. The video conference apparatus according to claim 1, wherein the video data has been subjected to a compression process, and the compression format is MPEG. 19. The video data has been subjected to compression processing, and its compression format is Motion JPEG.
The video conference device according to any one of claims 1, 4, 6, and 7, wherein 20. The apparatus according to claim 1, wherein said command interpreting means validates only a command from a predetermined terminal.
The video conference device according to any one of the above. 21. The video conference apparatus according to claim 1, wherein said command interpreting means validates commands from all terminals.