JP2019068242A - Electronic conference system, processing method thereof, and program - Google Patents

Abstract

To provide a mechanism to create an environment where an electronic conference can proceed smoothly even when photographing is switched.MEANS FOR SOLVING THE PROBLEM: An electronic conference system that displays an image taken with a terminal includes: reception means for receiving an image of a terminal participating in an electronic conference; display means for displaying the image received by the reception means in the electronic conference; temporary storage means for temporarily storing an image to be displayed on the display means; and acquisition means for acquiring the image stored in the temporary storage means if a photographing state of the terminal participating in the electronic conference is switched. The display means displays the image acquired by the acquisition means in the electronic conference.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an electronic conference system that displays an image captured by a terminal, a processing method thereof, and a program.

  In recent years, web conferencing systems have become widespread. A web conference is a video conference that is one of electronic conferences.

  Moreover, with the spread of smart devices, web conferencing systems using portable terminals are also generally provided. Devices such as an in-camera, an out-camera (imaging device), and an acceleration sensor are installed in portable terminals sold these days.

  For example, Patent Document 1 describes a technology for conducting a web conference using a portable terminal. According to Patent Document 1, when a web conference is performed on a portable terminal, for example, an image is captured using a camera mounted on the portable terminal itself or a camera separately installed and connected on the portable terminal, and the image To other terminals participating in the conference.

  Further, Patent Document 2 describes a technique for performing a hands-free videophone call using a mobile phone. According to the distance between the mobile phone and the user determined from the image captured by the in-camera, the normal call and the hands-free call are switched, and the transition from the normal phone to the video phone can be smoothly performed.

JP, 2015-82207, A JP, 2008-187221, A

  When conducting an electronic conference, the user wants to show his / her own image to the other party of the conference, so it is basic to place the terminal used for the electronic conference on a desk and make a conference using an in-camera . On the other hand, since electronic conferences have come to be conducted with portable terminals, it has been more common to hold electronic devices at hand and conduct electronic conferences at work sites such as construction sites.

  In this case, the camera is switched from the in-camera (for example, taking a face image of itself with the camera on the front of the portable terminal and shared in the electronic conference) to the out-camera during the electronic conference, and the out-camera It has become necessary to capture the situation and share the captured image at a meeting.

  In the current electronic conference, it is necessary to set devices (cameras, microphones) used in the electronic conference, and in many cases, a camera used in the electronic conference is set as an in-camera. An image showing the participant in the in-camera is used in the electronic conference and displayed on the display of the other participant.

  However, if you switch to an out-camera (the display of this portable terminal will display an image taken by the out-camera) and take an on-site image, the in-camera image will not be sent, so join the electronic conference The display area of the participant of the terminal switched to the out camera is blacked out on the terminal of the other participant, and the other portable terminal has a problem and the conference does not proceed smoothly. Problem that led to

  Therefore, an object of the present invention is to provide a mechanism for creating an environment in which an electronic conference can be smoothly advanced even when imaging is switched.

  An electronic conference system for displaying an image captured by a terminal, comprising: receiving means for receiving an image of a terminal participating in the electronic conference; display means for displaying the image received by the receiving means in the electronic conference; The temporary storage means for temporarily storing the image to be displayed by the display means, and the acquisition means for acquiring the image stored by the temporary storage means when the photographing state of the terminal participating in the electronic conference is switched, The display unit displays the image acquired by the acquisition unit in an electronic conference.

  According to the present invention, it is possible to create an environment in which the electronic conference can be smoothly advanced even when the imaging is switched.

It is a figure which shows an example of a system configuration of the electronic conference system in this embodiment. It is a block diagram which shows an example of the hardware constitutions of a client terminal in this embodiment. It is a figure which shows an example of a function structure of the electronic conference system in this embodiment. It is a flowchart which shows an example of a meeting start process in this embodiment. It is a flowchart which shows an example of the display process at the time of 2 unit control mode in this embodiment. It is a flowchart which shows an example of the display process at the time of one unit control mode in this embodiment. It is a figure which shows an example of the management information stored on the memory of a server in this embodiment. It is a figure which shows an example of the mode setting information managed by the client terminal in this embodiment, and specification apparatus information. It is an image figure showing an example of a screen before camera change of personal digital assistant 150 in this embodiment. It is an image figure showing an example of the screen after camera change of personal digital assistant 150 in this embodiment. It is an image figure which shows an example of the screen before switching in PC 100 in this embodiment, and after switching. FIG. 8 is an image diagram showing an example of a shooting switching setting screen at the time of one-device control in the PC 100 in the present embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an electronic conference system according to an embodiment of the present invention.

  As shown in FIG. 1, in the electronic conference system of the present invention, a server 200, a PC 100 (PC 100A, PC 100B / hereinafter, generically described as PC 100) and a portable terminal 150 are communicably connected via a network 101. It is configured. Hereinafter, the PC 100 and the portable terminal 150 will be collectively referred to as a client terminal.

  The server 200 has various functions for executing a web conference using a client terminal. For example, create and store a room for a web conference (virtual conference room). For example, a list of meeting rooms (meeting room A, meeting room B,...) In the management information of FIG. 7 is stored in advance in the external memory of user information of users who can log in.

  Also, the server 200 receives, from the identification information of the client terminal and each client terminal, for example, a conference image (camera image / video) which is image data obtained by imaging a user who is a conference participant. Then, the conference image and the identification information of each client terminal are associated and stored.

  Further, identification information of the client terminal and user information (for example, user ID) of the user who is logging in to the server 200 by the client terminal are stored in association with each other (see FIG. 7). The server 200 distributes the image, voice, and other data received from each client terminal to the client terminals in the same conference room (participating in the conference).

  The server 200 may be a server in a company or a server in a cloud environment.

  The PCs 100A and 100B are terminals having a camera function. In the present embodiment, it is assumed that 100A and 100B have one camera. The camera function may be a built-in camera or a camera connected via USB.

  Image data (meeting image) taken as an image captured and acquired by the camera by logging in to the server 200 (including entering the meeting room) according to an operation from the user (for example, in response to an input / transmission instruction of authentication information) (Video / dynamic data) are sequentially transmitted via the server 200 to another PC 100 that is in the same room (virtual meeting room) while participating in the meeting. Here, the PC 100 communicates with the server 200 via the LAN.

  The portable terminal 150 is a portable terminal (a tablet, a smartphone, etc.) provided with a camera function and a touch panel function (operation unit).

  The portable terminal 150 will be described as having two cameras. The two cameras are an in-camera (front camera) provided on the liquid crystal display side and an out-camera (rear camera) provided on the opposite side of the liquid crystal display (the back surface of the portable terminal 150). Although this embodiment will be described as a built-in camera, a camera connected via USB may be used.

  As in the PCs 100A and 100B, the server 200 logs in (including entering the conference room) in response to an operation from the user (for example, in response to an input / transmission instruction of authentication information), an image captured and acquired by the camera Image data (meeting image / video / dynamic data) are sequentially transmitted to another PC 100 in the same room (virtual meeting room) via the server 200 while participating in the meeting. . Here, it is assumed that the portable terminal 150 communicates with the server 200 via a wireless LAN or a public communication line and a communication method (3G, 4G, etc.).

  In addition to the image data acquired from the camera, each client terminal can transmit, via the server 200, data stored in the client terminal, for example, static data such as a photo image and document data such as a PDF file. And a data sharing function to transmit to another client terminal for sharing.

  When transmitting the static data to another client terminal, for example, all the client terminals perform control to display the static data on the display screen. In the data sharing function, data (shared image) in the same state is normally displayed on the plurality of client terminals as data being shared among the plurality of client terminals. For example, when the shared image is enlarged in the PC 100A, the sharing image being displayed on the other terminal (PC 100B, portable terminal 150) is also enlarged at the same enlargement position and enlargement ratio.

  In addition, when a file (such as a file of a captured image) is uploaded during a meeting (at the time of file upload), the display is controlled according to the material.

  If operation / editing on the displayed shared image (for example, an image of document data) from all the terminals is enabled, there is a possibility that the operation (processing (display control) corresponding to each operation) may conflict. Here, since the mobile terminal 150 (the terminal that uploaded the file) that started the data sharing is the host, control is performed so that only the host terminal can edit the data being shared. As an example of operation competition, the case where enlargement and reduction operations are performed simultaneously from a plurality of terminals can be mentioned.

  Each client terminal accesses the server 200 by receiving input of user information (user ID, password). Immediately after access (login), for example, a conference selection screen (not shown) is displayed. Then, they enter the meeting room and execute the meeting.

  A client application for executing a web conference is installed in the client terminal, and can communicate with the server 200 to control the device used by the client terminal. The above is the description of FIG.

  As a suitable example of the present embodiment, a meeting is performed via the server 200. However, when a meeting is performed by two terminals, it goes without saying that the terminals may directly communicate in the form of P2P. Yes. That is, the server 200 may not be used.

  Further, in the present embodiment, although the electronic conference system is described, it may be used as a communication tool as well as the conference mode. In other words, it can be rephrased as a communication system.

  Next, hardware configurations of the portable terminal 150 as the information processing apparatus shown in FIG. 1 and the PCs 100A and 100B will be described using FIG.

  (A) of FIG. 2 is a block diagram showing a hardware configuration of the portable terminal 150 as the information processing apparatus shown in FIG.

  In FIG. 2, reference numeral 201 denotes a CPU, which centrally controls devices and controllers connected to the system bus 204. Further, the ROM 202 or the external memory 211 is required to realize a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter referred to as OS), and functions executed by each server or each PC. The various programs described later are stored.

  A RAM 203 functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 loads a program and the like necessary for execution of processing from the ROM 202 or the external memory 211 to the RAM 203, and implements the various operations by executing the loaded program.

  A display controller 205 controls input from the touch panel 209 and display (output) on the touch panel 209. Although not shown, a controller for receiving an input of a hard key provided in the apparatus may also be provided.

  A memory controller 207 is a hard disk (HD) for storing a boot program, various applications, font data, user files, editing files, various data, etc., and an external memory such as an SD card connected to a card slot via an adapter. Control access to 211.

  A communication I / F controller 208 connects and communicates with an external device via a network (for example, LAN / WAN shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the touch panel 209 by executing, for example, outline font rasterization processing on a display information area in the RAM 203. In addition, the CPU 201 enables a user instruction by a touch operation on the touch panel 209.

  Reference numerals 212 and 213 denote cameras which are built in the portable terminal 150. An in-camera (front camera) 212 and an out-camera (rear camera) 213 are controlled by the camera controller 206. Further, in accordance with an instruction from the application of the portable terminal 150, control (switching) of a camera to be used can be performed. That is, switching between in-camera and out-camera is possible.

  Various programs to be described later for realizing the present invention are stored in the external memory 211, and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. Furthermore, definition files and various information tables used at the time of execution of the program are also stored in the external memory 211, and a detailed description of these will be described later.

  FIG. 2B is a block diagram showing the hardware configuration of the PC 100A, 100B as the information processing apparatus shown in FIG. In addition, the server 200 is also the same structure and abbreviate | omits description.

  In FIG. 2, reference numeral 221 denotes a CPU, which centrally controls devices and controllers connected to the system bus 224. Further, the ROM 222 or the external memory 231 is required to realize a BIOS (Basic Input / Output System) which is a control program of the CPU 221, an operating system program (hereinafter referred to as OS), and functions executed by each server or each PC. The various programs described later are stored.

  A RAM 223 functions as a main memory, a work area, and the like of the CPU 221. The CPU 221 loads programs necessary for execution of processing from the ROM 222 or the external memory 231 into the RAM 223 and executes the loaded programs to realize various operations.

  An input controller 225 controls an input from a keyboard (KB) 229 or a pointing device such as a mouse (not shown). A video controller 226 controls display on a display such as the display 230. Although the display 230 is described in FIG. 2, the display may be another display such as a CRT or a liquid crystal display. These are used by the administrator as needed.

  A memory controller 227 is connected via an adapter to a hard disk (HD) storing a boot program, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. Access to an external memory 231 such as a compact flash (registered trademark) memory.

  A communication I / F controller 228 connects and communicates with an external device through a network (for example, the network 101 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  The CPU 221 enables display on the display 230 by executing, for example, outline font rasterization processing on a display information area in the RAM 223. In addition, the CPU 221 enables user instruction with a mouse cursor (not shown) or the like on the display 230.

  Various programs to be described later for realizing the present invention are stored in the external memory 231, and are executed by the CPU 221 by being loaded into the RAM 223 as necessary. Furthermore, definition files and various information tables used at the time of execution of the program are also stored in the external memory 231, and a detailed description of these will be described later.

  Reference numeral 233 denotes a camera which is built in the PC 100A or 100B. Also, it is controlled by the camera controller 232. In the present embodiment, as described above, one camera is connected, but the PC 100A or 100B may have two cameras like the portable terminal 150.

  The above is the description of an example of the hardware configuration of various devices in the embodiment of the present invention shown in FIG.

  Next, with reference to FIG. 3, an example of a functional configuration of the electronic conference system in the embodiment of the present invention will be described. The electronic conference system includes a client terminal (mobile terminal 150, PC 100A, PC 100B) and a server. The conference system includes a reception unit 311, a display unit 312, a temporary storage unit 313, an acquisition unit 314, an instruction unit 315, a specification unit 316, and a setting unit 317, and is executed by the CPU.

  First, the electronic conference system according to the present embodiment is an electronic conference system that displays an image captured by a terminal.

  The receiving unit 311 is a functional unit that receives an image of a terminal participating in the electronic conference.

  The display unit 312 is a functional unit that displays the image received by the reception unit 311 in an electronic conference. The function is also a function of displaying an image acquired by the acquisition unit 314 in an electronic conference.

  In addition, the display unit 312 has a function of displaying the image acquired by the acquisition unit 314 in the electronic conference and displaying the acquired image acquired by switching when the imaging state of the terminal participating in the electronic conference is switched. It is a department.

  Further, the display unit 312 is a functional unit that displays the image acquired by the acquisition unit 314 in an electronic conference in accordance with the imaging switching instruction from the instruction unit 315. The switching of photographing is switching of a plurality of photographing devices.

  The temporary storage unit 313 is a functional unit that controls temporary storage of the image displayed on the display unit 312.

  The acquisition unit 314 is a functional unit that acquires the image stored in the temporary storage unit 313 when the imaging state of the terminal participating in the electronic conference is switched.

  The instruction unit 315 is a functional unit that instructs switching of imaging in the terminal.

  The identifying unit 316 is a functional unit that identifies an imaging device available at the terminal. The identifying unit 316 identifies the number of connected imaging devices.

  The setting unit 317 is a functional unit that performs switching control of photographing according to a specific result (for example, the number of photographing devices) by the specifying unit 316. The switching control setting of this photographing is setting of control by plural units (for example, two units) or control by one unit.

  Next, processing in the system of the present embodiment will be described using the flowcharts of FIG. 4, FIG. 5, and FIG. The processing shown in each flowchart is executed by the CPU of each device using the function of the functional unit of each device. The communication between the client terminal and the server is controlled by the communication I / F 208 of the client terminal and the communication I / F 228 and the communication I / F of the server. Moreover, when communication control can be performed between communication I / Fs of the client terminal, it may be executed without passing through the server.

  First, with reference to FIG. 4, the flow of the conference start process in the present embodiment will be described. First, it is assumed that the user is logged in to the conference system. For login, it is assumed that login is performed with reference to the user table 702 in FIG.

  In step S401, a request for entering a meeting room (for example, meeting room A) designated by the list of meeting rooms stored in advance in the server 200 is transmitted. It is assumed that an entry request is made from each of the mobile terminal 150 and the PC 100A, 100B (client terminal).

  In step S402, an entry request is received.

  In step S403, when an entry request is received, management information of a user ID and terminal identification information (session at login) is registered (updated) as indicated by 701 in FIG. The management information may be registered by determining whether or not the user can enter the room according to the room entry permission information (user information and the like which can be admitted into the room) registered in the conference room.

  Using this management information, it is possible to manage users who enter the meeting room and terminals, and to distribute video and audio.

  In step S404, after entering the room, a list screen of users who have entered the room is displayed. An example of the user list screen is 901 in FIG. 9 and 1101 in FIG. Also, each terminal enters the room, and the process proceeds to step S405. The list screen displays a layout screen according to the number of participants. For example, 902 in FIG.

  In step S405, the server 200 sends a conference start notification to the client terminal, and the client terminal receives the conference start notification and starts a web conference. A web conference is held by sharing voices and images of the respective terminals with each other.

  The meeting is composed of the chairperson and other participants. The chairman may register in advance the user who registered the conference room, the user who entered the conference room first, or the chairman user in the conference room in advance.

  In step S406, in order to perform setting processing of the client terminal, it is determined whether there are two camera connections to its own terminal. If two units are connected (including built-in), the process moves to step S407. If one unit is connected, the process proceeds to step S408.

  In step S407, setting of the two-unit control mode is performed as setting information managed by the client terminal. Information indicating “two-unit control” is stored in the setting mode 800 held in the external memory or RAM of the client terminal.

  In step S408, one-unit control mode is set as setting information managed by the client terminal. Information indicating "one unit control" is stored in the setting mode 800 held in the external memory or RAM of the client terminal.

  In addition, although the process of step S406-step S408 was carried out after the process of step S405, you may carry out at any timing. For example, there are cases where the application for web conferencing is launched, the timing at which the user enters the conference room, and the application for web conferencing detects a camera connection.

  The process of the web conference in the two control mode will be described with reference to FIG. The process of the web conference in the single-device control mode will be described with reference to FIG. Although FIGS. 5 and 6 explain the image sharing at the time of camera switching for the sake of explanation, it is assumed that the audio sharing is performed as described in step S405.

  Next, display processing in the two-unit control mode in the present embodiment will be described with reference to FIG. Although this embodiment is described as a configuration in which two cameras are switched, a configuration in which a plurality of cameras such as two built-in cameras and one externally connected camera may be switched may be employed. In other words, it can be reworded as a multiple unit control mode.

  In FIG. 5, first, it is assumed that the default camera (main camera) used in the web conference is set as the built-in camera 1 (in-camera). Moreover, it demonstrates as a process of the portable terminal 150. FIG. The specification device information managed by the external memory or RAM of the portable terminal 150 includes the microphone 801, the speaker 802, and the video 803 in FIG. It holds camera information to be used in the video 803.

  In step S501, photographing is performed by the in-camera (903 in FIG. 9), and in step S502, the photographed image is transmitted. This shooting and transmission shall be repeated. That is, the video of the portable terminal 150 is transmitted.

  In step S503, the captured image is received from the portable terminal 150, terminals of the users participating in the conference (PCs 100A and 100B) are specified, and in step S504, the captured image is transmitted. It is assumed that the terminals of the users participating in the conference can be identified by 701 in FIG. Note that information 701 is managed for each meeting room.

  In step S505, a photographed image is received from the server 200, and in step S506, the image is displayed on the display. An example of the display is 1101 in FIG.

  At this time, similarly, the captured image of the participant's terminal is displayed on the portable terminal 150. An example of the display is 901 in FIG.

  In step S507, the captured image is temporarily stored (cached) in the RAM. Although the captured image stored here is the image displayed most recently, a plurality of images within a predetermined time may be cached. This cached image is used when switching cameras. Also, the cache image is managed for each participant's terminal.

  In step S508, it is determined whether the built-in camera 2 (out-camera) has been switched. The switching of the camera is switched by selecting the camera to be used from the cameras provided in the portable terminal 150 on the setting screen (912 in FIG. 9) of the application for web conference of the portable terminal 150. If there is no switching, the photographing and transmission of the photographed image are repeated. If the camera has been switched, the process proceeds to step S509. Reference numeral 912 denotes an example of a shooting switching instruction.

  Here, 911 in FIG. 9 will be described. When an instruction for setting the specification device is given on the screen 901, the screen 911 shown in FIG. 9 is obtained. When the camera is selected on the setting screen 912 (913) and the switching button (914) is pressed, the camera is switched.

  In step S 509, information indicating that camera shooting has been paused (camera shooting pause information) is transmitted to the server 200.

  In step S510, camera imaging pause information is received from the portable terminal 150, and in step S511, camera imaging pause information is transmitted to the participant's terminal (PC 100A, 100B).

  In step S512, camera imaging pause information is received. The camera imaging pause information is managed in association with the camera imaging paused terminal. This makes it possible to identify which terminal is switching the camera.

  In step S513, the captured image being cached is acquired. This is because each terminal can not receive an image from the server 200 if the photographed image (video) is not sent from the participating party, and the terminal is in a non-display state.

  This process is an example of an acquisition process for acquiring the temporarily stored image when the imaging state of the terminal (mobile terminal 150) participating in the electronic conference is switched.

  In step S514, the acquired captured image (cache image) is displayed in the display area of the target participant. As a result, the web conference can be continued without being undisplayed and alleviating discomfort. In addition, since it looks like it has stopped by using the cache image, it is desirable to notify that “the user: Mr. XX is in the process of camera switching” at a place where the cache image is displayed or the like.

  In step S515, shooting is started with the out camera (built-in camera 2) which is the switched camera. At this time, the image of the out camera is displayed on the display on the screen 1001 in FIG.

  In step S516, an operation instruction of the still image shooting button 1002 is received. When an operation instruction (depression) of the still image shooting button 1002 is received, the shot image is transmitted. In addition, in this embodiment, although a still picture is acquired and it explains as an example which transmits to a participant, a form which transmits an animation may be sufficient.

  In step S517, the photographed image is transmitted to the server 200 in accordance with the instruction to press the still image photographing button 1002.

  In step S518, the photographed image is received from the portable terminal 150, and in step S519, the photographed image is transmitted to the participant's terminal (PC 100A, 100B). When the photographed image is transmitted and shared by the participant's terminal, the state of the portable terminal 150 is changed as shown by 1011 in FIG. A camera switching button 1012 and an image sharing end button 1013 are displayed, and the captured images of the terminals of other participants are displayed.

  In step S520, in the state where the camera imaging pause information is received, when the photographed image is received, the display layout of the terminal (PC 100A, 100B) on the receiving side is changed.

In step S521, the received photographed image is displayed after the layout change.
An example of display after layout change is 1102 in FIG. At 1102, the received image is displayed, and a cache image corresponding to the mobile terminal 150 of the reception source is displayed. In the present embodiment, the cache image of the mobile terminal 150 of the reception source is displayed. This is because the participant intentionally shoots with the out-camera and transmits it, so the images shot by the terminals of other participants are omitted in order to make the image look large. When the display is large, the configuration may not be omitted. Therefore, according to the setting of the user, the photographed images of the terminals of other participants may be arranged and displayed.

  Reference numeral 1102 in FIG. 11 is an example of displaying an image temporarily stored at the electronic conference and displaying an image captured and acquired by switching when the imaging state of the terminal participating in the electronic conference is switched. is there.

  In step S522, it is determined whether the camera has been switched. Specifically, the camera selection screen 912 is displayed by the camera switching button 1012, and it is determined whether the camera used on the camera selection screen has been changed (switched to the in-camera). When the camera is switched to the in-camera, the normal mode is set, and the process returns to step S501. If the camera is not switched to the in-camera, the still image sharing state continues. Note that a still image shooting button 1002 may be arranged in 1011 so that shooting and transmission of a still image can be performed again.

  When the camera is switched, the layout also becomes the normal display 1101. Since the portable terminal 150 no captured image is received on the PC 100A, 100B side, the camera imaging pause information is deleted on the PC 100A, 100B side. In addition, when the camera is switched, control may be performed by transmitting a deletion instruction of camera shooting pause information from the portable terminal 150.

  In step S523, the image sharing end button 1013 is pressed on the portable terminal 150, and an image sharing end notification is transmitted to the server 200. In step S524, the received image sharing end notification is transmitted to each terminal (PC 100A, 100B) of the participant.

  In step S525, when the image sharing end notification is received, the display layout of the web conference is changed (returned to the original). Specifically, a screen 1101 is displayed, and again, the captured image transmitted from the participant's terminal is acquired and displayed in each display area. At this time, the portable terminal 150 is assumed to be automatically switched to the in-camera, and the photographed image acquired by the in-camera is transmitted again via the server 200.

  In addition, since the layout is changed and the pause state of the camera of the portable terminal 150 is canceled, the stored camera shooting pause information is deleted.

  In step S526, when a meeting termination instruction (not shown) is received, a meeting termination request is transmitted to the server 200, and in step S527, the server 200 notifies each terminal of the meeting termination. In step S528, when the notice of the end of the conference is received, the displayed image is hidden (deleted).

  Next, with reference to FIG. 6, the display processing in the one-unit control mode in the present embodiment will be described.

  In FIG. 6, first, one camera is connected, and a default camera (main camera) used in the web conference is set as the built-in camera 1. That is, the embodiment will be described as an embodiment in which there is one camera and no camera to be switched. In the PC 100, as in the case of the portable terminal 150, the specification device information is managed by an external memory or RAM.

  In step S601, shooting is performed by the camera 1233, and in step S602, the captured image is transmitted. This shooting and transmission shall be repeated. That is, the video of the PC 100A is transmitted.

  In step S603, the captured image is received from the PC 100A, terminals of the users participating in the conference (the portable terminal 150 and the PC 100B) are specified, and in step S504, the captured image is transmitted. It is assumed that the terminals of the users participating in the conference can be identified by 701 in FIG. Note that information 701 is managed for each meeting room.

  In step S605, the captured image is received from the server 200, and in step S606, the image is displayed on the display. Examples of display are 901 in FIG. 9 and 1101 in FIG.

  At this time, similarly, the captured image of the participant's terminal is displayed on the portable terminal 150. An example of the display is 901 in FIG.

  In step S 607, the captured image is temporarily stored (cached) in the RAM. Although the captured image stored here is the image displayed most recently, a plurality of images within a predetermined time may be cached. This cached image is used when one camera is virtually switched. Also, the cache image is managed for each participant's terminal.

  In step S608, it is determined whether imaging switching has been performed. The shooting switching is instructed on the shooting switching on the setting screen (1201 in FIG. 12) of the PC 100 </ b> A. An example 1201 indicates a shooting switching instruction. If there is no shooting switch (if cancel is pressed), shooting and transmission of the shot image are repeated. If there is imaging switching (if OK is pressed), the process moves to step S609.

  In step S 609, information (camera shooting pause information) indicating that camera shooting has been paused is transmitted to the server 200. Here, since the camera shooting is paused, control is performed so as not to transmit a shot image shot by the camera 123.

  In step S610, camera imaging pause information is received from the PC 100A, and in step S611, camera imaging pause information is transmitted to the participant's terminal (mobile terminal 150 or PC 100B).

  In step S612, camera imaging pause information is received. The camera imaging pause information is managed in association with the camera imaging paused terminal. This makes it possible to identify which terminal is switching imaging.

  In step S613, the captured image being cached is acquired. This is because each terminal can not receive an image from the server 200 if the photographed image (video) is not sent from the participating party, and the terminal is in a non-display state.

  This process is an example of an acquisition process for acquiring the temporarily stored image when the imaging state of the terminal (PC 100A) participating in the electronic conference is switched.

  In step S614, the acquired captured image (cache image) is displayed in the display area of the target participant. As a result, the web conference can be continued without being undisplayed and alleviating discomfort. In addition, since it looks like it has stopped by using the cache image, it is desirable to notify that “the user: Mr. XX is in the process of camera switching” at a place where the cache image is displayed or the like.

  In step S615, the camera 1 233 continues shooting. At this time, the layout is changed to the same layout as the screen 1001 in FIG. 10 described with reference to FIG. 5, and the image captured by the camera 123 is displayed on the display. Hereinafter, since the screen layout is the same as that of the portable terminal 150, the operation of the PC 100A will be described using FIG.

  In step S616, an operation instruction of the still image shooting button 1002 is received. When an operation instruction (depression) of the still image shooting button 1002 is received, the shot image is transmitted. In addition, in this embodiment, although a still picture is acquired and it explains as an example which transmits to a participant, a form which transmits an animation may be sufficient.

  In step S617, in accordance with the instruction to press the still image shooting button 1002, the captured image is transmitted to the server 200.

  In step S618, the photographed image is received from the PC 100A, and in step S619, the photographed image is transmitted to the participant's terminal (the portable terminal 150 or the PC 100B). When the photographed image is transmitted and shared by the participant's terminal, the PC 100A is changed to the layout display state similar to 1011 in FIG. A camera switching button (shooting switching button) 1012 and an image sharing end button 1013 are displayed, and the captured images of the terminals of other participants are displayed.

  In step S620, in the state where the camera imaging pause information is received, when the photographed image is received, the display layout of the terminal (the portable terminal 150 or the PC 100B) on the receiving side is changed.

In step S621, the received captured image is displayed after the layout change.
An example of display after layout change is 1102 in FIG. At 1102, the received image is displayed, and a cache image corresponding to the PC 100A of the reception source is displayed. In the present embodiment, the cache image of the reception source PC 100A is displayed. This is because the participant intentionally shoots with the out-camera and transmits it, so the images shot by the terminals of other participants are omitted in order to make the image look large. When the display is large, the configuration may not be omitted. Therefore, according to the setting of the user, the photographed images of the terminals of other participants may be arranged and displayed.

  Reference numeral 1102 in FIG. 11 is an example of displaying an image temporarily stored at the electronic conference and displaying an image captured and acquired by switching when the imaging state of the terminal participating in the electronic conference is switched. is there.

  In step S 622, it is determined whether the imaging has been switched. Specifically, it is determined by the camera switching button (shooting switching button) 1012 whether shooting has been changed (changed to normal shooting). When the imaging is switched, the normal mode is set, and the process returns to step S601. If the shooting does not switch, the still image sharing state continues. Note that the still image shooting button 1002 may be disposed, and still images may be photographed and transmitted again.

  In addition, when imaging | photography is switched, layout also becomes a normal display of 1101. FIG. Since the captured image is received on the side of the portable terminal 150 or the PC 100B, the camera imaging pause information is deleted on the side of the portable terminal 150 or the PC 100B. In addition, when the imaging is switched, the control may be performed by transmitting a deletion instruction of camera imaging pause information from the PC 100A.

  In step S623, the image sharing end button 1013 is pressed on the PC 100A, and an image sharing end notification is transmitted to the server 200. In step S624, the received image sharing end notification is transmitted to each participant terminal (mobile terminal 150 or PC 100B).

  In step S625, when the image sharing end notification is received, the display layout of the web conference is changed (returned to the original). Specifically, the screen 901 or 1101 is displayed, and the captured image transmitted from the participant's terminal is acquired again and displayed in each display area.

  In addition, since the layout is changed and the pause state of the camera of the PC 100A is released, the stored camera shooting pause information is deleted.

  In step S626, when a meeting termination instruction (not shown) is received, a meeting termination request is transmitted to the server 200, and in step S527, the server 200 notifies each terminal of the meeting termination. In step S528, when the notice of the end of the conference is received, the displayed image is hidden (deleted).

  In the present embodiment, caching of images is performed at each terminal, but the server 200 may be configured to cache.

  In this case, the camera (shooting) is switched, the camera shooting pause information is sent, and the cached image is sent. This image is used for display to prevent non-display state.

  By caching the images taken, it is possible to display an image when originally participating, even when the camera or the imaging switching is performed, and a smooth web conference can be performed.

  In particular, web conferencing is a remote environment where it is difficult to know what participants are doing. Therefore, when the camera or the imaging switching is changed to a non-display state, the participant who has a problem may misidentify. Therefore, by displaying using the cached image, it is possible to prevent false recognition at the time of switching.

  As described above, according to the present embodiment, it is possible to prevent an erroneous recognition and to create an environment in which an electronic conference can be smoothly advanced even when imaging is switched.

  That is, it is possible to facilitate remote communication (web conference).

  The present invention can also be embodied as, for example, a system, an apparatus, a method, a program, a storage medium, etc. Specifically, the present invention may be applied to a system composed of a plurality of devices, and The invention may be applied to an apparatus consisting of one device.

  Note that the present invention includes one that directly or remotely supplies a program of software that implements the functions of the above-described embodiments to a system or an apparatus. The present invention is also included in the present invention if the computer of the system or apparatus reads out and executes the supplied program code.

  Therefore, the program code itself installed in the computer to realize the functional processing of the present invention by the computer also implements the present invention. That is, the present invention also includes a computer program itself for realizing the functional processing of the present invention.

  Examples of recording media for supplying the program include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, an MO, a CD-ROM, a CD-R, and a CD-RW. There are also magnetic tapes, non-volatile memory cards, ROMs, DVDs (DVD-ROMs, DVD-Rs) and the like.

  In addition, as a program supply method, a browser on a client computer is used to connect to an Internet home page. Then, the program can be supplied by downloading the computer program of the present invention itself or a compressed file including an automatic installation function from the home page to a recording medium such as a hard disk.

  The present invention can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from different home pages. That is, the present invention also includes a WWW server which allows a plurality of users to download program files for realizing the functional processing of the present invention by a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, and distributed to users, and the user who has cleared predetermined conditions downloads key information that decrypts encryption from the homepage via the Internet. Let Then, it is possible to execute the program encrypted by using the downloaded key information and install it on a computer.

  Also, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiment can be realized by the processing.

  Furthermore, the program read from the recording medium is written to a memory provided in a function expansion board inserted in the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program, a CPU or the like provided in the function expansion board or the function expansion unit performs part or all of the actual processing, and the function of the above-described embodiment is also realized by the processing.

The embodiments described above merely show examples of implementation in practicing the present invention, and the technical scope of the present invention should not be interpreted limitedly by these.
That is, the present invention can be implemented in various forms without departing from the technical concept or the main features thereof.

100A PC (client terminal)
100B PC (client terminal)
150 Mobile Terminal (Client Terminal)
200 servers 201 CPUs
203 RAM
211 External memory

Claims (8)

An electronic conference system for displaying an image captured by a terminal,
Receiving means for receiving an image of a terminal participating in the electronic conference;
Display means for displaying the image received by the reception means at an electronic conference;
Temporary storage means for temporarily storing an image to be displayed by the display means;
An acquisition unit configured to acquire an image stored in the temporary storage unit when a photographing state of a terminal participating in the electronic conference is switched;
The electronic conference system, wherein the display means displays the image acquired by the acquisition means in an electronic conference.   The display unit displays the image acquired by the acquisition unit in the electronic conference and displays the image captured and acquired by switching when the imaging state of the terminal participating in the electronic conference is switched. The electronic conference system according to claim 1, wherein The terminal includes an instruction unit that instructs switching of imaging.
3. The electronic conference system according to claim 1, wherein the display unit displays the image acquired by the acquisition unit in an electronic conference in accordance with a switching instruction of photographing by the instruction unit.   The electronic conference system according to claim 2 or 3, wherein the switching of the photographing is switching a plurality of photographing devices. Specifying means for specifying an imaging device available at the terminal;
The electronic conference system according to any one of claims 1 to 4, further comprising: setting means for setting switching control of photographing according to a specific result by the specifying means.   The electronic conference system according to claim 5, wherein the switching control setting of photographing by the setting means is setting of control by plural units or control by one unit. A processing method of an electronic conference system for displaying an image captured by a terminal, comprising:
A receiving step of receiving an image of a terminal participating in the electronic conference;
A display step of displaying the image received in the reception step in an electronic conference;
A temporary storage step for temporarily storing an image to be displayed in the display step;
An acquisition step of acquiring the temporarily stored image when the imaging state of the terminal participating in the electronic conference is switched,
The processing step of displaying the image acquired in the acquisition step in an electronic conference. A program of an electronic conference system for displaying an image captured by a terminal,
The electronic conference system
Receiving means for receiving an image of a terminal participating in the electronic conference;
Display means for displaying the image received by the reception means at an electronic conference;
Temporary storage means for temporarily storing an image to be displayed by the display means;
When the imaging state of the terminal participating in the electronic conference is switched, it functions as an acquisition means for acquiring the image stored in the temporary storage means,
A program for causing the display means to display an image acquired by the acquisition means in an electronic conference.

Images