JP2019118114A - Tv conference system, terminal, and display method - Google Patents

Abstract

To make it possible to simply grasp an output state of the other communication apparatus of data transmitted to the other communication apparatus.SOLUTION: A terminal 10 outputs data received from another terminal 10. The terminal 10 comprises: an output state determination unit 21 for determining a state of outputting data received from the other terminal 10; a notification unit 22 for notifying an external relay server 30 of the output state; and a state display processing unit 23 that acquires, in the course of establishing a session for receiving the data with the other communication apparatus, the output state of the other terminal 10 of data transmitted to the other terminal 10, from the relay server 30 through a management server 40, and causes a display to display output state information showing the acquired output state.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a communication apparatus, a communication system, a communication management system, a communication control method, and a program.

  In a communication system such as a video conference system that realizes a remote conference using a communication network, it is possible to realize smooth communication by determining whether data transmitted from one site to another site is correctly output at another site. It is important to do. From such a point of view, conventionally, there is known a technique of confirming in advance by using a test signal whether or not the voice of the own terminal properly reaches the opposite terminal. However, in this technology, there is a problem that the meeting can not be started promptly because confirmation and adjustment before the meeting are required.

  On the other hand, according to Patent Document 1, whether the video of the own base properly reaches the other base by combining the video of the other base being received with the video taken at the own base during the meeting and transmitting the result to each other A system is disclosed that allows for the identification of In this system, it is possible to confirm the status of the video of the own base displayed at another base during the meeting.

  However, with the technology described in Patent Document 1, since it is necessary to send data received from another location back to another location in some way, communication with the other locations is carried out when the number of locations transmitting and receiving data increases. There is a disadvantage that the network bandwidth used when doing this increases. Therefore, there is a need for a mechanism that can grasp the output state of data by a simpler method.

  In order to solve the problems described above, the present invention is a communication device for outputting data received from another communication device, and a notification for notifying an external information processing device of an output state of data received from another communication device. Unit, and while establishing a session for receiving the data with another communication apparatus, acquires from the information processing apparatus the output state of the data transmitted to the other communication apparatus in the other communication apparatus And a display processing unit that causes the display device to display output state information indicating the output state.

  According to the present invention, the output state of data transmitted to another communication device in another communication device can be easily grasped.

FIG. 1 is a schematic configuration diagram of a video conference system according to the embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the terminal. FIG. 3 is a block diagram showing an example of the hardware configuration of the relay server. FIG. 4 is a block diagram showing an example of a functional configuration of a terminal. FIG. 5 is a diagram showing an example of the output state management table. FIG. 6 is a block diagram showing a functional configuration example of the relay server. FIG. 7 is a diagram showing an example of the all output state management table. FIG. 8 is a block diagram showing an example of a functional configuration of the management server. FIG. 9 is a diagram showing an example of the terminal authentication management table. FIG. 10 is a diagram showing an example of the terminal management table. FIG. 11 is a diagram showing an example of the destination list management table. FIG. 12 shows an example of the session management table. FIG. 13 is a sequence diagram showing an example of the preparation stage operation. FIG. 14 is a sequence diagram showing an example of the conference participation operation. FIG. 15 is a sequence diagram showing an example of data transmission / reception operation between terminals. FIG. 16 is a flowchart illustrating an example of processing procedures by the output state determination unit and the notification unit. FIG. 17 is a sequence diagram showing an example of the operation including the output abnormality notification and the output abnormality recovery notification. FIG. 18 is a view showing a display example of output state information. FIG. 19 is a sequence diagram showing an example of an operation of acquiring and displaying a list of output states. FIG. 20 is a view showing a display example of output state information.

  Embodiments of a communication apparatus, a communication system, a communication management system, a communication control method, and a program according to the present invention will be described in detail with reference to the accompanying drawings. In the following, as an example of a communication system to which the present invention is applied, a video conference system that transmits and receives video data and audio data between a plurality of video conference terminals (corresponding to "communication devices") to realize teleconferencing at multiple sites (Also referred to as "video conferencing system"). However, the communication system to which the present invention is applicable is not limited to this example. The present invention is widely applicable to various communication systems that transmit and receive data between a plurality of communication devices, and various communication terminals used in this communication system.

  FIG. 1 is a schematic block diagram of the video conference system 1 of the present embodiment. As shown in FIG. 1, the video conference system 1 according to the present embodiment manages a plurality of video conference terminals (hereinafter simply referred to as “terminals”) 10 and displays 20 installed at each site, a relay server 30, and management. A server 40 and a program providing server 50 are provided.

  The terminal 10 performs transmission and reception of data with another terminal 10 using the video conference system 1 and output of received data. As data handled here, for example, video data showing each site of a video conference, screen data of a PC (personal computer) shared among the sites, etc. (hereinafter, these are generically referred to as “image data”. And voice data recorded at each site. The terminal 10 may be a dedicated terminal dedicated to the video conference system 1, or may be a general-purpose terminal such as a PC, a smartphone, or a tablet terminal. These general-purpose terminals realize the function of the terminal 10 as one of the applications by installing a terminal program described later.

  The display 20 is connected to the terminal 10 by wire or wirelessly. The display 20 may be integrated with the terminal 10.

  The terminal 10 is connected to, for example, a router in a LAN (Local Area Network). The router is a network device that selects a route for data transmission, and in the example shown in FIG. 1, a router 60a in the LAN 2a, a router 60b in the LAN 2b, a router 60c in the LAN 2c, and a router 60d in the LAN 2d, The router 60e is connected to the router 60a and the router 60b by the leased line 2e and connected to the Internet 2i, and the router 60f connected to the router 60c and the router 60d by the leased line 2f and connected to the Internet 2i.

  In addition, LAN2a and LAN2b are constructed in another place in area X, and LAN2c and LAN2d are assumed to be constructed in another place in area Y. For example, Region X is Japan, Region Y is the United States, LAN 2a is built in a Tokyo office, LAN 2b is built in an Osaka office, LAN 2c is built in a New York office, and LAN 2d is Washington D. C. It is built in the office of In the present embodiment, the communication network 2 is constructed by the LAN 2a, the LAN 2b, the dedicated line 2e, the Internet 2i, the dedicated line 2f, the LAN 2c, and the LAN 2d. The communication network 2 may include not only wired portions but also portions where wireless communication such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed.

  In the video conference system 1 of the present embodiment, transmission and reception of image data and audio data via the relay server 30 are performed between the plurality of terminals 10. At this time, a management information session Sei for transmitting and receiving various management information is established between the plurality of terminals 10 via the management server 40. Further, a data session Sed for transmitting and receiving video data and audio data is established between the plurality of terminals 10 via the relay server 30.

  The relay server 30 is a computer that relays transmission of image data and audio data between the plurality of terminals 10. In addition, the format of data transmission is arbitrary. For example, transmission of image data is described in H.264. H.264 / SVC format may also be used. H.264 / MPEG4 AVC, H.264. Other forms such as 265 may be used.

  The management server 40 is a computer that manages the entire video conference system 1 according to the present embodiment. For example, the management server 40 performs authentication management, state management, session management, and the like of each terminal 10. In the present embodiment, each terminal 10 is assigned a terminal ID which is unique identification information. The terminal ID need not necessarily be identification information unique to the terminal, and may be identification information used to specify the terminal 10 of the communication destination. For example, the identification information input by the user to the terminal may be used. In addition, a relay server ID, which is unique identification information, is assigned to the relay server 30. Also, a session ID is assigned to a data session Sed for transmitting and receiving data between the terminals 10, and a data ID is assigned to each of data transmitted and received in the data session Sed. The management server 40 performs the various management described above using these IDs (identification information).

  The program providing server 50 is a computer that provides various programs to the terminal 10, the relay server 30, the management server 40, and the like. For example, the program providing server 50 stores a terminal program for causing the terminal 10 to realize various functions, and can transmit the terminal program to the terminal 10. Further, the program providing server 50 stores a relay server program for causing the relay server 30 to realize various functions, and can transmit the relay server program to the relay server 30. Further, the program providing server 50 stores a management server program for causing the management server 40 to realize various functions, and can transmit the management server program to the management server 40.

  Next, the hardware configuration of the terminal 10, the relay server 30, and the management server 40 in the video conference system 1 according to the present embodiment will be described. FIG. 2 shows an example of the hardware configuration of the terminal 10, and FIG. 3 shows an example of the hardware configuration of the relay server 30. As shown in FIG. Since the management server 40 can adopt the same hardware configuration as the relay server 30, illustration is omitted.

  As shown in FIG. 2, the terminal 10 is a ROM (Read Only Memory) that stores programs used to drive the CPU 101 such as a central processing unit (CPU) 101 that controls the overall operation of the terminal 10 and an initial program loader (IPL). 102, RAM (Random Access Memory) 103 used as a work area of CPU 101, flash memory 104 for storing various data such as programs for terminals, image data and audio data, various data to flash memory 104 under control of CPU 101). Data to a recording medium 106 such as a solid state drive (SSD) 105 for controlling reading and writing, flash memory, etc. Media drive 107 for controlling reading and writing (storage) of the device, an operation button 108 operated when selecting another terminal 10 as a communication partner, and a power switch 109 for switching on / off the power of the terminal 10 , And a network I / F (Interface) 111 for data transmission using the communication network 2.

  The terminal 10 also has a built-in camera 112 that captures an object and obtains image data according to control of the CPU 101, an imaging device I / F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs sound, and sound A built-in speaker 115 for outputting the sound, an audio input / output I / F 116 for processing input / output of an audio signal between the microphone 114 and the speaker 115 under the control of the CPU 101, and image data to be displayed on the display 20 under the control of the CPU 101 Display I / F 117, an external device connection I / F 118 for connecting various external devices, an alarm lamp 119 for notifying abnormality of various functions of the terminal 10, and an address bus for electrically connecting the above-mentioned respective components And a bus line 110 such as a data bus.

  The camera 112, the microphone 114, and the speaker 115 do not necessarily have to be built in the terminal 10, and may be externally attached. The display 20 may be built in the terminal 10. Although the display 20 is assumed to be a display device such as a liquid crystal panel, the present invention is not limited to this, and may be a projection device such as a projector. The hardware configuration of the terminal 10 shown in FIG. 2 is merely an example, and hardware other than the above may be added.

  The terminal program provided by the program providing server 50 described above is stored, for example, in the flash memory 104, read out on the RAM 103 under the control of the CPU 101, and executed. The memory for storing the terminal program may be a non-volatile memory, and is not limited to the flash memory 104, and an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used. Also, the terminal program may be provided by being recorded in a recording medium such as the computer readable recording medium 106 by a file in an installable format or an executable format. The terminal program may be provided as an embedded program stored in advance in the ROM 102 or the like.

  As shown in FIG. 3, the relay server 30 controls the entire operation of the relay server 30. The ROM 202 stores programs used to drive the CPU 201 such as IPL. The RAM 203 used as a work area for the CPU 201. Programs such as HD (Hard Disk) 204 for storing various data such as HDD, HDD (HD Drive) 205 for controlling reading and writing of various data to HD 204 under the control of CPU 201, reading data from recording media 206 such as flash memory A media drive 207 for controlling writing (storage), a display 208 for displaying various information, a network I / F 209 for data transmission using the communication network 2, a keyboard 211, A mouse 212, a CD-ROM drive 214 for controlling reading and writing of various data to a compact disc read only memory (CD-ROM) 213 as an example of a removable recording medium, and the above-described components are electrically connected. A bus line 210 such as an address bus or a data bus is provided.

  The relay server program provided from the program providing server 50 described above is stored, for example, in the HD 204, read out on the RAM 203 under the control of the CPU 201, and executed. Also, the relay server program may be provided by being recorded in a computer readable recording medium such as the recording medium 206 or the CD-ROM 213 by a file in an installable format or an executable format. Further, the relay server program may be provided as an embedded program stored in advance in the ROM 202 or the like.

  The management server 40 can adopt the same hardware configuration as the relay server 30 shown in FIG. However, the management server program provided by the program providing server 50 is recorded in the HD 204. Also in this case, the management server program may be provided by being recorded in a computer readable recording medium such as the recording medium 206 or the CD-ROM 213 by a file in an installable format or an executable format. Good. Also, the management server program may be provided as an embedded program stored in advance in the ROM 202 or the like.

  In addition, as another example of the removable recording medium, a computer-readable recording medium such as a compact disc recordable (CD-R), a digital versatile disc (DVD), and a Blu-ray disc may be mentioned. The various programs described above may be recorded and provided on these recording media.

  Next, the functional configuration of the terminal 10 will be described. FIG. 4 is a block diagram showing a functional configuration example of the terminal 10, and FIG. 5 is a view showing an example of an output state management table held by the terminal 10. As shown in FIG.

  As shown in FIG. 4, the terminal 10 includes a transmitting / receiving unit 11, an operation input receiving unit 12, a login request unit 13, an image input unit 14, an audio input unit 15, a display control unit 16, an audio output unit 17, storage and readout processing. A unit 18, an output state determination unit 21, a notification unit 22, and a state display processing unit 23 are included. Each of these units is, for example, a function realized by the CPU 101 executing a terminal program developed on the RAM 103 from the flash memory 104 shown in FIG. The terminal 10 further includes, for example, a storage unit 1000 configured by the RAM 103 and the flash memory 104 illustrated in FIG. 2.

  The storage unit 1000 stores a terminal ID and a password for identifying the terminal 10. The storage unit 1000 is also used as a reception buffer for image data and audio data received when performing a video conference with another terminal 10.

  Further, in the storage unit 1000, an output state management DB 1100 having an output state management table as shown in FIG. 5 is constructed. The output state management table manages a list of data shared with other terminals 10 in a conference in which the terminal 10 is currently participating. For example, in the output state management table, data ID for identifying the data, final output time and current output state are associated with each data shared with the other terminals 10, and managed. It is done.

  The final output time is the time when the terminal 10 finally outputs the data. For example, in the case of image data, the time at which the display control unit 16 performed the last processing for displaying on the display 20, and in the case of audio data, the audio output unit 17 performs processing for reproducing and outputting audio from the speaker 115. The time when it went to is the final output time. The time may be the system time counted in the terminal 10.

  The current output state indicates the current output state of the data by the terminal 10. The output status is whether the terminal 10 can perform output processing normally for the corresponding data (output normal) or is an abnormal status (output abnormality) where output is about to be output but output processing can not be performed correctly for some reason, It indicates whether the terminal 10 has selected not to output the corresponding data (output stop). Here, “normal” means that the output of the data is not delayed, and “abnormal” means that the output of the data is delayed for a predetermined time or more.

  The transmitting and receiving unit 11 transmits and receives various data to and from the relay server 30 and the management server 40 via the communication network 2. The transmission / reception unit 11 is realized by, for example, the network I / F 111 and the CPU 101 illustrated in FIG.

  The operation input receiving unit 12 receives various input operations by the user who uses the terminal 10. The operation input reception unit 12 is realized by, for example, the operation button 108, the power switch 109, and the CPU 101 illustrated in FIG.

  The login request unit 13 transmits the login request information and the IP address of the own terminal to the management server 40 from the transmission / reception unit 11 via the communication network 2 in response to the on operation of the power switch 109. The login request unit 13 is realized by, for example, the CPU 101 illustrated in FIG.

  The image input unit 14 inputs image data such as video data of the own site taken by the camera 112 and screen data of a PC shared with another terminal 10. The image input unit 14 is realized by, for example, the imaging device I / F 113, the external device connection I / F 118, and the CPU 101 illustrated in FIG.

  The voice input unit 15 inputs voice data of the own location acquired by the microphone 114. The voice input unit 15 is realized by, for example, the voice input / output I / F 116 and the CPU 101 shown in FIG.

  The display control unit 16 performs a drawing process or the like for displaying the image data received by the transmission / reception unit 11 as one television conference screen, and passes the processed image data to the display 20 to display the television conference on the display 20. Display the screen. In addition, when a display request for output state information is made from the state display processing unit 23 described later, the display control unit 16 controls the display 20 to display the output state information. The display control unit 16 is realized by, for example, the CPU 101 and the display I / F 117 illustrated in FIG.

  The display control unit 16 also has the function of a time management unit 16a. Each time the time management unit 16a performs processing for causing the display 20 to display each of the image data received by the transmission / reception unit 11, the output state management described above takes the time (system time) at that time as the final output time. Do the process of writing to the table.

  The audio output unit 17 performs a process of reproducing and outputting the audio data received by the transmitting and receiving unit 11 from the speaker 115. The voice output unit 17 is realized by, for example, the voice input / output I / F 116 and the CPU 101 shown in FIG.

  Further, the audio output unit 17 has a function of the time management unit 17a. Each time the time management unit 17a performs the process of causing the speaker 115 to reproduce and output each of the audio data received by the transmission / reception unit 11, the output state management table described above takes the time (system time) at that time as the final output time. Process to write to

  The storage / read processing unit 18 performs processing of storing and reading various data in the storage unit 1000. The storage / read processing unit 18 is realized by, for example, the SSD 105 and the CPU 101 illustrated in FIG.

  The output state determination unit 21 determines the output state by confirming the final output time of the output state management table described above for each of the image data and audio data being output by the terminal 10. Further, the notification unit 22 performs processing of notifying the relay server 30 of the output state determined by the output state determination unit 21. For example, when the output state determination unit 21 detects data in the output abnormal state, the notification unit 22 performs processing to notify the relay server 30 to that effect. In addition, when the output state determination unit 21 detects data restored from the output abnormal state to the output normal state by the output state determination unit 21, the notification unit 22 notifies the relay server 30 to that effect. In addition, about the specific example of a process of the output state determination part 21 and the notification part 22, a detail is mentioned later. The output state determination unit 21 is realized by, for example, the CPU 101 illustrated in FIG. The notification unit 22 is realized by, for example, the network I / F 111 and the CPU 101 illustrated in FIG.

  The state display processing unit 23 performs processing for causing the display 20 to display output state information indicating the output state of the data transmitted by the terminal 10 to another terminal 10 in the other terminal 10. For example, the state display processing unit 23 indicates that there is an output abnormality in another terminal 10 when an output abnormality notification indicating that the image data or the voice data is in an output abnormality state is received by the other terminal 10. Generate output state information. In addition, when the status display processing unit 23 receives an output abnormality recovery notification indicating that the image data and voice data that were in the output abnormality state in the other terminal 10 have recovered to the normal state, the state display processing unit 23 operates in the other terminal 10. Output status information is generated to indicate that the output abnormality of has been recovered. Then, the state display processing unit 23 passes the output state information to the display control unit 16 to make a display request of the output state information, and causes the display 20 to display the output state information as a pop-up screen on the television conference screen, for example See Figure 18).

  Further, the state display processing unit 23 requests, for example, a list of the output states of other data of each data being transmitted from the terminal 10 to another terminal 10 in response to the user operating the operation button 108 or the like. . Then, in response to the request, when receiving the list of output states provided from the relay server 30 via the management server 40, the state display processing unit 23 uses the data to select each site participating in the video conference. Output state information indicating a list of output states in the terminal 10 is generated. Then, the state display processing unit 23 passes the output state information to the display control unit 16 to make a display request for the output state information, and causes the display 20 to display the output state information as a participating base list screen (see FIG. 20). .

  Next, the functional configuration of the relay server 30 will be described. FIG. 6 is a block diagram showing a functional configuration example of the relay server 30, and FIG. 7 is a diagram showing an example of a total output state management table held by the relay server 30. As shown in FIG.

  As shown in FIG. 6, the relay server 30 includes a transmission / reception unit 31, a storage / read processing unit 32, and an output state management unit 33. Each of these units is, for example, a function realized by the CPU 201 executing the relay server program expanded on the RAM 203 from the HD 204 shown in FIG. 3. Further, the relay server 30 includes, for example, a storage unit 3000 configured by the HD 204 illustrated in FIG. 3.

  In the storage unit 3000, an all-output state management DB 3100 having an all-output state management table as shown in FIG. 7 is constructed. In the all output state management table, a list of output states of all data relayed by the relay server 30 is managed. For example, in the all output state management table, for each data relayed by the relay server 30, the data ID for identifying the data and the terminal 10 in which the data is in the output abnormality state (output It is managed in association with the IP address of the abnormal terminal).

  The transmission / reception unit 31 transmits / receives various data to / from each terminal 10 or the management server 40 via the communication network 2. The transmission / reception unit 31 is realized by, for example, the network I / F 209 and the CPU 201 illustrated in FIG. 3.

  The storage / read processing unit 32 performs processing of storing and reading various data in the storage unit 3000. The storage / read processing unit 32 is realized by, for example, the HDD 205 and the CPU 201 illustrated in FIG. 3.

  The output state management unit 33 manages the output states of all the data relayed by the relay server 30 using the all output state management table described above. Specifically, the output state management unit 33 receives a notification (output abnormality notification) indicating that certain data is in the abnormal state of output from each terminal 10, or some data is in the state of normal output from the state of abnormal output When receiving a notification of recovery (output abnormal recovery notification), the above-mentioned all output state management table is corrected according to the received notification, and an error occurs in which data output at which terminal 10 at present. Manage the

  Further, when receiving the output abnormality notification from the terminal 10, the output state management unit 33 causes the terminal 10 of the transmission source of the data in the output abnormality state to perform the display processing of the output state information. The management server 40 is notified of an output abnormality notification to which 10 IP addresses have been added. Also, when receiving the output state list request from the terminal 10, the output state management unit 33 uses the all output state management table described above to cause the terminal 10 to perform the display processing of the output state information. , And adds the IP address of the terminal 10 that has made the output state list request and passes it to the management server 40.

  Next, the functional configuration of the management server 40 will be described. FIG. 8 is a block diagram showing a functional configuration example of the management server 40. As shown in FIG. 9 to 12 are diagrams showing an example of a terminal authentication management table, a terminal management table, a destination list management table, and a session management table held by the management server 40, respectively.

  As shown in FIG. 8, the management server 40 includes a transmitting / receiving unit 41, a terminal authentication unit 42, a state management unit 43, a terminal extraction unit 44, a terminal state acquisition unit 45, a session management unit 46, and a storage / read processing unit 47. doing. These units are, for example, functions realized by the CPU 201 executing the management server program developed on the RAM 203 from the HD 204 shown in FIG. The management server 40 further includes, for example, a storage unit 4000 configured by the HD 204 illustrated in FIG. 3.

  In the storage unit 4000, a terminal authentication management DB 4100 having a terminal authentication management table as shown in FIG. 9 is constructed. In the terminal authentication management table, passwords are associated with terminal IDs of all the terminals 10 managed by the management server 40 and managed. For example, the terminal authentication management table shown in FIG. 9 indicates that the password of the terminal 10 whose terminal ID is "01aa" is "aaaa".

  Further, in the storage unit 4000, a terminal management DB 4200 having a terminal management table as shown in FIG. 10 is constructed. In the terminal management table, for each terminal ID of each terminal 10, the display name of each terminal 10 (hereinafter referred to as "terminal name"), the operating state of each terminal 10, and the login for logging in to the video conference system 1 The reception date and time when the request information was received by the management server 40 and the IP address of the terminal 10 are managed in association with each other.

  For example, in the terminal management table shown in FIG. 10, the terminal 10 with the terminal ID "01aa" has the terminal name "Japan Tokyo Office AA terminal" and the operating status is "Online (transmittable)" The date when the login request information was received by the server 40 is “13:40 on November 10, 2009”, and the IP address is “1.2.1.3”.

  Here, the terminal ID of the terminal management table is uniquely assigned, for example, when the terminal 10 is registered in the video conference system 1. The terminal name is registered or changed by the manager of the management server 40. The terminal name may be registered or changed in response to a request from the terminal 10 to the management server 40.

  Further, the operating state "online (transmittable)" indicates that the terminal 10 is in a state of not participating in a video conference in a state where the terminal 10 is logged in to the video conference system 1, and "online (transmitting)" Indicates that the terminal 10 is in a state of participating in a video conference in a state where the terminal 10 is logged in to the video conference system 1, and the operation state is “offline” means that the terminal 10 is logged in to the video conference system 1 It represents that there is no condition.

  Further, in the storage unit 4000, a destination list management DB 4300 having a destination list management table as shown in FIG. 11 is constructed. In the destination list management table, the terminal ID of the terminal 10 (request source terminal) requesting the start of the video conference is registered as a candidate of another terminal 10 (destination terminal) participating in the video conference. All terminal IDs are associated and managed.

  For example, in the destination list management table shown in FIG. 11, when the start of a video conference is requested from the terminal 10 with the terminal ID "01ab", the destination terminal candidate is the terminal 10 with the terminal ID "01aa", the terminal ID It is shown that the terminal 10 is "01ca" and the terminal 10 whose terminal ID is "01cb". Candidates of destination terminals are added or deleted according to a request made from the terminal 10 to the management server 40.

  Further, in the storage unit 4000, a session management DB 4400 having a session management table as shown in FIG. 12 is constructed. In the session management table, a relay server ID for identifying the relay server 30 used for relaying data, a terminal ID of a request source terminal, and a destination for each session ID assigned to a data session Sed for transmitting and receiving data. A terminal ID of a terminal and a data ID identifying data transmitted and received in the data session Sed are associated and managed.

  For example, in the session management table shown in FIG. 12, in the data session Sed with session ID “se1,” the request source terminal with the terminal ID “01aa” and the destination terminal with the terminal ID “01 db” are used. It shows that data of data ID “RS100” and data of data ID “RS101” are transmitted and received via relay server 30 of relay server ID “111a”.

  The transmitting and receiving unit 41 transmits and receives various data to and from each of the terminals 10 and the relay server 30 via the communication network 2. The transmission / reception unit 41 is realized by, for example, the network I / F 209 and the CPU 201 illustrated in FIG. 3.

  The terminal authentication unit 42 searches the terminal authentication management DB 4100 of the storage unit 4000 using the terminal ID and password included in the login request information from the terminal 10 received via the transmission / reception unit 41 as a search key, and manages the terminal authentication. Terminal authentication is performed by determining whether the same terminal ID and password are managed in the DB 4100. The terminal authentication unit 42 is realized by, for example, the CPU 201 illustrated in FIG. 3.

  The state management unit 43 associates the terminal management DB 4200 of the storage unit 4000 with the terminal ID of the terminal 10 in order to manage the operation state of the terminal 10 as the transmission source of the login request information. Stores the date and time when the login request information was received and the IP address. Also, the state management unit 43 operates the terminal 10 in the terminal management DB 4200 based on the power-off state information sent from the terminal 10 by switching the state of the power switch 109 of the terminal 10 from on to off. Change the status from online to offline. The state management unit 43 is realized by, for example, the CPU 201 illustrated in FIG.

  The terminal extraction unit 44 searches the destination list management DB 4300 of the storage unit 4000 using the terminal ID of the terminal 10 that is the transmission source of the login request information as a search key, and when the terminal 10 becomes a request source terminal The terminal ID of the terminal 10 is extracted. Also, the terminal extraction unit 44 searches the destination list management DB 4300 using the terminal ID of the terminal 10 as the transmission source of the login request information as a search key, and the terminal ID of the terminal 10 is registered as a candidate of the destination terminal. The terminal ID of the terminal 10 is also extracted. The terminal extraction unit 44 is realized by, for example, the CPU 201 illustrated in FIG.

  The terminal state acquisition unit 45 searches the terminal management DB 4200 of the storage unit 4000 using the terminal ID extracted by the terminal extraction unit 44 as a search key, and reads out the operation state for each detected terminal ID. Thus, the terminal state acquisition unit 45 can acquire the operating states of all the terminals 10 that are candidates for the destination terminal with respect to the terminal 10 that is the transmission source of the login request information. The terminal state acquisition unit 45 also searches the terminal management DB 4200 using the terminal ID of the terminal 10 that is the transmission source of the login request information as a search key, and acquires the operating state of the terminal 10. The terminal state acquisition unit 45 is realized by, for example, the CPU 201 illustrated in FIG.

  When establishing a data session Sed between the terminals 10 via the relay server 30, the session management unit 46 assigns a session ID to the data session Sed. The session management unit 46 stores the terminal ID of the request source terminal, the terminal ID of the destination terminal, and the data ID of the data to be transmitted / received in the session management DB 4400 of the storage unit 4000 in association with the session ID. The session management unit 46 is realized by, for example, the CPU 201 illustrated in FIG.

  Next, the operation of the video conference system 1 according to the present embodiment will be described. FIG. 13 is a sequence diagram showing an example of the preparatory stage operation until the terminal 10 becomes ready to participate in a video conference. FIG. 13 shows an example in which the terminal 10aa is a request source terminal, the terminal 10ba and the terminal 10db are destination terminals, and the request source terminal 10aa is in a preparation state. In this preparation stage operation, various information is transmitted and received in the management information session Sei between the terminals 10aa, 10ba, 10db and the management server 40.

  First, in response to the operation of the power switch 109 of the terminal 10aa, the power of the terminal 10aa is turned on (step S11). Then, the login request unit 13 of the terminal 10aa sends login request information representing a login request to the management server 40 from the transmitting / receiving unit 11 via the communication network 2 when the power is turned on (step S12). Note that the timing of transmitting the login request information may be triggered by an instruction input of the user of the terminal 10.

  The login request information includes the terminal ID and the password of the request source terminal 10aa. The terminal ID and the password are read from the storage unit 1000 by the storage / read processing unit 18. The terminal ID and the password may be information read from an external recording medium such as a SIM card (Subscriber Identity Module Card) connected to the terminal 10. When the login request information transmitted from the terminal 10aa is received by the management server 40, the management server 40 can grasp the IP address of the terminal 10aa that is the transmission side.

  Next, the terminal authentication unit 42 of the management server 40 searches the terminal authentication management DB 4100 of the storage unit 4000 using the terminal ID and the password included in the login request information received via the transmission / reception unit 41 as a search key, Terminal authentication is performed by determining whether the terminal ID and password identical to the search key are managed in the terminal authentication management DB 4100 (step S13).

  Here, when it is determined by the terminal authentication unit 42 that the terminal ID and the password identical to the search key are not managed, that is, when it is determined that the login request is not from the terminal 10 having a valid usage right. The transmission / reception unit 41 transmits the authentication result information representing the authentication result obtained by the terminal authentication unit 42 to the terminal 10aa that has made the login request via the communication network 2, and the operation of the preparation stage ends.

  On the other hand, when it is determined by the terminal authentication unit 42 that the same terminal ID and password as the search key are managed, that is, when it is determined that the login request is from the terminal 10 having a legitimate usage right. The state management unit 43 stores the terminal ID of the terminal 10aa, the operating state, the date and time when the login request information was received, and the IP address of the terminal 10aa in association with each other in the terminal management DB 4200 (step S14).

  Next, the transmission / reception unit 41 of the management server 40 transmits the authentication result information indicating the authentication result obtained by the terminal authentication unit 42 to the terminal 10aa that has made the login request via the communication network 2 (step S15).

  Next, the terminal extraction unit 44 of the management server 40 searches the destination list management DB 4300 using the terminal ID of the terminal 10aa that has made the login request as a search key, and extracts the terminal IDs of candidate destination terminals for the terminal 10aa S16).

  Next, the terminal state acquisition unit 45 of the management server 40 searches the terminal management DB 4200 using the terminal ID of the destination terminal candidate extracted by the terminal extraction unit 44 as a search key, and the terminal extracted by the terminal extraction unit 44 By reading out the operating states “offline”, “online (transmittable)”, and “online (in transmission)” for each ID, each operating state of each terminal 10 which is a candidate of the destination terminal is acquired (step S17).

  Next, the transmitting and receiving unit 41 of the management server 40 operates the terminal ID used as the search key in step S17 and the operating states "offline", "online (transmittable)", and "online" of each terminal 10 corresponding to each. The destination state information including “during transmission” is transmitted to the terminal 10 aa via the communication network 2 (step S 18). As a result, the terminal 10aa can grasp the current operation statuses "offline", "online (transmittable)", and "online (in transmission)" of the respective terminals 10 which are candidates for the destination terminal.

  The terminal extraction unit 44 of the management server 40 searches the destination list management DB 4300 using the terminal ID of the terminal 10aa that has made the login request as a search key, and registers the terminal ID of the terminal 10aa as a candidate of the destination terminal. The terminal ID of the terminal 10 is extracted (step S19).

  Next, the state management unit 43 of the management server 40 searches the terminal management DB 4200 using the terminal ID of the terminal 10aa that has made the log-in request as a search key, and the operating state “online (transmittable available) of the terminal 10aa that has made the log-in request ) Is acquired (step S20).

  Next, of the terminals 10 identified by the terminal ID extracted in step S19, the transmission / reception unit 41 of the management server 40 has a terminal whose operation status is “online” in the terminal management DB 4200 (here, the terminal 10ba , 10 db) transmit destination state information including the terminal ID of the terminal 10 aa obtained in step S 20 and the operation state “online (transmittable)” (step S 21-1, step S 21-2). When transmitting destination status information to the terminals 10ba and 10db, the transmission / reception unit 41 refers to the IP address of the terminal managed in the terminal management table shown in FIG. 10 based on the terminal ID of the terminals 10ba and 10db. Do.

  With respect to the terminals 10ba and 10db that can transmit and receive data to and from the terminal 10aa that has made a log-in request by the above preparatory stage operation, the terminal ID of the terminal 10aa that has made a log-in request and the operation status “Online (transmittable) I can tell. In addition, when step S18 of the preparatory stage operation is completed, the display control unit 16 of the terminal 10aa determines from the information on the destination list displayed on the display 20 that the terminal participating in the video conference, that is, the operating state is “online Allowing the user to recognize the terminal 10 participating in the video conference or the terminal 10 capable of participating in the video conference by highlighting the information of the terminal which is (in transmission) or “online (transmission possible)” on the display 20 or the like. Can.

  FIG. 14 is a sequence diagram showing an example of a conference participation operation until the terminal 10 is in a joining state to participate in a video conference. FIG. 14 shows an example in which the terminal 10aa participates in a video conference with the terminal 10db. In the conference participation operation, various types of information are transmitted and received in the management information session Sei between the terminals 10aa and 10db and the management server 40.

  First, when the user of the terminal 10aa operates the operation button 108 to select participation in a video conference, the operation input receiving unit 12 of the terminal 10aa receives a request to participate in a video conference with the terminal 10db (step S31). In response to this request, the transmission / reception unit 11 of the terminal 10aa transmits participation request information including the terminal ID of the terminal 10aa and the terminal ID of the terminal 10db and indicating participation in the video conference to the management server 40 (step S32) ). Thereby, the transmission / reception unit 41 of the management server 40 can confirm the IP address of the terminal 10aa that is the transmission source while receiving the participation request information.

  Next, the state management unit 43 of the management server 40 searches the terminal management DB 4200 using the terminal ID of the terminal 10aa included in the participation request information as a search key, and the operating state of the terminal 10aa that has requested participation is "online (transmission Medium) is set (step S33).

  Next, the session management unit 46 of the management server 40 generates a session ID for identifying a data session Sed for transmitting and receiving data between the terminal 10aa and the terminal 10db (step S34). Then, the session management unit 46 causes the session management DB 4400 to store the session ID generated in step S34, the relay server ID of the relay server 30 to be used, the terminal IDs of the terminals 10aa and 10db, and the data ID for identifying the data to be transmitted / received. It associates and stores (step S35).

  Next, the transmission / reception unit 41 of the management server 40 transmits start request information including the terminal ID of the terminal 10aa to the terminal 10db via the communication network 2 (step S36). As a result, the terminal 10 db can grasp which terminal 10 has made a request for participation in the video conference. The terminal 10 db that has received the start request information from the management server 40 transmits reception completion information indicating that the reception of the start request information has been completed to the management server 40 from the transmission / reception unit 11 via the communication network 2 (step S37 ).

  Next, the transmission / reception unit 41 of the management server 40 transmits relay start request information indicating a request to start relaying data to the relay server 30 via the communication network 2 (step S38). The relay start request information includes the IP addresses of the terminal 10aa and the terminal 10db. Receiving this relay start request information, the relay server 30 establishes a data session Sed for transmitting and receiving data between the terminal 10aa and the terminal 10db (step S39). Thereby, the terminal 10aa can participate in the video conference with the terminal 10db.

  In addition, the transmission / reception unit 41 of the management server 40 broadcasts terminal state change information including the terminal ID of the terminal 10 aa and the operating state “online (in transmission)” to each of the terminals 10 registered in the terminal management DB 4200. (Step S40). Thereby, each terminal 10 having received the terminal state change information can store and display the operating state of the terminal 10 aa as necessary.

  FIG. 15 is a sequence diagram showing an example of the data transmission / reception operation between the terminals 10. As shown in FIG. FIG. 15 shows an example of transmitting and receiving data between the terminal 10aa and the terminal 10db. In this data transmission and reception operation, image data and audio data are transmitted and received in a data session Sed between the terminal 10aa and the terminal 10db.

  When the data session Sed is established between the terminal 10 aa and the terminal 10 db as described with reference to FIG. 14, the transmitting and receiving unit 11 of the terminal 10 aa receives the image data input to the image input unit 14 and the voice input unit 15. And the voice data input to the relay server 30 are transmitted to the relay server 30 via the communication network 2 (step S51).

  The relay server 30, which has received the image data and the voice data from the terminal 10aa, transmits the received image data and the voice data to each of the terminals participating in the video conference (here, the terminals 10aa and 10db) (step S52-1) , Step S52-2).

  Each of the terminals 10aa and 10db receiving the image data and the voice data from the relay server 30 causes the display control unit 16 to display the image represented by the received image data on the display 20, and outputs the voice represented by the received voice data The reproduction is output from the speaker 115 by the speaker 17 (step S53-1, step S53-2). At this time, each time the time management unit 16a of the display control unit 16 performs a process for displaying the image data on the display 20, the output state management of the output state management DB 1100 described above is performed with the time at that time as the final output time. A process of writing in a table (see FIG. 5) is performed. Further, each time the time management unit 17a of the audio output unit 17 performs processing for reproducing and outputting audio data from the speaker 115, the time management unit 17a performs processing for writing the time at that time in the output state management table as the final output time.

  Next, the transmitting and receiving unit 11 of the terminal 10 db transmits the image data input to the image input unit 14 and the voice data input to the voice input unit 15 to the relay server 30 via the communication network 2 (step S54).

  The relay server 30, which has received the image data and the voice data from the terminal 10db, transmits the received image data and the voice data to each of the terminals participating in the video conference (here, the terminals 10aa and 10db) (step S55-1) , Step S55-2).

  Each of the terminals 10aa and 10db receiving the image data and the voice data from the relay server 30 causes the display control unit 16 to display the image represented by the received image data on the display 20, and outputs the voice represented by the received voice data The reproduction is output from the speaker 115 by the speaker 17 (step S56-1, step S56-2). At this time, each time the time management unit 16a of the display control unit 16 performs a process for displaying the image data on the display 20, the output state management of the output state management DB 1100 described above is performed with the time at that time as the final output time. A process of writing in a table (see FIG. 5) is performed. Further, each time the time management unit 17a of the audio output unit 17 performs processing for reproducing and outputting audio data from the speaker 115, the time management unit 17a performs processing for writing the time at that time in the output state management table as the final output time.

  As described above, the terminals 10 aa and 10 db can perform a video conference by transmitting and receiving image data and audio data via the relay server 30. Although in FIG. 15 the terminals 10 participating in the video conference are two terminals 10 aa and 10 db in order to simplify the description, there are three or more terminals 10 participating in the video conference The image data and the voice data are also transmitted and received between the terminals 10 as in the data transmission and reception operation shown in FIG.

  Next, a specific example of processing by the output state determination unit 21 and the notification unit 22 of the terminal 10 will be described with reference to FIG. FIG. 16 is a flowchart illustrating an example of processing procedures by the output state determination unit 21 and the notification unit 22.

  The output state determination unit 21 refers to the output state management table (see FIG. 5) of the output state management DB 1100 described above, and among the image data and audio data shared in the video conference in which the terminal 10 participates, The output state is determined for each of the data being output by the terminal 10 (data whose output state is not "output stop"), and the output state management table is updated as necessary. The notification unit 22 notifies the relay server 30 when there is a change in the output state.

  As shown in FIG. 16, the processing routine executed by the output state determination unit 21 and the notification unit 22 includes a first loop process (step S100) and a second loop process (step S200) subsequent thereto. . The output state determination unit 21 and the notification unit 22 repeatedly execute the routine including the two loop processes at predetermined timing (for example, every 5 seconds).

  In the first loop processing (step S100), among the data managed by the output state management table of the output state management DB 1100, data whose current output state is "output normal" (hereinafter, such as The data will be described as "data A".) The following steps S101 to S104 are executed.

  First, the output state determination unit 21 checks the final output time of the data A with reference to the output state management table, and determines whether the elapsed time from that time to the present time exceeds the threshold T1 (step S101). . Then, when the elapsed time from the final output time does not exceed the threshold T1 (step S101: No), the output state of the data A remains “output normal”, so the determination process of the next data A ( It moves to step S101).

  On the other hand, when it is determined in step S101 that the elapsed time from the final output time has exceeded the threshold T1 (step S101: Yes), the output state determination unit 21 determines that the recovery counter used to determine the recovery of the output state is 0. (Step S102), and the output state of the data A in the output state management table is changed from "output normal" to "output abnormal" (step S103). Then, the notification unit 22 notifies the relay server 30 that the output state of the data A has become "output abnormal" (step S104). The output abnormality notification includes the data ID of the data A whose output state is “output abnormality” and the IP address of the terminal 10.

  Thereafter, if there is data A for which the first loop process (step S100) has not been performed, the process returns to step S101 and the subsequent processes are repeated, and the first loop process (step S100) ends for all data A Then, the process proceeds to the second loop process (step S200).

  In the second loop process (step S200), among the data managed by the output state management table of the output state management DB 1100, data in which the current output state is "output abnormal" (hereinafter, such as The data will be described as “data B”.) The following steps S201 to S206 are executed.

  First, the output state determination unit 21 checks the final output time of the data B with reference to the output state management table, and determines whether the elapsed time from that time to the present is less than the threshold T2 (step S201). . The threshold value T2 at this time is preferably smaller than the threshold value T1 used in the determination processing in step S101, and can be determined with certainty that the data output state has recovered. When the elapsed time from the final output time is not less than the threshold T2 (step 201: No), the output state determination unit 21 clears the recovery counter to 0 (step S202), and determines the next data B ( It moves to step S201).

  On the other hand, if it is determined in step S201 that the elapsed time from the final output time is less than the threshold T2 (step S201: Yes), the output state determination unit 21 increments the recovery counter by 1 (increment) (step S203) It is determined whether the value of the return counter is equal to or greater than a threshold N (step S204). Then, if the value of the return counter is less than the threshold N (step S204: No), the recovery of the output state may be temporary, so the process proceeds to the determination process of the next data B (step S201). .

  On the other hand, when it is determined in step S204 that the value of the return counter is equal to or greater than the threshold N (step S204: Yes), the output state determination unit 21 outputs the output state of the data B in the output state management table as “output abnormality”. To “output normal” (step S205). Then, the notification unit 22 notifies the relay server 30 that the output state of the data B has recovered from “output abnormal” to “output normal” (step S206). The output abnormality recovery notification includes the data ID of the data B whose output state has been recovered to “output normal”, and the IP address of the terminal 10.

  Thereafter, if there is data B for which the second loop process (step S200) has not been performed, the process returns to step S201, and the subsequent processes are repeated, and the second loop process (step S200) ends for all data B. Then, a series of processes shown in the flowchart of FIG.

  Next, the notification of the data output state and the operation of the terminal 10, the relay server 30, and the management server 40 associated therewith will be described with reference to FIG. FIG. 17 is a sequence diagram showing an example of the operation including the output abnormality notification and the output abnormality recovery notification. FIG. 17 shows an example in which the output state is notified to the image data and audio data transmitted from the terminal 10 aa to the terminal 10 bd and the terminal 10 dc via the relay server 30.

  Image data and voice data from the terminal 10aa participating in the video conference are transmitted to the terminal 10 bd and the terminal 10 dc participating in the same video conference via the relay server 30 (steps S61 to S63). Hereafter, although illustration is abbreviate | omitted in FIG. 17, it is assumed that the data from this terminal 10aa are continuously transmitted with respect to terminal 10bd and terminal 10dc. In the terminal 10bd and the terminal 10dc, the determination of the output state according to the flow shown in FIG. 16 is performed as needed (for example, at intervals of 5 seconds) on the data from the terminal 10aa.

  Here, it is assumed that the output state of the data transmitted from the terminal 10 aa is detected as “output abnormality” in the terminal 10 db. In this case, the terminal 10 db transmits an output abnormality notification including the data ID of the data determined to be “output abnormality” and the IP address of the own terminal to the relay server 30 (step S 64).

  When receiving the output abnormality notification from the terminal 10 db, the relay server 30 updates the all output state management table of the all output state management DB 3100 according to the output abnormality notification (step S 65). Specifically, the relay server 30 detects the “output abnormality” in the “output abnormal terminal” field corresponding to the data ID of the data in the “output abnormality” state of the all output state management table (here Now add the IP address of the terminal 10 db).

  Next, the relay server 30 transmits the data ID of the data in the output abnormality state, the IP address of the terminal (here, the terminal 10 db) which detected the “output abnormality”, and the data in the “output abnormality” state. An output abnormality notification including the IP address of the source terminal (here, the terminal 10aa) is transmitted to the management server 40 (step S66).

  When the management server 40 receives the output abnormality notification from the relay server 30, the management server 40 searches the terminal management DB 4200 using the IP address of the terminal (in this case, the terminal 10db) which has detected the "output abnormality" as a search key. The name is acquired (step S67). Then, the management server 40 transmits the output abnormality notification including the data ID of the data in the “output abnormality” state and the terminal name of the terminal (here, the terminal 10 db) which has detected the “output abnormality”, the data transmission source terminal It transmits to (here, the terminal 10aa) (step S68). As a result, the terminal 10 aa can grasp which data transmitted by itself is in the “output abnormal” state in which terminal.

  In the terminal 10aa receiving the output abnormality notification from the management server 40, the state display processing unit 23 performs processing for displaying the output state information on the display 20 based on the output abnormality notification from the management server 40 (step S69) ). For example, based on the output abnormality notification from the management server 40, the state display processing unit 23 determines that the image data is in the state of "output abnormality" at the terminal 10db (here, the terminal name is the establishment B). Generate output state information to indicate. Then, the state display processing unit 23 passes the generated output state information to the display control unit 16 and makes a display request of the output state information, for example, a pop-up screen on the video conference screen 300 as shown in FIG. As, the display 20 is displayed on the display 20 the output state information 310 composed of a message such as “an error has occurred in the image output at the office B”.

  Thereafter, it is assumed that the terminal 10 db detects that the output state of the data transmitted from the terminal 10 aa has recovered from “output abnormal” to “output normal”. In this case, the terminal 10 db transmits an output abnormality recovery notification including the data ID of the data whose output state has been recovered and the IP address of the own terminal to the relay server 30 (step S 70).

  When the relay server 30 receives the output abnormality recovery notification from the terminal 10 db, the relay server 30 updates the all output state management table of the all output state management DB 3100 according to the output abnormality recovery notification (step S71). Specifically, the relay server 30 detects the recovery of the output state from the field of “output abnormal terminal” corresponding to the data ID of the data whose output state of the all output state management table has recovered (here, the terminal 10 db Remove the IP address of).

  Next, relay server 30 transmits the data ID of the data whose output state has been recovered, the IP address of the terminal (here, terminal 10 db) which has detected the recovery of the output state, and the source terminal of the data whose output state has recovered (here Then, the output abnormality recovery notification including the IP address of the terminal 10aa) is transmitted to the management server 40 (step S72).

  When the management server 40 receives the output abnormality recovery notification from the relay server 30, the management server 40 searches the terminal management DB 4200 using the IP address of the terminal (in this case, the terminal 10db) which has detected the recovery of the output state as a search key. The terminal name is acquired (step S73). Then, the management server 40 sends an output abnormality recovery notification including the data ID of the data whose output state has recovered and the terminal name of the terminal (here, the terminal 10 db) which has detected the recovery of the output state to the data transmission source terminal (here Then, it transmits to the terminal 10aa) (step S74). As a result, the terminal 10aa can grasp which output state of which data it is transmitting has recovered at which terminal.

  In the terminal 10aa that has received the output abnormality recovery notification from the management server 40, the state display processing unit 23 performs processing for causing the display 20 to display the output state information based on the output abnormality recovery notification from the management server 40 ( Step S75). For example, on the basis of the output abnormality recovery notification from the management server 40, the state display processing unit 23 receives image data that has been in the “output abnormality” state at the terminal 10db (here, the display name is the office B). Output state information indicating that the state of “output normal” is recovered is generated. Then, the state display processing unit 23 passes the generated output state information to the display control unit 16 and makes a display request of the output state information, for example, a pop-up screen on the video conference screen 300 as shown in FIG. As, the display 20 is displayed on the display 20 the output state information 320 configured by a message such as “the output state of the image at the office B has been recovered”.

  In the example shown in the sequence diagram of FIG. 17, the terminal 10 aa performing display processing of output state information detects an output abnormality notification including the terminal name of the terminal 10 db that has detected “output abnormality” and recovery of the output state. An output error recovery notification including the terminal name of the terminal 10 db is received from the management server 40. However, when the terminal 10 that performs display processing of output state information stores the correspondence between the terminal ID and the terminal name of another terminal participating in the video conference, the terminal ID is used instead of the terminal name. The output abnormality notification and the output abnormality recovery notification may be received from the management server 40.

  Further, in the example shown in the sequence diagram of FIG. 17, a terminal that has detected an “output abnormality” by transmitting an output abnormality notification from the relay server 30 or an output abnormality recovery notification to the terminal 10 via the management server 40. 10, the IP address of the terminal 10 which has detected the recovery of the output state is converted into a display name, and the display 10 is notified of the display state of the output state information. However, without particularly specifying the terminal 10 in which the “output abnormality” or the recovery of the output state has occurred, the output state information indicating only the “output abnormality” or the recovery of the output state is displayed in any of the terminals 10 In this case, the output abnormality notification and the output abnormality recovery notification from the relay server 30 may be directly transmitted to the terminal 10 without the management server 40.

  Further, also in the case where at least one of the relay server 30 and the terminal 10 stores the correspondence between the IP address of the terminal 10 participating in the video conference and the terminal name, similarly, the output abnormality from the relay server 30 The notification and the output abnormality recovery notification can be directly transmitted to the terminal 10 without passing through the management server 40.

  Next, an operation of causing the display 20 to display a list of output states of data transmitted from the terminal 10 to another terminal 10 as output state information will be described with reference to FIG. FIG. 19 is a sequence diagram showing an example of an operation of acquiring and displaying a list of output states. FIG. 19 shows an example of acquiring and displaying a list of output states of each data transmitted from the terminal 10aa to the other terminal 10.

  First, when the user of the terminal 10 aa performs an operation of requesting a list of output states using the operation button 108 or the like, the terminal 10 aa transmits an output state list request including the IP address of the own terminal to the relay server 30 ( Step S81).

  When relay server 30 receives the output state list request from terminal 10 aa, each data referring to the all output state management table of all output state management DB 3100 is transmitted to other terminals 10 by the request source terminal 10 aa. , And generates a list of IP addresses of the terminals 10 that have detected "output abnormality" (step S82). Specifically, in the all-output state management table, the relay server 30 uses the IP in the “output abnormal terminal” field corresponding to the data ID of the data being transmitted to the other terminal 10 by the request source terminal 10aa. Check if the address is stored. Then, when the IP address is stored in the field of “output abnormal terminal”, the combination of the IP address and the data ID is taken out. This process is repeated for all data transmitted to the other terminal 10 by the request source terminal 10 aa, and a list of combinations of data ID and IP address is generated.

  Next, the relay server 30 transmits an output state list notification including the IP address of the request source terminal 10aa and the generated list to the management server 40 (step S83).

  When the management server 40 receives the output state list notification from the relay server 30, the management server 40 performs a search process for the session management DB 4400 and the terminal management DB 4200 (step S84). Specifically, the management server 40 refers to the session management table of the session management DB 4400, and is a list of terminal names of other terminals 10 participating in the same video conference as the request source terminal 10aa of the output state list. Generate a list. Furthermore, the management server 40 refers to the terminal management table of the terminal management DB 4200 to acquire the terminal name corresponding to the IP address of the terminal 10 that has detected "output abnormality", and the data ID of the data in the output abnormality state. And a list of combinations of the terminal names of the terminals 10 in which the output abnormality has occurred.

  Next, the management server 40 lists the terminal names of other terminals 10 participating in the same video conference as the request source terminal 10aa, and the data ID of the data in the state of “output abnormality” and “output abnormality” An output state list notification including a list formed of a combination with the terminal name of the terminal 10 that has detected "" is transmitted to the request source terminal 10aa (step S85).

  In the terminal 10 aa receiving the output state list notification from the management server 40, the state display processing unit 23 causes the display 20 to display output state information representing a list of output states based on the output state list notification from the management server 40. The process for (step S86) is performed. For example, the state display processing unit 23 generates screen information of the participating base list screen based on the output state list notification from the management server 40. Then, the state display processing unit 23 passes the generated screen information to the display control unit 16 to make a display request of the output state information, and, for example, as shown in FIG. The display 20 displays a participating base list screen 400 showing what kind of output state the image data and audio data from the own base are in.

  In the example shown in the sequence diagram of FIG. 19, the terminal 10aa makes an output state list request to the relay server 30 (step S81). However, the terminal 10aa may make an output state list request to the management server 40. In this case, the terminal 10aa transmits, for example, an output state list request including the terminal ID of the own terminal to the management server 40. The management server 40 specifies the IP address of the terminal 10aa based on the terminal ID of the terminal 10aa, and transmits an output state list request including the IP address to the relay server 30. The subsequent processing is the same as that of the example shown in the sequence diagram of FIG.

  Further, in the example shown in the sequence diagram of FIG. 19, the terminal 10 aa performing the display processing of the output state information has detected the terminal name of another terminal 10 participating in the same video conference and “terminal abnormal”. An output status list notification including the terminal name of is received from the management server 40. However, when the terminal 10 that performs display processing of output state information stores the correspondence between the terminal ID and the terminal name of another terminal participating in the video conference, the terminal ID is used instead of the terminal name. The output status list notification may be received from the management server 40.

  Further, in the example shown in the sequence diagram of FIG. 19, the IP address of the terminal 10 participating in the video conference is transmitted by transmitting the notification of the output state list from the relay server 30 to the terminal 10 via the management server 40. Is converted to a display name, and notified to the terminal 10 that performs display processing of output state information. However, when at least one of relay server 30 and terminal 10 stores the correspondence between the IP address of the terminal 10 participating in the video conference and the terminal name, the output state list from relay server 30 is listed. The notification may be transmitted directly to the terminal 10 without passing through the management server 40.

  As described above in detail while giving a specific example, in the video conference system 1 of the present embodiment, each of the terminals 10 participating in the video conference determines the output state of the data received from the other terminal 10 Then, the relay server 30 is notified. Further, each of the terminals 10 participating in the video conference acquires the output state of the data transmitted to the other terminal 10 in the other terminal 10 from the relay server 30 via the management server 40, and outputs the acquired output state The state information is displayed on the display 20. Therefore, according to the video conference system 1, each terminal 10 participating in the video conference can easily grasp the output state of the data transmitted to the other terminal 10 in the other terminal 10.

  The present invention is not limited to the above-described embodiment as it is, and various modifications and changes may be made and embodied at the stage of implementation without departing from the scope of the invention. That is, the specific configurations and operations of the video conference system 1, the terminal 10, the relay server 30, the management server 40, and the like described in the above-described embodiment are merely examples, and various modifications can be made depending on the application and purpose. is there.

  For example, the relay server 30, the management server 40, and the program providing server 50 in the above-described embodiment may be constructed by a single computer, or the functions of each server may be divided and arbitrarily assigned by a plurality of computers It may be built. Also, some or all of the functions of each server may be arbitrarily combined to construct a server apparatus or system having a framework different from that of the above-described embodiment. For example, the function of the relay server 30 and the function of the management server 40 may be integrated to construct one server device.

  Moreover, although the video conference system 1 was illustrated as an example of the communication system to which this invention was applied in embodiment mentioned above, it is not restricted to this. The present invention relates to, for example, a telephone system such as an IP (Internet Protocol) telephone which transmits and receives voice data bi-directionally between terminals, map data and route information for a car navigation apparatus mounted on a car from a terminal of a management center The present invention can be effectively applied to various communication systems such as a car navigation system that distributes etc.

  Moreover, in the embodiment mentioned above, although the video conference terminal (terminal) 10 was illustrated as an example of the communication apparatus to which this invention is applied, it is not restricted to this. The present invention is effectively applicable to various communication devices such as, for example, a PC, a tablet terminal, a smartphone, an electronic blackboard, and a car navigation device mounted on a car.

  The embodiment described above can be understood as follows. That is, the terminal 10 according to the embodiment is a terminal 10 that outputs data received from another terminal 10, and a notification unit 22 that notifies the external relay server 30 of an output state of data received from the other terminal 10 While the session for receiving the data is established with the other terminal 10, the output state of the data transmitted to the other terminal 10 in the other terminal 10 is acquired from the relay server 30 via the management server 40. And a state display processing unit 23 which causes the display 20 to display output state information indicating the acquired output state.

  The video conference system 1 according to the embodiment is a video conference system 1 including a plurality of terminals 10 outputting data received from other terminals 10, a relay server 30 and a management server 40, and the plurality of terminals Each of the other 10 notifies the relay server 30 of an output state of data received from another terminal 10, and while establishing a session for receiving the data with the other terminal 10, The output state of the data transmitted to the terminal 10 of the other terminal 10 is acquired from the relay server 30 via the management server 40, and the state display processing unit 23 causes the display 20 to display the output state information indicating the acquired output state. , And the relay server 30 outputs the output state of all data transmitted and received among the plurality of terminals 10 based on the output state notified from the terminal 10. And an output state management unit 33 to sense the notified output state from the terminal 10, a transceiver 31 for notifying via the management server 40 to the source terminal 10 of the data of the output conditions, the.

  Also, the relay server 30 and the management server 40 of the embodiment construct a communication management system that manages transmission and reception of data between a plurality of terminals 10 that output data received from other terminals 10. Each of the plurality of terminals 10 whose transmission and reception of data are managed by the communication management system is to receive the data with the notification unit 22 that notifies the output state of the data received from the other terminals 10 and the other terminals 10 And a state display processing unit 23 for displaying on the display 20 output state information indicating an output state of the data transmitted to the other terminal 10 in the other terminal 10 while the session of (1) is established. Then, the communication system acquires the output state notified from the terminal 10, and controls the output state of all data transmitted and received among the plurality of terminals 10. The output state management unit 33 and the output state management unit 33 And transmitting / receiving units 31 and 41 for notifying the terminal 10 of the transmission source of the data in the output state.

  The communication control method according to the embodiment is a communication control method executed in a terminal 10 that outputs data received from another terminal 10, and the relay server 30 outputs the output state of data received from another terminal 10. The relay server 30 controls the output state of the data transmitted to the other terminal 10 from the relay server 30 to the management server 40 while the process of notifying and the session for receiving the data with the other terminal 10 are established. And displaying on the display 20 the output state information indicating the acquired output state acquired via the.

  The terminal program of the embodiment also has a function of notifying the relay server 30 of an output state of data received from another terminal 10 to the terminal 10 that outputs data received from the other terminal 10, and the other terminal 10 And the output state of the data transmitted to the other terminal 10 in the other terminal 10 is acquired from the relay server 30 through the management server 40 while the session for receiving the data is established, and the acquired output And a function of displaying output state information indicating a state on the display 20.

DESCRIPTION OF SYMBOLS 1 video conference system 2 communication network 10 terminal 11 transmission / reception part 16 display control part 16a time management part 17 audio | voice output part 17a time management part 20 display 21 output state determination part 22 notification part 23 state display process part 30 relay server 31 transmission / reception part 33 Output state management unit 40 Management server 41 Transmission / reception unit 46 Session management unit 1100 Output state management DB
3100 All Output Status Management DB
4400 Session Management DB

JP, 2011-71822, A

The present invention relates to a video conference system, a terminal and a display method .

In order to solve the problems described above, the present invention is a video conference system that transmits and receives data between a plurality of terminals via a server, and each of the plurality of terminals determines the output state of the data being output. A state display processing unit that generates output state information representing an output state of data transmitted to another terminal in another terminal based on a judgment result by the output state judging unit and the output state judgment unit of the other terminal And a display control unit that performs drawing processing for displaying the received data on the display, and causes the display to display the output state information generated by the state display processing unit .

According to the present invention, an effect that it is possible to easily grasp the output state of the other terminal of the data transmitted to another terminal.

Claims (13)

A communication device for outputting data received from another communication device, the communication device comprising:
A notification unit that notifies an external information processing apparatus of an output state of data received from another communication apparatus;
While establishing a session for receiving the data with another communication device, the output state of the data transmitted to the other communication device in the other communication device is acquired from the information processing device, and the acquired output state A display processing unit that causes the display device to display output state information indicating.   The communication device according to claim 1, wherein the display processing unit causes the display device to display the output state information in which the acquired output state is associated with an identification name of another communication terminal corresponding to the output state.   The display processing unit acquires an output state after change from the information processing apparatus when an output state of data transmitted to another communication device changes in another communication device, and the display state information is displayed on the display device. The communication apparatus according to claim 1, wherein the communication apparatus displays the information. There are multiple other communication devices to which data can be sent,
The display processing unit acquires, from the information processing apparatus, a list of output states in each of a plurality of other communication devices of data transmitted to a plurality of other communication devices, and outputs the plurality of other output states included in the acquired list. The communication device according to any one of claims 1 to 3, wherein the output state information associated with each identification name of the communication device is displayed on the display device.   The communication apparatus according to claim 4, wherein the display processing unit requests the information processing apparatus to acquire the list, and acquires the list transmitted from the information processing apparatus in response to the request. A storage unit that stores a final output time at which data received from another communication device is last output;
A determination unit that determines an output state of data received from another communication device;
The determination unit determines that the data is in an output abnormality state, when the elapsed time from the final output time exceeds a first threshold,
The communication device according to any one of claims 1 to 5, wherein the notification unit notifies the information processing device that the data is in an output abnormality state. When the elapsed time from the final output time of the data determined to be in the output abnormality state is less than a second threshold smaller than the first threshold, the determination unit determines that the data is in the output abnormality state It is determined that it has recovered,
The communication device according to claim 6, wherein the notification unit notifies the information processing device that the data has been recovered from an output abnormality state. The determination unit determines that the data is output abnormal if the state where the elapsed time from the final output time of the data determined to be the output abnormal state is less than the second threshold is repeatedly detected a predetermined number of times Judged that he recovered from
The communication device according to claim 7, wherein the notification unit notifies the information processing device that the data is recovered from the output abnormality state. The information processing apparatus includes a relay apparatus that relays data transmitted and received between a plurality of communication apparatuses, and a management apparatus that manages transmission and reception of data between the plurality of communication apparatuses.
The notification unit notifies the relay device of an output state of data received from another communication device,
The communication device according to any one of claims 1 to 8, wherein the display processing unit acquires an output state of another communication device from the relay device via the management device. A communication system including a plurality of communication devices that output data received from another communication device, and an information processing device,
Each of the plurality of communication devices is
A notification unit for notifying the information processing apparatus of an output state of data received from another communication apparatus;
While establishing a session for receiving the data with another communication device, the output state of the data transmitted to the other communication device in the other communication device is acquired from the information processing device, and the acquired output state A display processing unit that causes the display device to display output state information indicating
The information processing apparatus is
A management unit that manages an output state of all data transmitted and received among the plurality of communication devices based on the output state notified from the communication device;
A notification unit for notifying the communication device of the transmission source of the data of the output state of the output state notified from the communication device. A communication management system that manages transmission and reception of data between a plurality of communication devices that output data received from another communication device, comprising:
Each of the plurality of communication devices is
A notification unit that notifies an output state of data received from another communication device;
A display process for displaying on a display device output state information indicating an output state of data transmitted to another communication device while the session for receiving the data is established with another communication device. With the department,
A management unit that acquires an output state notified from the communication device and manages an output state of all data transmitted and received among the plurality of communication devices;
A communication management system comprising: a notification unit that notifies an output state managed by the management unit to the communication device that is a transmission source of data in the output state. A communication control method executed in a communication device for outputting data received from another communication device, comprising:
Notifying an external information processing apparatus of an output state of data received from another communication apparatus;
While establishing a session for receiving the data with another communication device, the output state of the data transmitted to the other communication device in the other communication device is acquired from the information processing device, and the acquired output state And D. displaying the output state information on the display device. In a communication device that outputs data received from another communication device,
A function of notifying an external information processing apparatus of an output state of data received from another communication apparatus;
While establishing a session for receiving the data with another communication device, the output state of the data transmitted to the other communication device in the other communication device is acquired from the information processing device, and the acquired output state And a program for realizing a function of displaying output state information indicating the on the display device.

Images