JP7017155B2 - Transmission system, first transmission terminal, transmission method, display method, and program - Google Patents

Description

The present invention relates to an invention for managing communication of content data for having a conversation between a plurality of transmission terminals.

As an example of a transmission system for transmitting and receiving content data between a plurality of transmission terminals via a relay device, there is a video conference system for conducting a video conference or the like via a communication network such as the Internet. The need for such a video conference system is increasing with the recent reduction of business trip expenses and business trip time. In such a video conference system, a plurality of video conference terminals, which are an example of transmission terminals, are used. Then, the video conference can be realized by transmitting and receiving image data and audio data between these video conference terminals.

However, there is an unsuitable environment for video conferences, such as a private place such as home. As a countermeasure for conducting a video conference in such an environment, a transmission terminal that acquires the environmental state in which the transmission terminal of the other party is placed and regulates image communication or voice communication has been proposed (Patent Document 1). reference).

However, the transmission terminal disclosed in Patent Document 1 only regulates image communication or voice communication in consideration of the environment around the transmission terminal on the other side, and the transmission terminal on the other side itself. Image communication is not regulated in consideration of the image communication status. That is, the transmission terminal disclosed in Patent Document 1 has an image even if the transmission terminal on the other side has a voice communication function but does not have an image communication function, or has an image communication function. Image communication is not regulated in consideration of the image communication status such as no communication. Therefore, for example, even if the transmission terminal on the other side performs image communication when the transmission terminal on the other side is in the image communication state as described above, the transmission terminal on the other side cannot support the image communication, so that unnecessary image communication is performed. There is a challenge to do.

The invention according to claim 1 is
A first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A transmission system including a plurality of transmission terminals including a management system for managing a session for communicating image data, wherein the management system is between the first transmission terminal and the second transmission terminal. When the session management means for managing the session established in the above and the first transmission terminal and the second transmission terminal transmit and receive image data by the session, the second transmission terminal has the second transmission terminal. When the communication of the image data captured by the second imaging means is stopped, the transmission means for transmitting the information indicating the communication state of the image data captured by the second imaging means to the first transmission terminal. The first transmission terminal comprises a receiving means for receiving information indicating a communication state of image data captured by the second imaging means transmitted by the transmitting means, and the first imaging means. It is possible to execute the first display control for displaying the first state indicating that the communication of the image data captured by the above is started on the display screen by the first icon, and the first display control is executed. In response to the reception of information indicating the communication state of the image data captured by the second imaging means in the state, the information indicating the communication state of the image data captured by the second image pickup means received. Based on this, the second display control for displaying the second state indicating that the communication of the image data captured by the second imaging means from the second transmission terminal is stopped is executed on the display screen. A third display control means, which comprises possible display control means, indicating that the first image pickup means has been turned off and communication of image data captured by the first image pickup means has stopped. The state is displayed on the display screen by the second icon, and the first transmission terminal accepts the pressing of the second icon to start communication of the image data captured by the first imaging means. Then, when the display is changed from the second icon to the first icon and the second display control is executed, the first state is displayed by the first display control. The image data captured by the first imaging means is transmitted to the second transmission terminal by the session. It is a transmission system.

As described above, according to the present invention, there is an effect that unnecessary image communication can be prevented.

It is a schematic diagram of the transmission system which concerns on 1st Embodiment of this invention. It is a conceptual diagram which showed the state of transmission / reception of image data, audio data, and various management information in a transmission system. It is a conceptual diagram explaining the image quality of image data. It is an external view of the terminal which concerns on this embodiment. It is a hardware block diagram of the terminal which concerns on this embodiment. It is a hardware block diagram of the management system, the relay device, or the program providing system which concerns on this embodiment. It is a functional block diagram of each terminal, apparatus and system constituting the transmission system which concerns on this embodiment. It is a functional block diagram of the final narrowing-down part. It is a functional block diagram of the primary narrowing part. It is a conceptual diagram which shows the change quality control table. It is a conceptual diagram which shows the relay device management table. It is a conceptual diagram which shows the terminal authentication management table. It is a conceptual diagram which shows the terminal management table. It is a conceptual diagram which shows the destination list management table. It is a conceptual diagram which shows the session management table. It is a conceptual diagram which shows the address priority management table. It is a conceptual diagram which shows the transmission speed priority management table. It is a conceptual diagram which shows the quality control table. It is a sequence diagram which showed the process which manages the state information which shows the operation state of each relay device. It is a sequence diagram which showed the process of the preparatory stage to start the remote communication between terminals. It is a sequence diagram which showed the process of narrowing down a relay device. It is a process flow diagram which showed the process of narrowing down a relay device. It is a figure which showed the calculation state of the priority point at the time of narrowing down process of a relay device. FIG. 3 is a sequence diagram showing a process of selecting a relay device by a transmission terminal according to the first embodiment of the present invention. It is a processing flow diagram which showed the process of selecting a relay device in a transmission terminal. It is a sequence diagram which showed the process of making a communication control message and sending and receiving. It is a flow diagram which mainly showed the process of creating a communication control message. It is a flow diagram which showed a part of the process which creates a communication control message in detail. It is a figure which showed an example of the screen for a video conference displayed on the display of a transmission terminal side. It is a figure which showed an example of the screen for a video conference displayed on the display of a transmission terminal side. It is a figure which showed an example of the screen for a video conference displayed on the display of a transmission terminal side. It is a figure which showed an example of the screen for a video conference displayed on the display of a transmission terminal side. It is a sequence diagram which showed the process of exchanging image data and audio data between transmission terminals.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS. 1 to 33. In the present embodiment, the transmission terminal performs voice communication by transmitting voice data to the transmission terminal on the other side, and performs image communication by transmitting image data.

<< Overall configuration of the embodiment >>
FIG. 1 is a schematic diagram of a transmission system 1 according to an embodiment of the present invention. FIG. 2 is a conceptual diagram showing a state of transmission / reception of image data, audio data, and various management information in a transmission system. FIG. 3 is a conceptual diagram illustrating the image quality of the image data.

In addition, the transmission system includes a data providing system that transmits content data from one transmission terminal to the other transmission terminal in one direction via a transmission management system, and information and information among a plurality of transmission terminals via a transmission management system. It includes a communication system that communicates emotions and the like. This communication system is a system for mutually transmitting information, emotions, etc. between a plurality of communication terminals (corresponding to "transmission terminal") via a communication management system (corresponding to "transmission management system"), and is a television. Examples include a conference system and a videophone system.

In the present embodiment, a transmission system, a transmission management system, assuming a video conference system as an example of a communication system, a video conference management system as an example of a communication management system, and a video conference terminal as an example of a communication terminal, And the transmission terminal will be described. That is, the transmission terminal and transmission management system of the present invention are applied not only to a video conference system but also to a communication system or a transmission system. In this embodiment, although it is described as "video conference", it is sometimes called "video conference", and both have the same contents.

First, the transmission system 1 shown in FIG. 1 has a display (120aa, 120ab, ...) As a display means for a plurality of transmission terminals (10aa, 10ab, ...) And each transmission terminal (10aa, 10ab, ...). , A plurality of relay devices (30a, 30b, 30c, 30d), a transmission management system 50, a program providing system 90, and a maintenance system 100.

The plurality of terminals 10 transmit by transmitting and receiving image data and audio data as an example of content data.

In the following, the "transmission terminal" is simply referred to as a "terminal", and the "transmission management system" is simply referred to as a "management system". Further, any terminal among the plurality of terminals (10aa, 10ab, ...) Is represented as "terminal 10", and any display among the plurality of displays (120aa, 120ab, ...) Is represented as "display 120". , Any relay device among the plurality of relay devices (30a, 30b, 30c, 30d) is represented as "relay device 30". Further, the terminal as the request source for requesting the start of the video conference is represented as the "request source terminal", and the terminal as the destination (relay destination) which is the request destination is represented as the "destination terminal".

Further, as shown in FIG. 2, in the transmission system 1, there is a management information session sei for transmitting and receiving various management information between the request source terminal and the destination terminal via the management system 50. Established. Further, in order to transmit and receive four types of data, high-resolution image data, medium-resolution image data, low-resolution image data, and audio data, between the requesting terminal and the destination terminal via the relay device 30. 4 sessions are established. Here, these four sessions are collectively shown as a session sed for image / audio data.

Here, the image resolution of the image data handled in the present embodiment will be described. As shown in FIG. 3 (a), a low-resolution image consisting of 160 pixels in the horizontal direction and 120 pixels in the vertical direction, which is a base image, and a horizontal image as shown in FIG. 3 (b). There is a medium-resolution image consisting of 320 pixels and 240 pixels in the vertical direction, and a high-resolution image consisting of 640 pixels in the horizontal direction and 480 pixels in the vertical direction as shown in FIG. 3C. Of these, when passing through a narrow band path, low-quality image data consisting of only low-resolution image data as a base image is relayed. When the band is relatively wide, low-resolution image data as a base image and medium-quality image data including medium-resolution image data are relayed. When the band is very wide, high-quality image data including low-resolution image data as base image quality, medium-resolution image data, and high-resolution image data is relayed.

The relay device 30 shown in FIG. 1 relays content data between a plurality of terminals 10. The management system 50 centrally manages login authentication from the terminal 10, management of the call status of the terminal 10, management of the destination list, and the like, and the communication status of the relay device 30. The image of the image data may be a moving image or a still image, and may be both a moving image and a still image.

The plurality of routers (70a, 70b, 70c, 70d, 70ab, 70cd) select the optimum route for image data and audio data. In the following, any router among the routers (70a, 70b, 70c, 70d, 70ab, 70cd) is referred to as "router 70".

The program providing system 90 includes an HD (Hard Disk) 204 described later, and stores a terminal program for realizing various functions in the terminal 10 (or making the terminal 10 function as various means), and stores the terminal 10. You can send a program for the terminal to. Further, the HD204 of the program providing system 90 also stores a program for a relay device for realizing various functions in the relay device 30 (or making the relay device 30 function as various means), and the relay device 30 stores the program for the relay device. A program for a relay device can be transmitted. Further, the HD204 of the program providing system 90 also stores a transmission management program for realizing various functions in the management system 50 (or making the management system 50 function as various means), and the management system 50 stores the transmission management program. A transmission management program can be transmitted.

The maintenance system 100 is a computer for maintaining, managing, or maintaining at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90. For example, when the maintenance system 100 is installed in Japan and the terminal 10, the relay device 30, the management system 50, or the program providing system 90 is installed overseas, the maintenance system 100 is remotely installed via the communication network 2. , Terminal 10, relay device 30, management system 50, and program providing system 90, and perform maintenance such as maintenance, management, and maintenance of at least one of them. Further, the maintenance system 100 includes a model number, a serial number, a sales destination, maintenance and inspection in at least one of the terminal 10, the relay device 30, the management system 50, and the program providing system 90 without going through the communication network 2. Or perform maintenance such as management of failure history.

By the way, the terminals (10aa, 10ab, 10ac, ...), The relay device 30a, and the router 70a are communicably connected by LAN2a. The terminals (10ba, 10bb, 10bc, ...), The relay device 30b, and the router 70b are communicably connected by LAN2b. Further, LAN2a and LAN2b are communicably connected by a dedicated line 2ab including a router 70ab, and are constructed in a predetermined area A. For example, region A is Japan, LAN2a is built in a business office in Tokyo, and LAN2b is built in a business office in Osaka.

On the other hand, the terminals (10ca, 10cc, 10cc, ...), The relay device 30c, and the router 70c are communicably connected by LAN2c. The terminal 10d (a, 10db, 10dc, ...), The relay device 30d, and the router 70d are communicably connected by LAN2d. Further, LAN2c and LAN2d are communicably connected by a dedicated line 2cd including a router 70cd, and are constructed in a predetermined area B. For example, Region B is the United States, LAN2c is built in a New York office, and LAN2d is Washington D.C. C. It is built in the office of. Area A and area B are connected to each other from routers (70ab, 70cd) so as to be able to communicate with each other via the Internet 2i.

Further, the management system 50 and the program providing system 90 are communicably connected to the terminal 10 and the relay device 30 via the Internet 2i. The management system 50 and the program provision system 90 may be installed in the area A or the area B, or may be installed in the area other than these.

In the present embodiment, the communication network 2 of the present embodiment is constructed by the LAN2a, LAN2b, the leased line 2ab, the Internet 2i, the leased line 2cd, the LAN2c, and the LAN2d. The communication network 2 may have a place where wireless communication such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed as well as wired communication.

Further, in FIG. 1, the four sets of numbers shown under each terminal 10, each relay device 30, the management system 50, each router 70, and the program providing system 90 simplify the IP address in general IPv4. Is shown. For example, the IP address of the terminal 10aa is "1.2.1.3". Further, IPv6 may be used instead of IPv4, but for the sake of brevity, IPv4 will be used.

It should be noted that each terminal 10 can be used not only for calls between a plurality of business establishments and between different rooms in the same business establishment, but also for calls within the same room, outdoors and indoors, or outdoors and outdoors. May be used. When each terminal 10 is used outdoors, wireless communication such as a mobile phone communication network is performed.

<< Hardware configuration of the embodiment >>
Next, the hardware configuration of this embodiment will be described. FIG. 4 is an external view of the terminal 10 according to the present embodiment. Hereinafter, the longitudinal direction of the terminal 10 will be described as the X-axis direction, the direction orthogonal to the X-axis direction in the horizontal plane will be described as the Y-axis direction, and the directions orthogonal to the X-axis direction and the Y-axis direction (vertical direction) will be described as the Z-axis direction.

As shown in FIG. 4, the terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. Of these, the front side wall surface 1110 of the housing 1100 is provided with an intake surface (not shown) formed by a plurality of intake holes, and the rear side wall surface 1120 of the housing 1100 is formed with a plurality of exhaust holes. The exhaust surface 1121 is provided. As a result, by driving the cooling fan built in the housing 1100, the outside air behind the terminal 10 can be taken in through the intake surface (not shown) and exhausted to the rear of the terminal 10 through the exhaust surface 1121. A sound collecting hole 1131 is formed in the right wall surface 1130 of the housing 1100, and sounds such as voice, noise, and noise can be picked up by a built-in microphone 114 described later.

An operation panel 1150 is formed on the right side wall surface 1130 side of the housing 1100. The operation panel 1150 is provided with a plurality of operation buttons (108a to 108e) described later, a power switch 109 described later, and an alarm lamp 119 described later, and outputs sound from the built-in speaker 115 described later. A sound output surface 1151 formed by a plurality of audio output holes for passing is formed. Further, on the left side wall surface 1140 side of the housing 1100, an accommodating portion 1160 as a recess for accommodating the arm 1200 and the camera housing 1300 is formed. The right wall surface 1130 of the housing 1100 is provided with a plurality of connection ports (1132a to 1132c) for electrically connecting a cable to the external device connection I / F 118 described later. On the other hand, the left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting the cable 120c for the display 120 to the external device connection I / F 118 described later.

In the following, "operation button 108" is used to indicate an arbitrary operation button among the operation buttons (108a to 108e), and "operation button 108" is used to indicate an arbitrary connection port among the connection ports (1132a to 1132c). This will be described with reference to the connection port 1132.

Next, the arm 1200 is attached to the housing 1100 via the torque hinge 1210, and the arm 1200 is configured to be rotatable in the vertical direction within a tilt angle θ1 of 135 degrees with respect to the housing 1100. ing. FIG. 4 shows a state where the tilt angle θ1 is 90 degrees.

The camera housing 1300 is provided with a built-in camera 112, which will be described later, and can capture images of users, documents, rooms, and the like. Further, a torque hinge 1310 is formed in the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via the torque hinge 1310, and the pan angle θ2 of ± 180 degrees with respect to the arm 1200 with the state shown in FIG. 4 as 0 degree. It is configured to be rotatable in the vertical and horizontal directions within the range of ± 45 degrees and the tilt angle θ3.

Since the relay device 30, the management system 50, and the program providing system 90 each have the same appearance as a general server / computer, the description of the appearance will be omitted.

FIG. 5 is a hardware configuration diagram of the terminal 10 according to the present embodiment of the present invention. As shown in FIG. 5, the terminal 10 of the present embodiment includes a program used to drive the CPU 101 such as a CPU (Central Processing Unit) 101 and an IPL (Initial Program Loader) that control the operation of the entire terminal 10. Control of the stored ROM (Read Only Memory) 102, the RAM (Random Access Memory) 103 used as the work area of the CPU 101, the flash memory 104 for storing various data such as terminal programs, image data, and audio data, and the CPU 101. SSD (Solid State Drive) 105 that controls reading or writing of various data to the flash memory 104, media drive 107 that controls reading or writing (storage) of data to the recording media 106 such as the flash memory, and the destination of the terminal 10. Operation button 108 operated when selecting, power switch 109 for switching the power ON / OFF of the terminal 10, network I / F (Interface) 111 for data transmission using the communication network 2. It is equipped with.

Further, the terminal 10 has a built-in camera 112 that captures a subject and obtains image data under the control of the CPU 101, an image pickup element I / F 113 that controls the drive of the camera 112, a built-in microphone 114 that inputs voice, and voice. The built-in speaker 115 that outputs the image data, the audio input / output I / F 116 that processes the input / output of the audio signal between the microphone 114 and the speaker 115 under the control of the CPU 101, and the image data on the external display 120 under the control of the CPU 101. As shown in FIG. 5, the display I / F 117 to be transmitted, the external device connection I / F 118 for connecting various external devices, the alarm lamp 119 notifying the abnormality of various functions of the terminal 10, and the above-mentioned components are shown in FIG. A bus line 110 such as an address bus or a data bus for electrically connecting to the speaker is provided. Note that some models of the terminal 10 do not have the camera 112 and the image sensor I / F 113.

The display 120 is a display unit composed of a liquid crystal display or an organic EL that displays an image of a subject, an operation icon, or the like. Further, the display 120 is connected to the display I / F 117 by the cable 120c. The cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or for HDMI (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals. It may be a cable of.

The camera 112 includes a lens and a solid-state image sensor that converts light into electric charges to digitize an image (video) of the subject, and as solid-state image sensors, CMOS (Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device). Etc. are used.

External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 118 by a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven in preference to the built-in camera 112 according to the control of the CPU 101. Similarly, when an external microphone is connected or an external speaker is connected, the external microphone and the external speaker are given priority over the built-in microphone 114 and the built-in speaker 115 according to the control of the CPU 101. The external speaker is driven.

The recording medium 106 has a structure that can be attached to and detached from the terminal 10. Further, as long as it is a non-volatile memory that reads or writes data under the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

Further, the terminal program may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 106. Further, the terminal program may be stored in the ROM 102 instead of the flash memory 104.

FIG. 6 is a hardware configuration diagram of the management system according to the present embodiment of the present invention. The management system 50 stores various data such as a CPU 201 that controls the operation of the entire management system 50, a ROM 202 that stores a program used to drive the CPU 201 such as an IPL, a RAM 203 that is used as a work area of the CPU 201, and a transmission management program. HDD (Hard Disk Drive) 205 that controls reading or writing of various data to HD204 according to the control of HD204 to be stored, media drive 207 that controls reading or writing (storage) of data to recording media 206 such as flash memory , A display 208 that displays various information such as cursors, menus, windows, characters, or images, a network I / F 209 for data transmission using the communication network 2, for inputting characters, numerical values, various instructions, etc. Keyboard 211 with multiple keys, mouse 212 for selecting and executing various instructions, selecting processing targets, moving the cursor, etc., CD-ROM (Compact Disc Read Only Memory) as an example of removable recording medium A CD-ROM drive 214 that controls reading or writing of various data to the 213, and a bus line 210 such as an address bus or a data bus for electrically connecting each of the above components as shown in FIG. It is equipped with.

The transmission management program may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. .. Further, the transmission management program may be stored in the ROM 202 instead of the HD 204.

Further, since the relay device 30 has the same hardware configuration as the management system 50, the description thereof will be omitted. However, a relay device program for controlling the relay device 30 is recorded in the HD 204. In this case as well, the program for the relay device may be a file in an installable format or an executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. good. Further, the relay device program may be stored in the ROM 202 instead of the HD 204.

Further, since the program providing system 90 and the maintenance system 100 have the same hardware configuration as the management system 50, the description thereof will be omitted. However, a program providing program for controlling the program providing system 90 is recorded in the HD 204. In this case as well, the program providing program is a file in an installable format or an executable format, and can be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. good. Further, the program for the program providing system may be stored in the ROM 202 instead of the HD 204.

As another example of the detachable recording medium, it is configured to be recorded and provided on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray disc. You may.

<< Functional configuration of the embodiment >>
Next, the functional configuration of this embodiment will be described. In the present embodiment, when there is a delay in receiving image data at the terminal 10 as the destination (relay destination), the relay device 30 changes the image resolution of the image data, and then the terminal 10 as the relay destination. A case of transmitting image data will be described.

FIG. 7 is a functional block diagram of each terminal, device, and system constituting the transmission system 1 of the present embodiment. In FIG. 7, the terminal 10, the relay device 30, and the management system 50 are connected so as to be able to perform data communication via the communication network 2. Further, the program providing system 90 shown in FIG. 1 is omitted in FIG. 7 because it is not directly related to the communication of the video conference.

<Terminal function configuration>
The terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an image pickup unit 14, a voice input unit 15a, a voice output unit 15b, a final narrowing unit 16, a display control unit 17, a delay detection unit 18, and a storage unit. -Has a read processing unit 19 and a message creation unit 20. Each of these parts has a function realized by operating any of the components shown in FIG. 5 by an instruction from the CPU 101 according to a terminal program expanded on the RAM 103 from the flash memory 104, or a function realized by the operation. It is a means of functioning. Further, the terminal 10 has a storage unit 1000 constructed by the RAM 103 shown in FIG. 5 and the flash memory 104 shown in FIG.

(Each function configuration of the terminal)
Next, each functional configuration of the terminal 10 will be described in detail with reference to FIGS. 5 and 7. In the following, in explaining each functional configuration of the terminal 10, among the constituent elements shown in FIG. 5, the relationship with the main constituent elements for realizing each functional configuration of the terminal 10 will also be described. ..

The transmission / reception unit 11 of the terminal 10 shown in FIG. 7 is realized by the command from the CPU 101 shown in FIG. 5 and the network I / F 111 shown in FIG. Sends and receives various data (or information) to and from the terminal, device or system of. The transmission / reception unit 11 starts receiving state information indicating the state of each terminal as a destination candidate from the management system 50 even before starting a call with a desired destination terminal. It should be noted that this status information includes not only the operating status of each terminal 10 (online or offline status), but also whether it is possible to make a call even online, whether the call is in progress, whether the user is away from the desk, and the like. Shows the detailed state of. Further, this state information is not only the operating state of each terminal 10, but also the cable 120c is disconnected from the terminal 10 at the terminal 10, the sound is output but the image is not output, or the sound is not output. (MUTE), etc., showing various states. In the following, as an example, a case where the status information indicates an operating status will be described.

The operation input receiving unit 12 is realized by the command from the CPU 101 shown in FIG. 5 and the operation button 108 and the power switch 109 shown in FIG. 5, and receives various inputs by the user. For example, when the user turns on the power switch 109 shown in FIG. 5, the operation input receiving unit 12 shown in FIG. 7 receives the power ON and turns the power ON.

The login request unit 13 is realized by a command from the CPU 101 shown in FIG. 5, and the transmission / reception unit 11 requests the management system 50 to log in via the communication network 2 when the power is turned on. The login request information indicating the above and the current IP address of the requesting terminal are automatically transmitted. Further, when the user turns off the power switch 109 from the ON state, the operation input reception unit 12 completely turns off the power after the transmission / reception unit 11 sends the state information to the management system 50 to turn off the power. do. As a result, on the management system 50 side, it is possible to grasp that the terminal 10 has been turned from the power ON to the power OFF.

The image pickup unit 14 is realized by the command from the CPU 101 shown in FIG. 5 and the camera 112 and the image pickup element I / F 113 shown in FIG. 5, and the subject is imaged, and the image obtained by the image pickup is taken. Output the data. It should be noted that some models of the terminal 10 do not have the image pickup unit 14.

The voice input unit 15a is realized by the command from the CPU 101 shown in FIG. 5 and the voice input / output I / F 116 shown in FIG. 5, and the user's voice is converted into a voice signal by the microphone 114. After that, the voice data related to this voice signal is input. The audio output unit 15b is realized by the command from the CPU 101 shown in FIG. 5 and the audio input / output I / F 116 shown in FIG. 5, outputs the audio signal related to the audio data to the speaker, and outputs the audio signal to the speaker 115. Output audio from.

Since the final narrowing unit 16 performs the final narrowing process of finally narrowing down from the plurality of relay devices 30 to one relay device 30, the command from the CPU 101 shown in FIG. 5 is as shown in FIG. The measurement unit 16a, the calculation unit 16b, and the final selection unit 16c are realized.

Of these, the measurement unit 16a measures the reception date and time when the advance transmission information is received by the transmission / reception unit 11 for each of the advance transmission information described later received by the transmission / reception unit 11. The calculation unit 16b is based on the difference between the measured reception time and the transmission date and time included in the advance transmission information for each advance transmission information whose reception date and time is measured by the measurement unit 16a. Calculate the time required (T) from transmission to reception. The final selection unit 16c finally selects one relay device by selecting the relay device 30 to which the advance transmission information requiring the shortest required time among the required times calculated by the calculation unit 16b is relayed. ..

The display control unit 17 is realized by a command from the CPU 101 shown in FIG. 5 and a display I / F 117 shown in FIG. 5, and as described later, combines received image data having different resolutions. Control is performed to transmit this combined image data to the display 120. Further, the display control unit 17 can transmit the destination list information received from the management system 50 to the display 120 to display the destination list on the display 120. Further, the display control unit 17 causes the display 120 to display a screen as shown in FIGS. 29 to 32. The screens of FIGS. 29 to 32 will be described later.

The delay detection unit 18 is realized by a command from the CPU 101 shown in FIG. 5, and detects a delay time (ms) of image data or audio data sent from another terminal 10 via the relay device 30. ..

The storage / reading processing unit 19 is executed by the instruction from the CPU 101 shown in FIG. 5 and the SSD 105 shown in FIG. 5, or is realized by the instruction from the CPU 101, and various data are stored or stored in the storage unit 1000. A process of reading various data stored in the unit 1000 is performed. The storage unit 1000 stores a terminal ID (Identification) for identifying the terminal 10, a password, and the like.

The message creation unit 20 is realized by a command from the CPU 101 shown in FIG. 5, reads image communication state information stored in advance from the storage unit 1000, and includes the image communication state information. Create a message (m). The image communication status information and the image communication status message (m) will be described later.

Further, the storage unit 1000 stores the image communication state information described later. Further, the storage unit 1000 overwrites and stores the image data and the voice data received when making a call with the destination terminal every time it is received. Of these, the image is displayed on the display 120 by the image data before being overwritten, and the sound is output from the speaker 115 by the sound data before being overwritten.

The terminal ID of the present embodiment and the relay device ID described later indicate identification information such as a language, characters, symbols, or various marks used for uniquely identifying the terminal 10 and the relay device 30, respectively. Further, the terminal ID and the relay device ID may be identification information in which at least two of the above languages, characters, symbols, and various marks are combined.

<Functional configuration of relay device>
The relay device 30 has a transmission / reception unit 31, a state detection unit 32, a data quality confirmation unit 33, a change quality control unit 34, a data quality change unit 35, and a storage / reading processing unit 39. Each of these parts has a function or a function realized by operating any of the components shown in FIG. 6 by an instruction from the CPU 201 according to a program for a relay device developed on the RAM 203 from the HD 204. It is a means. Further, the relay device 30 has a storage unit 3000 constructed by the RAM 203 shown in FIG. 6 and / or the HD 204 shown in FIG. Note that FIG. 10 is a conceptual diagram showing a change quality control table.

(Change quality control table)
In the storage unit 3000, a change quality control DB (Data Base) 3001 configured by a change quality control table as shown in FIG. 10 is constructed. In the change quality control table, the IP address of the terminal 10 as the relay destination (destination) of the image data and the image quality of the image data relayed by the relay device 30 are managed in association with the relay destination.

(Each function configuration of the relay device)
Next, each functional configuration of the relay device 30 will be described in detail. In the following, in explaining each functional configuration of the relay device 30, among the components shown in FIG. 6, the relationship with the main components for realizing each functional configuration of the relay device 30 is also included. explain.

The transmission / reception unit 31 of the relay device 30 shown in FIG. 7 is realized by the command from the CPU 201 shown in FIG. 6 and the network I / F 209 shown in FIG. 6, and is realized via the communication network 2. Sends and receives various data (or information) to and from other terminals, devices, or systems.

The state detection unit 32 is realized by a command from the CPU 201 shown in FIG. 6, and detects the operating state of the relay device 30 having the state detection unit 32. The operating state includes an "online", "offline", "calling", or "temporarily suspended" state.

The data quality confirmation unit 33 is realized by the command from the CPU 201 shown in FIG. 6, searches the change quality control DB 3001 (see FIG. 10) using the IP address of the destination terminal as a search key, and relays the corresponding data. By extracting the image quality of the image data, the image quality of the relayed image data is confirmed.

The change quality control unit 34 changes the contents of the change quality control DB 3001 based on the quality information described later, which is realized by the command from the CPU 201 shown in FIG. 6 and sent from the management system 50. For example, a video conference is held by transmitting and receiving high-quality image data between a requesting terminal (terminal 10aa) having a terminal ID of "01aa" and a destination terminal (terminal 10db) having a terminal ID of "01db". At the destination terminal (terminal 10db), the requesting terminal (terminal 10bb) and the destination terminal (terminal 10ca) that hold another video conference start a video conference via the communication network 2 during the process. When the reception of the image data is delayed, the relay device 30 lowers the image quality of the image data relayed up to now from high image quality to medium image quality. In such a case, the content of the change quality control DB 3001 is changed so that the image quality of the image data relayed by the relay device 30 is lowered from the high image quality to the medium image quality based on the quality information indicating the medium image quality.

The data quality change unit 35 is realized by the command from the CPU 201 shown in FIG. 6, and the image quality of the image data sent from the source terminal is changed based on the contents of the changed quality control DB 3001. change.

The storage / reading processing unit 39 is realized by the command from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. 6, and various data are stored in the storage unit 3000 or stored in the storage unit 3000. Performs the process of reading various data.

<Functional configuration of management system>
The management system 50 includes a transmission / reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state extraction unit 55, a primary narrowing unit 56, a session management unit 57, a quality determination unit 58, and a storage / reading process. It has a unit 59, a delay time management unit 60, and a message creation unit 61. Each of these parts functions or functions as any of the components shown in FIG. 6 operates by an instruction from the CPU 201 according to a program for a management system developed on the RAM 203 from the HD 204. It is a means. Further, the management system 50 has a storage unit 5000 constructed by the HD 204 shown in FIG.

Note that FIG. 8 is a functional configuration diagram of the final narrowing section. FIG. 9 is a functional configuration diagram of the primary narrowing section. FIG. 11 is a conceptual diagram showing a relay device management table. FIG. 12 is a conceptual diagram showing a terminal authentication management table. FIG. 13 is a conceptual diagram showing a terminal management table. FIG. 14 is a conceptual diagram showing a destination list management table. FIG. 15 is a conceptual diagram showing a session management table. FIG. 16 is a conceptual diagram showing an address priority management table. FIG. 17 is a conceptual diagram showing a transmission speed priority management table. FIG. 18 is a conceptual diagram showing a quality control table.

(Relay device management table)
In the storage unit 5000, a relay device management DB 5001 configured by a relay device management table as shown in FIG. 11 is constructed. In this relay device management table, for each relay device ID of each relay device 30, the operating status of each relay device 30, the reception date and time when the status information indicating the operating status is received by the management system 50, and the IP address of the relay device 30. , And the maximum data transmission rate (Mbps) in the relay device 30 are associated and managed. For example, in the relay device management table shown in FIG. 11, the relay device 30a having the relay device ID "111a" is in the "online" operating state, and the date and time when the state information is received by the management system 50 is "2009". At 13:00 on November 10, 2014, it was shown that the IP address of the relay device 30a is "1.2.1.2" and the maximum data transmission speed of the relay device 30a is 100 Mbps. There is.

(Terminal authentication management table)
Further, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table as shown in FIG. 12 is constructed. In this terminal authentication management table, each password is associated and managed for each terminal ID of all terminals 10 managed by the management system 50. For example, in the terminal authentication management table shown in FIG. 12, it is shown that the terminal ID of the terminal 10aa is "01aa" and the password is "aaaaa".

(Terminal management table)
Further, in the storage unit 5000, a terminal management DB 5003 configured by a terminal management table as shown in FIG. 13 is constructed. In this terminal management table, for each terminal ID of each terminal 10, the destination name when each terminal 10 is the destination, the operating state of each terminal 10, the image communication state described later, and the login request information described later are stored in the management system 50. The received reception date and time and the IP address of the terminal 10 are associated and managed. For example, in the terminal management table shown in FIG. 13, the terminal 10aa having the terminal ID "01aa" has the terminal name "Japan Tokyo Office AA terminal" and the operating state "online (in transit)". The image communication status is "ON", the date and time when the login request information was received by the management system 50 is "13:40 on November 10, 2009", and the IP address of this terminal 10aa is "1.2.1.3". Is shown to be.

Here, the image communication state information will be described. The image communication state information indicates the state related to the image communication of the terminal 10, and the following four patterns exist.
ON: A state in which the terminal 10 is in image communication OFF: A state in which the terminal 10 is in a state in which image communication is stopped (However, the terminal 10 has an image communication function which is an image pickup unit 14 for performing image communication).
NA: A state in which the terminal 10 does not have an image communication function (the terminal 10 does not have an image communication function which is an image pickup unit 14 for performing image communication).
DC: A state in which the image communication state is unknown (a state in which the image communication state has not been sent from the terminal 10 due to the terminal 10 being offline or the like, or an initial state).
Of these, the storage unit 1000 initially stores the image communication state information indicating DC, and the image pickup unit 14 overwrites the image communication state information indicating ON or OFF from the DC. The image communication status information indicating NA is stored in the storage unit 1000 in advance from the time of shipment from the factory of the terminal 10.

(Destination list management table)
Further, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table as shown in FIG. 14 is constructed. In this destination list management table, all the terminal IDs of the destination terminals registered as candidates for the destination terminal are managed in association with the terminal ID of the requesting terminal that requests the start of the call in the video conference. For example, in the destination list management table shown in FIG. 14, a destination terminal (terminal 10db) capable of requesting the start of a call in a video conference from a requesting terminal (terminal 10aa) having a terminal ID of "01aa". Candidates are shown to be a terminal 10ab with a terminal ID of "01ab", a terminal 10ba with a terminal ID of "01ba", a terminal 10bb with a terminal ID of "01bb", and the like. The candidate of the destination terminal is updated by being added or deleted by a request for addition or deletion to the management system 50 from an arbitrary requesting terminal.

(Session management table)
Further, in the storage unit 5000, a session management DB 5005 configured by a session management table as shown in FIG. 15 is constructed. In this session management table, for each selection session ID used for executing a session for selecting the relay device 30, the relay device ID of the relay device 30 used for relaying image data and audio data, and the requesting terminal The management system receives the terminal ID, the terminal ID of the destination terminal, the delay time (ms) of reception when the image data is received at the destination terminal, and the delay information indicating the delay time from the destination terminal. The reception date and time received at 50 are associated and managed. For example, in the session management table shown in FIG. 15, the relay device 30a (relay device ID “111a”) selected in the session executed by using the selection session ID “se1” has a terminal ID of “01aa”. Image data and audio data are relayed between the requesting terminal (terminal 10aa) and the destination terminal (terminal 10db) whose terminal ID is "01db", and the destination terminal (terminal 10db) is "2009". It is shown that the delay time of the image data as of "14:00 on November 10" is 200 (ms). When conducting a video conference between two terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the requesting terminal instead of the destination terminal. However, when a video conference is held between three or more terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the terminal 10 on the receiving side of the image data and the audio data.

(Address priority management table)
Further, in the storage unit 5000, a priority management DB 5006 configured by an address priority management table as shown in FIG. 16 is constructed. In this address priority management table, in any terminal 10 and any relay device 30, the more "same" in the four sets of dot address (Dot Address) parts of the IP addresses in general IPv4, the more the address is prioritized. The difference in dot address and the address priority are managed in association with each other so that the point of degree becomes high. This "same" means that the dot address part is the same, and "different" means that the dot address part is different. For example, in the address priority management table shown in FIG. 16, when the three values from the upper to the lower of the dot address are the same IP address, the point of the address priority is "5". When the two values are the same IP address from the upper part to the lower part of the dot address, the point of the address priority is "3". In this case, whether or not the value of the lowest dot address is the same is irrelevant to the priority. When the IP address has the same highest value of the dot address and the second highest value is different, the point of the address priority is "1". In this case, whether or not the values of the third and lowest dot addresses from the top are the same is irrelevant to the priority. In the case of IP addresses having different top-level values of dot addresses, the point of address priority is "0". In this case, whether or not the values of the second, third, and lowest dot addresses from the top are the same is irrelevant to the priority.

(Transmission speed priority management table)
Further, the priority management DB 5006 constructed in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. In this transmission speed priority management table, the maximum data transmission speed and the transmission speed priority are set so that the larger the value of the maximum data transmission speed (Mbps) in the relay device 30, the higher the point of the transmission speed priority. Associated and managed. For example, in the transmission speed priority management table shown in FIG. 17, when the maximum data transmission speed in the relay device 30 is 1000 Mbps or more, the point of the transmission speed priority is “5”. When the maximum data transmission speed in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the point of the transmission speed priority is "3". When the maximum data transmission speed in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the point of the transmission speed priority is "1". When the maximum data transmission speed in the relay device 30 is less than 10 Mbps, the point of the transmission speed priority is "0".

(Quality control table)
Further, in the storage unit 5000, a quality control DB 5007 configured by a quality control table as shown in FIG. 18 is constructed. In this quality control table, the longer the delay time (ms) of the image data in the requesting terminal or the destination terminal, the lower the image quality of the image data relayed by the relay device 30, so that the delay time of the image data and the image quality of the image data are lowered. (Image quality) is associated and managed.

(Each function configuration of the management system)
Next, each functional configuration of the management system 50 will be described in detail. In the following, in explaining each functional configuration of the management system 50, among the constituent elements shown in FIG. 6, the relationship with the main constituent elements for realizing each functional configuration of the management system 50 is also included. explain.

The transmission / reception unit 51 is executed by the instruction from the CPU 201 shown in FIG. 6 and the network I / F 209 shown in FIG. 6, and various data (various data) with other terminals, devices or systems via the communication network 2. Or information) is sent and received.

The terminal authentication unit 52 is realized by the command from the CPU 201 shown in FIG. 6, and the storage unit 5000 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys. The terminal authentication management DB 5002 is searched, and the terminal authentication is performed by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002.

The state management unit 53 is realized by the command from the CPU 201 shown in FIG. 6, and in order to manage the operating state of the requesting terminal that has requested the login, the requesting terminal is in the terminal management DB 5003 (see FIG. 13). The terminal ID, the operating status of the requesting terminal, the reception date and time when the login request information was received by the management system 50, and the IP address of the requesting terminal are stored and managed in association with each other. Further, the state management unit 53 manages the terminal based on the state information sent from the terminal 10 to the effect that the power is turned off by the user turning the state of the power switch 109 of the terminal 10 from ON to OFF. The operating state indicating online of DB5003 (see FIG. 13) is changed to offline.

The terminal extraction unit 54 is realized by the instruction from the CPU 201 shown in FIG. 6, searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the requesting terminal for which the login request is made as a key, and searches for the requesting terminal. The terminal ID is extracted by reading out the terminal ID of the candidate of the destination terminal that can make a call with. Further, the terminal extraction unit 54 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the requesting terminal that has requested the login as a key, and registers the terminal ID of the requesting terminal as a candidate for the destination terminal. The terminal IDs of other requesting terminals are also extracted.

Further, the terminal extraction unit 54 searches the destination list management DB 5004 using the terminal ID of the terminal that has transmitted the image communication status message (m) as the terminal ID of the requesting terminal, and obtains all the terminal IDs of the corresponding destination terminals. Extract.

The terminal state extraction unit 55 is realized by the command from the CPU 201 shown in FIG. 6, and the terminal management DB 5003 (see FIG. 13) uses the terminal ID of the candidate terminal of the destination terminal extracted by the terminal extraction unit 54 as a search key. ), And the operating state is read out for each terminal ID extracted by the terminal extraction unit 54. As a result, the terminal state extraction unit 55 can acquire the operating state of the candidate of the destination terminal that can make a call with the requesting terminal that has requested the login. Further, the terminal state extraction unit 55 searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and also acquires the operating state of the requesting terminal that has requested the login.

Further, the terminal state extraction unit 55 extracts the corresponding image communication state information by searching the terminal management DB 5003 (see FIG. 13) using the terminal ID of the destination terminal as a search key. Further, the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and the IP of the terminal 10 during image communication, the terminal 10 with which image communication is stopped, or the terminal 10 having an image communication function. Extract the address.

The primary narrowing unit 56 is realized by an instruction from the CPU 201 shown in FIG. 6 and supports the final narrowing process of finally narrowing down from a plurality of relay devices 30 to one relay device 30. Therefore, this final narrowing process is performed. Perform the previous primary narrowing process. Therefore, the primary narrowing-down unit 56 receives a command from the CPU 201 shown in FIG. 5, and as shown in FIG. 9, the selection session ID generation unit 56a, the terminal IP address extraction unit 56b, and the primary selection unit 56. A unit 56c and a priority determination unit 56d are realized.

Of these, the selection session ID generation unit 56a generates a selection session ID used for executing a session for selecting the relay device 30. The terminal IP address extraction unit 56b obtains the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the requesting source terminal and the terminal ID of the destination terminal included in the start request information sent from the requesting source terminal. By searching, the IP address of each corresponding terminal 10 is extracted. The primary selection unit 56c relays by selecting the relay device ID of the relay device 30 whose operating state is "online" among the relay devices 30 managed by the relay device management DB 5001 (see FIG. 11). Select the device 30.

Further, the primary selection unit 56c searches the relay device management DB 5001 (see FIG. 11) based on the IP address of the requesting terminal and the IP address of the destination terminal extracted by the terminal IP address extraction unit 56b. For each dot address of the IP address of the selected relay device 30, it is investigated whether the dot address is the same as or different from each dot address in each IP address of the request source terminal and the destination terminal. Further, the primary selection unit 56c is a relay device having a higher point among the integrated points in which the higher point with respect to the terminal 10 in the address priority point and the transmission speed priority point are integrated for each relay device. By selecting 30, further selection of the relay device 30 is made. In the present embodiment, the top two relay devices 30 with high points are selected, but the present invention is not limited to this, and if as many relay devices 30 as possible can be narrowed down, the top two relay devices with high points are selected. You may choose three or more relay devices 30.

The priority determination unit 56d determines the address priority point for each relay device 30 investigated by the primary selection unit 56c with reference to the priority management DB 5006 (see FIG. 16). Further, the priority determination unit 56d searches the priority management DB 5006 (see FIG. 17) based on the maximum data transmission speed of each relay device 30 managed by the relay device management DB 5001 (see FIG. 11). The transmission speed priority point is determined for each relay device 30 narrowed down by the primary narrowing process by the primary selection unit 56c.

Subsequently, the session management unit 57 is realized by an instruction from the CPU 201 shown in FIG. 6, and is generated in the session management DB 5005 (see FIG. 15) of the storage unit 5000 by the selection session ID generation unit 56a. Session ID, request source terminal terminal ID, and destination terminal terminal ID are associated, stored, and managed. Further, the session management unit 57 relays the relay device 30 finally selected by the final selection unit 16c of the terminal 10 for each selection session ID with respect to the session management DB 5005 (see FIG. 15). ID is stored and managed.

The quality determination unit 58 searches the quality control DB 5007 (see FIG. 18) using the delay time as a search key, extracts the image quality of the corresponding image data, and determines the image quality of the image data to be relayed to the relay device 30. do.

The storage / reading processing unit 59 is executed by the instruction from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. 6, and various data are stored in the storage unit 5000 or stored in the storage unit 5000. Performs the process of reading various data.

The delay time management unit 60 is realized by a command from the CPU 201 shown in FIG. 6, and is a corresponding terminal by searching the terminal management DB 5003 (see FIG. 13) using the IP address of the destination terminal as a search key. The ID is extracted, and further, in the session management table of the session management DB 5005 (see FIG. 15), the delay time indicated by the delay information is stored in the field portion of the delay time in the record including the extracted terminal ID. And manage it.

The message creation unit 61 creates a communication control message according to the image communication state of each terminal 30. The process of creating this communication control message will be described in detail later with reference to FIG. 28. Further, the message creation unit 61 acquires the image communication status information related to the transmission source terminal 10 and the IP address of the terminal 10 from the image communication status message (m) sent from the terminal 10. Further, in the terminal management DB 5003 (see FIG. 13), the message creation unit 61 adds the image communication status message (m) to the attribute portion of the image communication status corresponding to the IP address acquired from the image communication status message (m). Overwrite and rewrite the acquired image communication status information.

Further, the message creation unit 61 extracts the terminal ID corresponding to the IP address acquired from the image communication status message (m) from the terminal management DB 5003 (see FIG. 13). Further, the message creation unit 61 refers to the image communication status information extracted from the terminal management DB 5003 (see FIG. 13), and determines whether the requesting terminal and all the destination terminals each have an image communication function. do.

<< Processing or operation of the embodiment >>
Next, the processing method in the transmission system 1 according to the present embodiment will be described with reference to FIGS. 19 to 33. Note that FIG. 19 is a sequence diagram showing a process of managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50. FIG. 20 is a sequence diagram showing a process in the preparatory stage for starting a call between a plurality of terminals 10. FIG. 21 is a sequence diagram showing a process of narrowing down the relay device 30. FIG. 22 is a processing flow diagram showing a process of narrowing down the relay device 30. FIG. 23 is a diagram showing a calculation state of priority points when performing narrowing down processing of the relay device 30. FIG. 24 is a sequence diagram showing a process in which the terminal 10 selects the relay device 30. FIG. 25 is a processing flow diagram showing a process of selecting the relay device 30 at the terminal. FIG. 33 is a sequence diagram showing a process of transmitting and receiving image data and audio data between terminals.

First, with reference to FIG. 19, a process of managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50 will be described. First, in each relay device 30, the state detection unit 32 shown in FIG. 7 periodically detects the operating state of the relay device 30 which is its own device (steps S1-1 to 4). Then, in order for the management system 50 to manage the operating state of each relay device 30 in real time, the transmission / reception unit 31 of each relay device 30 periodically transmits each state information to the management system 50 via the communication network 2. (Steps S2-1 to 4). Each of these state information includes a relay device ID for each relay device 30 and an operating state detected by the state detection unit 32 of the relay device 30 related to each relay device ID. In the present embodiment, the relay device (30a, 30b, 30d) is operating normally and is "online", while the relay device 30c is in operation, but the relay device 30c is operated as a relay. It is shown that the program to be executed has some trouble and is "offline".

Next, in the management system 50, the transmission / reception unit 51 receives each state information sent from each relay device 30, and the relay device management DB 5001 (see FIG. 11) of the storage unit 5000 via the storage / reading processing unit 59. ), The state information is stored and managed for each relay device ID (steps S3-1 to 4). As a result, the operating state of either "online", "offline", or "out of order" is stored and managed in the relay device management table as shown in FIG. 11 for each relay device ID. At this time, the reception date and time when the status information is received by the management system 50 is also stored and managed for each relay device ID. If the status information is not sent from the relay device 30, the field portion of the operating status and the field portion of the reception date and time in each record of the relay device management table shown in FIG. 11 are blank or blank. The operating status and reception date and time at the time of the previous reception are shown.

Next, with reference to FIG. 20, the transmission / reception processing of each management information in the preparatory stage before starting the call will be described between the terminal 10aa and the terminal 10db. In FIG. 20, various management information is transmitted and received by the management information session sei shown in FIG.

First, when the user turns on the power switch 109 shown in FIG. 5, the operation input receiving unit 12 shown in FIG. 7 receives the power ON and turns the power ON (step S21). Then, the login request unit 13 automatically transmits the login request information indicating the login request from the transmission / reception unit 11 to the management system 50 via the communication network 2 when the power is turned on (step S22). This login request information includes a terminal ID and a password for identifying the terminal 10aa which is the own terminal as the request source. These terminal IDs and passwords are data read from the storage unit 1000 via the storage / reading processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the terminal 10aa to the management system 50, the management system 50 on the receiving side can grasp the IP address of the terminal 10ab on the transmitting side.

Next, the terminal authentication unit 52 of the management system 50 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and the terminal authentication management DB 5002 of the storage unit 5000 (see FIG. 12). ), And terminal authentication is performed by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). Since the same terminal ID and password are managed by the terminal authentication unit 52, if it is determined that the login request is from a terminal 10 having a legitimate usage authority, the state management unit 53 manages the terminal. The terminal ID of the terminal 10aa, the operating state, the reception date and time when the login request information is received, and the IP address of the terminal 10aa are stored in the DB 5003 (see FIG. 13) in association with each other (step S24). As a result, in the terminal management table shown in FIG. 13, the terminal ID "01aa", the operating state "online", the reception date and time "2009.11.10.13: 40", and the IP address "1" of the terminal 10aa are displayed. 2.1.3 ”will be managed in association with each other.

Then, the transmission / reception unit 51 of the management system 50 requests the login of the authentication result information showing the authentication result obtained by the terminal authentication unit 52 via the communication network 2 (terminal 10aa). (Step S25). In the present embodiment, the case where the terminal authentication unit 52 determines that the terminal has a proper usage authority will be continuously described below.

The terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the requesting terminal (terminal 10aa) for which login is requested as a search key, and searches for the destination list management DB 5004 (see FIG. 14), and the requesting terminal (terminal 10aa). ) Is extracted by reading out the terminal ID of the candidate of the destination terminal capable of communicating with (step S26). Here, the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminals (terminals 10ab, 10ba, 10db) corresponding to the terminal ID “01aa” of the requesting terminal (terminal 10aa) are extracted. Will be.

Next, the terminal state extraction unit 55 uses the terminal ID (“01ab”, “01ba”, “01db”) of the candidate terminal of the destination terminal extracted by the terminal extraction unit 54 as a search key, and the terminal management DB 5003 (FIG. 13). By searching (see) and reading out the operating status (“offline”, “online”, “online”) for each terminal ID extracted by the terminal extraction unit 54, each operation of the terminal (10ab, 10ba, 10db) is performed. Acquire the state (step S27).

Next, the transmission / reception unit 51 has a terminal ID (“01ab”, “01ba”, “01db”) as a search key used in step S27, and a destination terminal (terminals 10ab, 10ba, 10db) corresponding to each. The destination status information including the respective operating statuses (“offline”, “online”, “online”) of the above is transmitted to the requesting terminal (terminal 10aa) via the communication network 2 (step S28). As a result, the requesting terminal (terminal 10aa) is in the current operating state (“offline) of the terminals (10ab, 10ba, 10db) that are candidates for the destination terminal capable of communicating with the requesting terminal (terminal 10aa). , "Online", "Online").

Further, the terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the requesting terminal (terminal 10aa) that has requested the login as a search key, and searches for the requesting source. The terminal ID "01aa" of the terminal (terminal 10aa) is registered as a candidate for the destination terminal, and the terminal ID of another requesting terminal is extracted (step S29). In the destination list management table shown in FIG. 14, the terminal IDs of the other requesting terminal to be extracted are "01ab", "01ba", and "01db".

Next, the terminal state extraction unit 55 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the terminal ID "01aa" of the requesting terminal (terminal 10aa) that has made the login request as a search key. , Acquire the operating state of the requesting terminal (terminal 10aa) for which the login request has been made (step S30).

Then, the transmission / reception unit 51 is among the terminals (10ab, 10ba, 10db) related to the terminal IDs (“01ab”, “01ba”, “01db”) extracted in step S29, and the terminal management DB 5003 (FIG. 13). The terminals (10ba, 10db) whose operating status is "online" in (see) include the terminal ID "01aa" of the requesting terminal (terminal 10aa) acquired in step S30 and the operating status "online". The destination status information is transmitted (steps S31-1 and 2). When the transmission / reception unit 51 transmits the destination status information to the terminals (10ba, 10db), it is managed by the terminal management table shown in FIG. 13 based on each terminal ID (“01ba”, “01db”). Refer to the IP address of the terminal. As a result, the requesting terminal (terminal 10aa) for which the login request is made to each of the other destination terminals (terminals 10db, 10ba) capable of communicating with the requesting terminal (terminal 10aa) for which the login request is made as the destination. The terminal ID "01aa" and the operating status "online" can be transmitted.

On the other hand, also in the other terminal 10, when the user turns on the power switch 109 shown in FIG. 6, the operation input receiving unit 12 shown in FIG. 7 accepts the power ON in the same manner as in step 21. , Since the same processing as the processing of steps S22 to S31-1 and 2 is performed, the description thereof will be omitted.

Subsequently, the process of narrowing down the relay device 30 will be described with reference to FIG. In FIG. 21, various management information is transmitted and received by the management information session sei shown in FIG. 21. Further, in the present embodiment, the requesting terminal (terminal 10aa) is a terminal (10ba, 10db) whose operating state is online based on the state information of the terminal received in step S32 among the terminals 10 as destination candidates. ) Can be talked to at least one of them. Therefore, in the following, a case where the user of the requesting terminal (terminal 10aa) selects to start a call with the destination terminal (terminal 10db) will be described.

First, when the user presses the operation button 108 shown in FIG. 5 and selects the terminal 10db, the operation input receiving unit 12 shown in FIG. 7 starts a call with the destination terminal (terminal 10db). The request to be made is accepted (step S41). Then, the transmission / reception unit 11 of the request source terminal (terminal 10aa) includes the terminal ID "01aa" of the terminal 10aa and the terminal ID "01db" of the destination terminal (terminal 10db), and is a start request indicating that the call is to be started. Information is transmitted to the management system 50 (step S42). As a result, the transmission / reception unit 51 of the management system 50 can receive the start request information and grasp the IP address "1.2.1.3" of the request source terminal (terminal 10aa) which is the transmission source. ..

Then, the state management unit 53 bases the terminal management DB 5003 (see FIG. 13) based on the terminal ID “01aa” of the request source terminal (terminal 10aa) and the terminal ID “01db” of the destination terminal (terminal 10db) included in the start request information. ) In the terminal management table, the field portion of the operating state of the record including the terminal ID "01aa" and the terminal ID "01db" is changed to "in a call" (step S43). In this state, the requesting source terminal (terminal 10aa) and the destination terminal (terminal 10db) have not started a call, but the call is in progress, and the other terminal 10 is the requesting source terminal (terminal 10aa) or the destination terminal. When trying to make a call with (terminal 10db), a notification sound or a display indicating a so-called in-call state is output.

Next, a process of executing a session for selecting the relay device 30 will be described. First, the selection session ID generation unit 56a shown in FIG. 9 generates a selection session ID used for executing a session for selecting the relay device 30 (step S44). Then, the session management unit 57 has the session management DB 5005 (see FIG. 15) of the storage unit 5000, the selection session ID “se1” generated in step S44, and the terminal ID “01aa” of the requesting terminal (terminal 10aa). , And the terminal ID “01db” of the destination terminal (terminal 10db) are associated, stored and managed (step S45).

Next, the primary narrowing unit 56 of the management system 50 shown in FIG. 7 has a request source terminal (terminal 10aa) and a destination terminal based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006. The primary narrowing down of the relay device 30 for relaying the call with (terminal 10db) is performed (step S46).

Here, the process in step S46 will be described in more detail with reference to FIGS. 9 and 22. First, the terminal IP address extraction unit 56b shown in FIG. 9 has the terminal ID "01aa" of the requesting source terminal (terminal 10aa) included in the start communication information sent from the requesting source terminal (terminal 10aa). By searching the terminal management DB 5003 (see FIG. 13) based on the terminal ID "01db" of the destination terminal (terminal 10db), the IP address ("1.2") of the corresponding terminal (10aa, 10db) is searched. 1.1.3 ”,“ 1.3.2.4 ”) are extracted (step S46-1).

Next, the primary selection unit 56c is a relay device (30a, 30b, 30d) that is "online" in the operating state of the relay device 30 managed by the relay device management DB 5001 (see FIG. 11). The relay device ID (111a, 111b, 111d) is selected (step S46-2). Further, the primary selection unit 56c has the IP address "1.2.1.3" of the requesting terminal (terminal 10aa) and the IP address "1" of the destination terminal (terminal 10db) extracted in step S46-1. By searching the relay device management DB 5001 (see FIG. 11) based on "3.2.4", each IP address ("30a, 30b, 30d" of the relay device (30a, 30b, 30d) selected by the above step S46-2. For each dot address of "1.2.1.2.", "1.2.2.2", "1.3.2.2"), the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) ) Is the same as or different from each dot address in each IP address (“1.2.1.3”, “1.3.2.4”) (step S46-3).

Next, the priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16), and points the address priority for each relay device (30a, 30b, 30d) investigated by step 46-3. Is determined (step S46-4). The result of this determination process is shown in a table as shown in FIG. 23. Note that FIG. 23 is a diagram showing a calculation state of priority points when narrowing down the relay device 30. In FIG. 23, an address priority point, a transmission speed priority point, and an integration point are shown for each relay device ID. Further, the points of the address priority are further shown as points for the request source terminal (terminal 10aa) and points for the destination terminal (terminal 10db) of each relay device 30. The integration point is the sum of the higher of the two address priority points and the transmission speed priority point.

In the present embodiment, the IP address "1.2.1.2" of the relay device 30a is "same. Same. Same. Since it is "different", the point of address priority is "5" as shown in FIG. 23. Further, as shown in FIG. 1, the IP address "1.2.1.2" of the relay device 30a is relative to the IP address "1.3.2.4" of the destination terminal (terminal 10db). As shown in FIG. 16, since it is “same. Different. Different. Different”, the address priority point is “1” as shown in FIG. 23. Further, the IP address "1.2.2.2" of the relay device 30b is "same. Same. Different. Different" with respect to the IP address "1.2.1.3" of the requesting terminal (terminal 10aa). Therefore, the point of address priority is "3". Further, the IP address "1.2.2.2" of the relay device 30b is "same, different, same, different" with respect to the IP address "1.3.2.4" of the destination terminal (terminal 10db). Therefore, the point of address priority is "1". Further, the IP address "1.3.2.2" of the relay device 30d is "same, different, different, different" with respect to the IP address "1.2.1.3" of the requesting terminal (terminal 10aa). Therefore, the point of address priority is "1". Further, the IP address "1.3.2.2" of the relay device 30d is "same. Same. Same. Different" with respect to the IP address "1.3.2.4" of the destination terminal (terminal 10db). Therefore, the point of address priority is "5".

Next, returning to FIG. 22, the priority determination unit 56d uses the priority management DB 5006 (FIG. 17) based on the maximum data transmission speed of each relay device 30 managed by the relay device management DB 5001 (see FIG. 11). By searching (see), the transmission speed priority point is determined for each relay device (30a, 30b, 30d) narrowed down by the primary narrowing process in step S46-2 (step S46-5). In the present embodiment, as shown in FIG. 11, the maximum data transmission speed of the relay device 30a is 100 (Mbps). Therefore, referring to the transmission speed priority shown in FIG. 17, the transmission speed priority is given. The degree becomes 3 points. Similarly, when the maximum data transmission speed of the relay device 30b is calculated, it is 1000 (Mbps), so that the transmission speed priority is 5 points. Similarly, when the maximum data transmission speed of the relay device 30d is calculated, it is 10 (Mbps), so that the transmission speed priority is 1 point.

Next, the primary selection unit 56c integrates the higher point of the terminals (10aa, 10db) at the address priority point and the transmission speed priority point for each relay device (30a, 30b, 30d). Among the integrated points, the top two relay devices 30 having the highest points are selected (step 46-6). In the present embodiment, as shown in FIG. 23, the relay device IDs (111a, 111b, 111d) have the integration points of "8", "8", and "6", respectively, so that the relay device ID " The relay device 30a related to "111a" and the relay device 30b related to the relay device ID "111b" will be selected.

When the narrowing down process in step S46 is completed, the transmission / reception unit 51 shown in FIG. 7 transmits the number of the narrowed down relay devices 30 to the destination terminal (terminal 10db) via the communication network 2. The relay device narrowing down information for the purpose is transmitted (step S47). The relay device narrowing information includes the number "2" of the relay devices 30 narrowed down by the step 46, the terminal ID "01aa" of the requesting terminal (terminal 10aa), and the session ID "se1" for selection. It has been. As a result, the terminal 10db can grasp how many relay devices 30 are in the execution of the session in the selection session ID "se1" and from which terminal 10 the request is made to start the call. At the same time, the IP address "1.1.1.2" of the management system 50, which is the source of the relay device narrowing-down information, can be grasped.

Then, the terminal 10db transmits the reception completion information indicating that the reception of the relay device narrowing information is completed from the transmission / reception unit 11 to the management system 50 via the communication network 2 (step S48). The reception completion information includes the session ID "se1". As a result, the management system 50 has completed the transmission of the number of relay devices executed by the session ID "se1", and at the same time, the IP address "1.3.2.4" of the destination terminal (terminal 10db) which is the transmission source. Can be grasped.

Next, a process in which the destination terminal (terminal 10db) selects the relay device 30 will be described with reference to FIGS. 24 and 25. In FIG. 24, various management information is transmitted / received by the management information session sei shown in FIG.

First, the management system 50 sends advance relay request information to each of the relay devices (30a, 30b) narrowed down by step S46 to request relay in advance before starting a video conference call. Transmission (steps S61-1 and 2). The pre-relay request information includes the session ID "se1", the IP address "1.2.1.3" of the request source terminal (terminal 10aa), and the IP address "1.3.2" of the destination terminal (terminal 10db). .4 "is included. As a result, each relay device (30a, 30b) can grasp which selection session it belongs to, what the requesting terminal is, and what the destination terminal is, and the advance relay request information. The IP address "1.1.1.2" of the management system 50, which is the source of the above, can be grasped.

Next, each of the relay devices (30a, 30b) from the transmission / reception unit 31 to the requesting terminal (terminal 10aa) grasped by the above steps S61-1 and 2 via the communication network 2 before the start of the call. The advance transmission request information indicating that the advance transmission information including the ping (Packet Internet Groper) described later is transmitted to each relay device (30a, 30b) as the own device is transmitted (steps S62-1 and 2). .. This advance transmission information includes the session ID "se1". As a result, the requesting terminal (terminal 10aa) grasps that the advance transmission information is transmitted to each relay device (30a, 30b) in the selection process of the relay device 30 executed by the session ID “se1”. , It is possible to grasp the respective IP addresses ("1.2.1.2", "1.2.2.2") of the relay devices (30a, 30b) that are the transmission sources of the advance transmission request information.

The management system 50 directly notifies the requesting terminal of the IP address of the destination terminal to the relay device 30a as in step S61-1 without notifying the IP address of the destination terminal. As in step S61-2, the relay device 30a requests the requesting terminal to transmit the advance transmission request information to the own device (relay device 30a), so that each terminal 10 is requested by another terminal. This is to ensure security by not notifying the IP address of 10.

Next, the requesting terminal (terminal 10aa) transmits advance transmission information from the transmission / reception unit 11 to each relay device (30a, 30b) via the communication network 2 (steps S63-1 and 2). This advance transmission information is transmitted to the destination terminal (terminal 10db) via the relay devices (30a, 30b) instead of the image data and the audio data prior to the transmission of the image data and the audio data. This is information used for measuring the time required from the transmission of the requesting terminal (terminal 10aa) to the reception of the destination terminal (terminal 10db). Further, the advance transmission information includes a ping for confirming that the request source terminal (terminal 10aa), the relay device (30a, 30b), and the destination terminal (terminal 10db) are communicably connected, and the request source. The transmission date and time when the advance transmission information is transmitted from the terminal (terminal 10aa) and the session ID "se1" are included. As a result, each relay device (30a, 30b) can grasp that the advance transmission information has been sent in the execution of the session in the selection session ID "se1", and is the transmission source of the advance transmission information. The IP address "1.2.1.3" of the requesting terminal (terminal 10aa) can be grasped.

Next, each relay device (30a, 30b) has an IP address "1.3.2.4" of the destination terminal (terminal 10db) included in the advance relay request information received in steps S61-1 and 2. The above-mentioned advance transmission information is relayed (steps S64-1 and 2). As a result, the destination terminal (terminal 10db) can grasp that the advance transmission information has been sent in the execution of the session at the session ID "se1", and is the transmission source (relay source) of the advance transmission information. The IP address ("1.2.1.2", "1.2.2.2") of the relay device (30a, 30b) can be grasped.

Next, the final narrowing unit 16 of the destination terminal (terminal 10db) narrows down to one relay device 30 that finally relays image data and audio data in a video conference call based on the advance transmission information (step S65). ).

Here, the process in step S65 will be described in more detail with reference to FIGS. 8 and 25. First, the measurement unit 16a of the final narrowing unit 16 shown in FIG. 8 receives each pre-transmission information relayed by each relay device (30a, 30b) when it is received by the transmission / reception unit 11 of the terminal 10db. The date and time are measured (step S65-1). Next, the calculation unit 16b starts from the transmission of each advance transmission information based on the difference between the reception date and time and the transmission date and time included in the advance transmission information for each advance transmission information for which the reception time is measured. The time required for reception is calculated (step S65-2). Next, the final selection unit 16c determines whether or not all the advance transmission information corresponding to the number “2” of the relay devices 30 scheduled to be relayed has been received in the execution of the session in the session ID “se1” (step S65). -3). Then, if not all have been received (NO), the final selection unit 16c determines whether a predetermined time (here, 1 minute) has elapsed since the advance transmission information was received by the terminal 10db (step S65-4). ). Further, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, if all are received in step 65-3 (YES), or if a predetermined time has elapsed in step S65-4 (YES), the final selection unit 16c has been calculated by the calculation unit 16b so far. Select one relay device 30 that relays the advance transmission information that required the shortest required time among the required required times (step S65-5). In the present embodiment, the relay device 30a is selected on the assumption that the advance transmission information relayed by the relay device 30a has a shorter time required from transmission to reception than the advance transmission information relayed by the relay device 30b. Is shown.

Next, the destination terminal (terminal 10db) transmits selection information indicating that the relay device 30a has been selected from the transmission / reception unit 11 to the management system 50 via the communication network 2 (step S66). This selection information includes the session ID "se1" and the relay device ID "111a" of the selected relay device 30a. As a result, the management system 50 can grasp that the relay device 30a has been selected in the execution of the session with the session ID "se1", and the IP address "1" of the destination terminal (terminal 10db) which is the source of the selection information. 3.2.4 "can be grasped.

Next, the session management unit 57 of the management system 50 finally puts 1 in the field portion of the relay device ID of the record including the session ID "se1" in the session management table of the session management DB 5005 (see FIG. 15). The relay device ID "111a" of the relay device 30a selected is stored and managed (step S67). Then, the transmission / reception unit 51 of the management system 50 transmits the relay start request information indicating the request to start the relay to the relay device 30a via the communication network 2 (step S68). The relay start request information includes the IP addresses of the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) to be relayed (“1.2.1.3”, “1.3.2.4”). It is included. As a result, the relay device 30a can grasp that the relay device 30a, which is its own device, has been selected. Therefore, three image data of low resolution, medium resolution, and high resolution are used among the terminals (10aa, 10db). At the same time, a session for talking voice data is established (step S69). Therefore, the terminal (10aa, 10db) can start a video conference call.

In step S47, the management system 50 transmits relay device narrowing information to the destination terminal (terminal 10db), and the destination terminal (terminal 10db) relays through steps S48 to S64-1 and 2. The device selection process (step S65) has been performed, but the present invention is not limited to this. In step S47, the management system 50 transmits relay device narrowing information to the requesting terminal (terminal 10aa), and then the step. Up to S64-1, the transmission source and the reception source of each information may be switched between the request source terminal (terminal 10aa) and the destination terminal (terminal 10db). As a result, the requesting terminal (terminal 10aa) can perform the selection process of the relay device in place of the step S65, and can also transmit the selection information in place of the step S66.

Subsequently, when a video conference is held at another base (plural terminals 10) using FIGS. 26 to 28, a communication control message is created and transmitted to each terminal 10, and then communication control is performed by each terminal 10. The processing until the processing based on the message is performed will be described. FIG. 26 shows a case where a video conference is performed by the terminal 10aa, the terminal 10ba, and the terminal 10db. Note that FIG. 26 is a sequence diagram showing a process of creating and transmitting / receiving a communication control message. FIG. 27 is a flow chart mainly showing a process of creating a communication control message. FIG. 28 is a flow chart showing in detail a part of the process of creating a communication control message. In this embodiment, a case where three terminals (10aa, 10ba, 10db) as multiple bases have a meeting will be described.

First, as shown in FIG. 26, the message creation unit 20 of each terminal 10 (here, the terminal 10aa) reads out the image communication state information stored in advance from the storage unit 1000, and performs this image communication. An image communication status message (m) including status information is created (step S71). The image communication status message (m) includes the IP address "1.2.1.3" of the terminal 10aa. The destination name may be included instead of the IP address. This image communication status message (m) can be created, for example, in the XML (eXtensible Markup Language) format according to XMPP (eXtensible Messaging and Presence Protocol) of RFC3921. Communication control messages and display control messages, which will be described later, are also created in the same format.

Next, the transmission / reception unit 11 of the terminal 10aa transmits the image communication status message (m) created in step S71 to the management system 50 via the communication network 2 (step S72). As a result, the transmission / reception unit 51 of the management system 50 receives the image communication status message (m).

Next, the message creation unit 61 of the management system 50 creates a communication control message according to the image communication state of each terminal 30 (step S73). This step S73 will be described in detail with reference to FIG. 27.

First, the message creation unit 61 obtains image communication status information related to the source terminal 10aa from the image communication status message (m) received by the transmission / reception unit 51, and the IP address "1.2.1.3" of the terminal 10aa. "(Step S73-1). Then, in the terminal management DB 5003 (see FIG. 13), the message creation unit 61 puts the image acquired in step S73-1 in the attribute portion of the image communication state corresponding to the IP address acquired in step S73-1. The communication status information is overwritten and rewritten (step S73-2).

Next, the message creation unit 61 extracts the terminal ID corresponding to the IP address acquired in step S73-1 from the terminal management DB 5003 (see FIG. 13) (step S73-3).

Next, the terminal extraction unit 54 extracts all the terminal IDs of the corresponding destination terminals by searching the destination list management DB 5004 using the terminal ID extracted in step S73-3 as the terminal ID of the requesting terminal. (Step S73-4). Then, the terminal state extraction unit 55 extracts the corresponding image communication state information by searching the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the destination terminal (step S73-5).

Next, the message creation unit 61 refers to the image communication status information extracted in step S73-5, and determines whether each of the requesting terminal and all the destination terminals has an image communication function. Step S73-6). If neither the requesting terminal nor all the destination terminals have the image communication function (NO), the process of step S73 ends. In this case, the message transmission process (see FIGS. 26 and 27) according to step S74 described later is not performed.

On the other hand, if either the requesting terminal or all the destination terminals has an image communication function (YES), the message creating unit 61 determines the communication control message described later according to the image communication state of each terminal 10. Is created (step S73-7).

Subsequently, with reference to FIG. 28, the above step S73-7 will be described in more detail. First, the message creation unit 61 refers to the image communication state extracted in step S73-5, and refers to a plurality of two or more terminals (10aa, 10ba, 10db) in conversation by multiple bases. It is determined whether the terminal 10 is in image communication (step S101). If it is determined by step S101 that a plurality of terminals 10 are in image communication (YES), the message creation unit 61 further asks whether there is a terminal 10 in which image communication is stopped among the remaining terminals 10. Is determined (step S102). If it is determined by step S103 that there is a terminal 10 whose image communication is stopped (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and is determined by step S102. The IP address of the terminal 10 whose image communication is stopped is extracted (step S103). Then, the message creation unit 61 creates a communication control message (M3) indicating that a plurality of terminals are in image communication (step S104). Specifically, the communication control message (M3) includes the IP address of the destination extracted in step S103 and the IP address of the management system 50 as the source, in addition to the message of "MESSAGE_IMAGE_ON".

On the other hand, when it is determined by step S101 that the plurality of terminals 10 are not in image communication (NO), the message creating unit 61 refers to the image communication state extracted by step S73-5 and refers to the image communication state. It is determined whether only a single terminal 10 is in image communication (step S105). If it is determined in step S105 that only a single terminal 10 is in image communication (YES), the message creation unit 61 further refers to the image communication state extracted in step S73-5. It is determined whether or not all the terminals 10 other than the terminal 10 during image communication have the image communication function (step S106). If it is determined by step S106 that all terminals 10 other than the terminal 10 during image communication do not have the image communication function (YES), the terminal state extraction unit 55 may use the terminal management DB 5003 (FIG. 13). (See), and the IP address of the single terminal 10 determined to be in image communication in step S105 is extracted (step S107). Then, the message creation unit 61 creates a communication control message (M4) to the effect that the image communication is forcibly stopped (step S108). Specifically, the communication control message (M4) includes the IP address of the destination extracted in step S107 and the IP address of the management system 50 as the source, in addition to the message of "DATA_IMAGE_OFF".

On the other hand, if it is not determined by step S106 that all terminals 10 other than the terminal 10 during image communication do not have the image communication function (NO), that is, other than the terminal 10 during image communication. When it is determined that at least one of the terminals 10 has an image communication function, the message creation unit 61 further refers to the image communication state extracted by step S73-5 and has an image communication function. It is determined whether or not there is a terminal 10 in which image communication is stopped among the terminals 10 having the above (step S109). If it is determined by step S109 that there is a terminal 10 whose image communication is stopped (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and is determined by step S105. The IP address of the terminal 10 during image communication determined to be the above is extracted (step S110). Then, the message creation unit 61 creates a communication control message (M1) to the effect that the image communication is stopped by the other plurality of terminals (step S104). Specifically, the communication control message (M1) includes the IP address of the destination extracted in step S110 and the IP address of the management system 50 as the source, in addition to the message of "MESSAGE_IMAGE_OFF".

Further, the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and extracts the IP address of the terminal 10 whose image communication is stopped, which is determined to be present by step S109 (step S112). Then, the message creation unit 61 creates a communication control message (M2) indicating that another single terminal is in image communication (step S113). Specifically, the communication control message (M2) includes the IP address of the destination extracted in step S112 and the IP address of the management system 50 as the source, in addition to the message of "MESSAGE_IMAGE_ON".

On the other hand, (1) when it is not determined by the step S102 that there is a terminal 10 whose image communication is stopped among the remaining terminals 10 (NO), (2) only a single terminal 10 is determined by the step S105. Is determined not to be in image communication (NO), that is, there is no terminal 10 in image communication at all, or (3) it is determined in step S109 that there is no terminal 10 in which image communication is stopped. In the case (NO), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and obtains the IP address of the terminal 10 determined to have the image communication function in step S73-6. Extract (step S114). Then, the message creation unit 61 creates a display control message (M5) to erase the predetermined display message displayed on each display 120 of the plurality of terminals 10 (step S115). Specifically, the display control message (M5) includes the IP address of the destination extracted in step S114 and the IP address of the management system 50 as the source, in addition to the message of "MESSAGE_IMAGE_ON".

From the above, the communication control message (M1, M2, M3, or M4) or the display control message (M5) is created by the message creation unit 61.

Next, the transmission / reception unit 51 of the management system 50 connects the image communication status of each of these terminals and the image of the terminal of the conversation partner to three terminals (10aa, 10ba, 10db) at multiple locations via the communication network 2. A communication control message (M1, M2, M3, or M4) created according to the communication state or a display control message (M5) is transmitted (step S74). In FIG. 26, among steps S74, transmission from the management system 50 to the terminal ba is step S74-1, transmission from the management system 50 to the terminal db is step S74-2, and transmission from the management system 50 to the terminal aa. Is represented in step S74-3.

Specifically, when the step S103 is executed, the transmission / reception unit 51 of the management system 50 sends the communication control message (M3) created in the step S104 to the IP address extracted in the step S103. ) Is sent. When the step S107 is executed, the transmission / reception unit 51 transmits the communication control message (M4) created in the step S108 to the IP address extracted in the step S107. Further, when the step S110 is executed, the transmission / reception unit 51 transmits the communication control message (M1) created in the step S111 to the extracted IP address. Further, when the step S112 is executed, the transmission / reception unit 51 transmits the communication control message (M2) created in the step S113 to the IP address extracted in the step S112. Further, when the step S114 is executed, the transmission / reception unit 51 transmits the communication control message (M5) created in the step S115 to the IP address extracted in the step S114.

As a result, each terminal (10aa, 10ba, 10db) performs processing based on the content of the communication control message (M1, M2, M3, or M4) or the display control message (M5) (steps S75-1 to 3). ..

Subsequently, the processes performed in steps S75-1 to 3 will be described with reference to FIGS. 7 and 29 to 32. 29 to 32 are views showing an example of a video conference screen displayed on the display 120 on the terminal 10 side. Of these, the video conference screen P1 displayed on the display 120 includes a main screen P2, a sub screen P3, and a sub screen P4. Two sub-screens at the bottom of the sub-screen P4 and three more sub-screens on the left side of the bottom sub-screen are included. Further, at the bottom of the video conference screen P1, there is a message display area P5 for displaying a display message, and "screen switching" pressed by a mouse pointer or the like when switching between the images displayed on the main screen and the sub screen. Button P6 and whether the power of the camera 112 of the own terminal is turned on and image communication is performed from the own terminal side, or the power of the camera 112 of the own terminal is turned off and image communication is performed from the own terminal side. The camera icon P7 indicating is displayed. When "x" is displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is on, and it is displayed that the user can turn off the power from now on. On the other hand, when "x" is not displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is turned off, and it is displayed that the user can turn on the power from now on.

First, in step S104, when a communication control message (M3) indicating that image communication is stopped for a plurality of other terminals is created, the display control unit 17 of the terminal 10 is as shown in FIG. 29. Displays a display message in the message display area P5 of the display 120, for example, "Images from other bases are being received. The camera of the own terminal is off." The main screen P2 displays the image of the other party as the main, and the sub screen P3 originally displays the image of the own terminal, but it is not displayed here, and further, the sub screen. An image of the other party as a sub is displayed on P4.

As a result, the user sees the video conference image P1 shown in FIG. 29, and is in a state where the image data is not transmitted from the own terminal and the image data is sent from another terminal. It can be recognized that it is in a state of being. If the user wants to transmit the image data from his / her own terminal, he / she can start the transmission of the image data by pressing the icon P7. When this camera icon P7 is pressed, an "x" is displayed on the camera icon P7.

Further, in step S108, when the communication control message (M4) for forcibly stopping the image communication is created, the image pickup unit 14 of the terminal 10 stops the image pickup and the transmission / reception unit according to the command from the CPU 101. 11 stops the transmission of image data (stops image communication). Further, as shown in FIG. 30, the display control unit 17 hides all the screens P2, P3, P4 and the like of the video conference screen P1 and erases "x" from the camera icon P7. .. The image pickup unit 14 includes not only the camera 112 shown in FIG. 5 but also an external camera connected to the external device connection I / F.

As a result, the user is in a state where the image data is not transmitted from the own terminal by viewing the image P1 for the television conference shown in FIG. 30, and the camera 112 of the own terminal has stopped imaging. It can be recognized that it is in a state and that no image data has been sent from another terminal.

Further, when a communication control message (M1) indicating that all other terminals are stopped in image communication is created in step S111, the display control unit 17 of the terminal 10 is as shown in FIG. In the message display area P5 of the display 120, for example, a display message stating "Images from other bases have not been received. The camera of the own terminal is on." Is displayed. In FIG. 31, nothing is displayed on the main screen P2 and the sub screen P4, and the image on the own terminal side is displayed on the sub screen P3.

As a result, the user sees the video conference screen P1 shown in FIG. 31, and is in a state where only the own terminal is transmitting the image data, and the image data is transmitted from all the other terminals. It can be recognized that it is in a state where it has not been received.

Furthermore, when a communication control message (M2) indicating that another single terminal is in image communication is created in step S113, the display control unit 17 of the terminal 10 is shown in FIG. 32. In this way, the message display area P5 is displayed with a display message such as "The image is being received from another location. The camera of the own terminal is off." In FIG. 32, the image of the other party is displayed in one place on the main screen P2, and nothing is displayed on the sub screens P3 and P4.

As a result, the user sees the video conference screen P1 shown in FIG. 32, and is in a state where the image data is not transmitted from the own terminal, and one of the plurality of terminals on the other side. It can be recognized that the image data is being sent only from the location.

In this way, when the image of FIG. 31 is displayed on one terminal 10 side and the image of FIG. 32 is displayed on the other terminal 10, each user adjusts the state of image communication on both sides. Then, it is possible to select whether to perform image communication with each other or not to perform image communication with each other.

Further, when the display control message (M5) to delete the display message is created in step S115, the display control unit 17 of the terminal 10 deletes the display message displayed in the message display area P5.

The terminal 10 having an image communication function can perform image communication not only by the processing in step S72 of FIG. 26 but also by pressing the camera icon P7 shown in FIG. 29 during the video conference. You can send a status message. Specifically, when the user selects the camera icon P7 in which "x" is displayed as shown in FIG. 29, the message creation unit 20 of the terminal 10 stops image communication. Create an image communication status message to the effect that it is inside. Further, when the user selects the camera icon P7 in which the “x” is not displayed as shown in FIG. 30, the message creation unit 20 of the terminal 10 is in the process of image communication. Create an image communication status message to that effect.

Subsequently, with reference to FIGS. 7 and 33, a process of transmitting and receiving image data and audio data in order to make a video conference call between the requesting terminal and the destination terminal will be described. It should be noted that the transmission / reception of image data and audio data is performed by one-way processing for transmitting image data and audio data from the terminal 10aa to the terminal 10db and reverse processing for transmitting image data and audio data from the terminal 10db to the terminal 10aa. Since the detection of the delay time and the like described later are the same processing, the above-mentioned one-way communication will be described, and the above-mentioned reverse-way communication will be omitted.

First, the requesting terminal (terminal 10aa) is the image data of the subject captured by the image pickup unit 14a and the voice input by the voice input unit 15a by the image / voice data session seed shown in FIG. The voice data is transmitted from the transmission / reception unit 11 to the relay device 30a via the communication network 2 (step S81). In this embodiment, high-quality image data consisting of low resolution, medium resolution, and high resolution shown in FIG. 3 and audio data are transmitted. As a result, in the relay device 30a, the transmission / reception unit 31 receives the image data and the audio data of the above three resolutions. Then, the data quality confirmation unit 33 searches the change quality management DB 3001 (see FIG. 10) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key, and the corresponding relayed image. By extracting the image quality of the data, the quality of the image of the relayed image data is confirmed (step S82). In the present embodiment, the image quality of the confirmed image data is "high image quality", which is the same as the image quality of the image data received by the transmission / reception unit 31, so that the relay device 30a is subjected to the image / audio data session seed. The image data of the same image quality and the audio data of the same sound quality are transferred to the destination terminal (terminal 10db) (step S83). As a result, the destination terminal (terminal 10db) receives high-quality image data and audio data including low resolution, medium resolution, and high resolution at the transmission / reception unit 11. Then, the display control unit 17 can combine the image data of the three image quality to display the image on the display 120, and the audio output unit 15b can output the audio based on the audio data.

Next, the delay detection unit 18 of the terminal 10db detects the delay time of reception of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every 1 second) (step S84). In the present embodiment, the case where the delay time is 200 (ms) will be described below.

The transmission / reception unit 11 of the destination terminal (terminal 10db) transmits the delay information indicating the delay time "200 (ms)" to the management system 50 via the communication network 2 by the management information session sei shown in FIG. Transmit (step S85). As a result, the management system 50 can grasp the delay time and the IP address "1.3.2.4" of the terminal 10db which is the transmission source of the delay information.

Next, the delay time management unit 60 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key. The corresponding terminal ID "01db" is extracted, and further, in the session management table of the session management DB5005 (see FIG. 15), the delay time field portion in the record of the terminal ID "01db" is indicated by the delay information. The delayed time "200 (ms)" is stored and managed (step S86).

Next, the quality determination unit 58 searches the quality control DB 5007 (see FIG. 18) using the delay time "200 (ms)" as a search key, and extracts the image quality "medium image quality" of the corresponding image data. , The image quality is determined to be "medium image quality" (step S87).

Next, the transmission / reception unit 51 uses the relay device ID "111a" associated with the terminal ID "01db" as a search key in the session management table of the session management DB (see FIG. 15) as the relay device management DB 5001 (FIG. 15). 11) is searched, and the IP address “1.2.1.2” of the corresponding relay device 30a is extracted (step S88). Then, the transmission / reception unit 51 conveys the image quality "medium image quality" of the image data determined in step 87 to the relay device 30a via the communication network 2 by the management information session sei shown in FIG. Information is transmitted (step S89). This quality information includes the IP address "1.3.2.4" of the destination terminal (terminal 10db) used as the search key in step S86. As a result, in the relay device 30a, the change quality control unit 34 tells the change quality control DB 3001 (see FIG. 10) the IP address "1.3.2.4" of the destination terminal 10 (here, the terminal 10db). , And the image quality "medium image quality" of the relayed image data is associated and stored and managed (step S90).

Next, the terminal 10aa is continuously subjected to the image / audio data session sed to the relay device 30a in the same manner as in step S81. The voice data is transmitted (step S91). As a result, in the relay device 30a, as in step S82, the data quality confirmation unit 33 uses the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key, and the change quality management DB 3001 ( By searching (see FIG. 10) and extracting the image quality "medium image quality" of the corresponding relayed image data, the quality of the image of the relayed image data is confirmed (step S92). In the present embodiment, the image quality of the confirmed image data is "medium image quality", which is lower than the image quality "high image quality" of the image data received by the transmission / reception unit 31, so that the data quality changing unit 35 uses the image data. The image quality of the image data is changed by suppressing the image quality of the image from "high image quality" to "medium image quality" (step S93).

Then, the transmission / reception unit 31 changes the image quality of the image data to "medium image quality" and the sound quality of the sound to the terminal 10db via the communication network 2 by the image / audio data session seed. The unrecorded audio data is transmitted (step S94). As a result, the destination terminal (terminal 10db) receives the medium-quality image data and audio data consisting of low resolution and medium resolution at the transmission / reception unit 11. Then, the display control unit 17 can combine the image data of the two resolutions to display the image on the display 120, and the audio output unit 15b can output the audio based on the audio data.

In this way, when the reception is delayed at the destination terminal (terminal 10db) that receives the image data, the relay device 30a changes the image quality and gives a sense of discomfort to the person participating in the video conference. Can be avoided.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, the management system 50 has a predetermined communication control for causing a predetermined terminal 10 to control image communication based on the image communication state information indicating the image communication state of each terminal 10. A communication control message is transmitted to the terminal 10 that creates a message and controls image communication. As a result, the transmission management system can cause a predetermined transmission terminal to perform image communication according to the image communication state of the terminal 10 on the other side, so that unnecessary image communication can be prevented. Play. In addition, this has the effect of eliminating unnecessary power consumption and communication costs.

Further, even if the environment of LAN2 such as the IP address of the relay device 30 can be grasped in the communication network 2, it is difficult to grasp the environment of the entire Internet 2i, so information on the environment that can be grasped first. From, narrow down to two or more out of 30 of a plurality of relay devices that relay image data and audio data. Then, before actually transmitting and receiving image data and audio data between the plurality of terminals 10, the advance transmission information is actually transmitted and received in place of the image data and audio data, so that the advance transmission is actually the fastest. It has the effect of being able to narrow down to one relay device 30 capable of relaying information.

That is, by selecting the relay device 30 to which the upper two or more IP addresses close to any of the IP addresses of the terminal 10 are assigned, two or more candidates for the relay device 30 to be finally used are left. be able to. As a result, after that, the advance transmission information is transmitted and received between the request source terminal and the destination terminal via each relay device 30 as a candidate, so that the relay device 30 of the two or more candidates can be transmitted and received. It is possible to narrow down to the relay device 30 that relays the advance transmission information with the shortest required time. Therefore, it is possible to realize the maximum transmission / reception of high-quality image data or audio data under the current environment of the communication network 2.

Further, in the present embodiment, when narrowing down the relay devices 30, not only the relay device 30 having an IP address close to the IP address of the terminal 10 for video conference is preferentially selected, but also the maximum data transmission in each relay device 30 is performed. Two or more relay devices 30 are selected in consideration of speed. This has the effect of narrowing down the candidates for the relay device 30 that match the environment of the actual communication network 2.

Further, in the present embodiment, when narrowing down the relay device 30, since the relay device 30 whose operating state is online is narrowed down, the candidates for the relay device 30 that are more suitable for the environment of the actual communication network 2 can be narrowed down. can.

[Supplement to the embodiment]
The relay device 30, the management system 50, the program providing system 90, and the maintenance system 100 in the above embodiment may be constructed by a single computer, or each part (function or means) may be divided and arbitrarily assigned. It may be built by multiple computers. Further, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may or may not be divided into a plurality of modules. It may be sent. Further, when the program providing system 90 is constructed by a plurality of computers, the plurality of modules may be separated and transmitted from each computer.

Further, a recording medium such as a CD-ROM in which a terminal program, a relay device program, or a transmission management program of each of the above embodiments is stored, an HD204 in which these programs are stored, and a program including the HD204. The provided system 90 is used as a program product when the terminal program, the relay device program, and the transmission management program are provided to a user or the like in Japan or overseas.

Further, in the above embodiment, the image is taken as an example of the image quality of the image data relayed by the relay device 30 by the modified quality control table shown in FIG. 10 and the quality control table shown in FIG. The management focused on the resolution of the image of the data, but the quality is not limited to this, and other examples of quality include the depth of image quality of the image data, the sampling frequency of the audio of the audio data, and the bit length of the audio of the audio data. You may manage by paying attention to such things. Further, the voice data may be divided into data having three types of resolutions (high resolution, medium resolution, and low resolution) and transmitted / received.

Further, although the reception date and time are managed in FIGS. 11, 13, and 15, the reception date and time is not limited to this, and at least the reception time of the reception date and time may be managed.

Further, in the above embodiment, the IP address of the relay device is managed in FIG. 11 and the IP address of the terminal is managed in FIG. 13, but the present invention is not limited to this, and the relay device 30 is specified on the communication network 2. If it is the relay device specific information of the above or the terminal specific information for specifying the terminal 10 on the communication network 2, each FQDN (Fully Qualified Domain Name) may be managed. In this case, a well-known DNS (Domain Name System) server will acquire the IP address corresponding to the FQDN. It should be noted that not only "relay device identification information for specifying the relay device 30 in the communication network 2" but also "relay device connection destination information indicating the connection destination to the relay device 30 on the communication network 2" or "communication". It may be expressed as "relay device destination information indicating the destination to the relay device 30 on the network 2". Similarly, not only "terminal identification information for specifying the terminal 10 in the communication network 2", but also "terminal connection destination information indicating the connection destination to the terminal 10 on the communication network 2" or "on the communication network 2". It may be expressed as "terminal destination information indicating the destination to the terminal 10."

Further, in the above embodiment, the case of a video conference system has been described as an example of the transmission system 1, but the present invention is not limited to this, and a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone may be used. .. Further, the transmission system 1 may be a car navigation system. In this case, for example, one of the terminals 10 corresponds to a car navigation device mounted on a car, and the other of the terminals 10 is mounted on a management terminal or a management server of a management center that manages car navigation, or another car. Corresponds to a car navigation device. Further, the transmission system 1 may be a communication system of a mobile phone. In this case, for example, the terminal 10 corresponds to a mobile phone.

Further, in the above embodiment, image data and audio data have been described as an example of content data, but the present invention is not limited to this, and may be tactile (touch) data. In this case, the feeling of contact by the user on one terminal side is transmitted to the other terminal side. Further, the content data may be smell data. In this case, the odor (smell) on one terminal side is transmitted to the other terminal side. Further, the content data may be at least one of image data, audio data, tactile data, and olfactory data.

Further, in the above embodiment, the case where the video conference is held by the transmission system 1 has been described, but the present invention is not limited to this, and the present invention is not limited to this, and is not limited to this. It may be used for presentation.

1 Transmission system 10 Transmission terminal 11 Transmission / reception unit (an example of receiving means, an example of communication stopping means)
14 Imaging unit (example of imaging means)
16 Final narrowing unit 16a Measurement unit 16b Calculation unit 16c Final selection unit 17 Display control unit (an example of display control means)
18 Delay detection unit 20 Message creation unit (an example of creation means)
30 Relay device 31 Transmission / reception unit 32 Status detection unit 33 Data quality confirmation unit 34 Change quality control unit 35 Data quality change unit 50 Transmission management system 51 Transmission / reception unit (example of transmission means)
52 Terminal authentication unit 53 Status management unit 54 Terminal extraction unit 55 Terminal status extraction unit 56 Primary narrowing unit 56a Session ID generation unit for selection 56b Terminal IP address extraction unit 56c Primary selection unit 56c
56d Priority determination unit 57 Session management unit 58 Quality determination unit 60 Delay time management unit 61 Message creation unit (an example of creation means)
70 Router 90 Program provision system 100 Maintenance system 101 CPU (an example of control means)
120 Display (an example of display means)
1000 Storage unit 3000 Storage unit 3001 Change quality control DB
5000 Storage unit 5001 Relay device management DB
5002 Terminal authentication management DB
5003 Terminal management DB (an example of terminal management means)
5004 Destination list management DB (an example of destination list management means)
5005 Session management DB
5006 Priority management DB
5007 Quality control DB

Japanese Unexamined Patent Publication No. 2008-131412

Claims (15)

A first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A transmission system including a plurality of transmission terminals including a management system for managing sessions for communicating image data.
The management system is
A session management means for managing the session established between the first transmission terminal and the second transmission terminal,
When the first transmission terminal and the second transmission terminal transmit and receive image data by the session, the communication of the image data captured by the second image pickup means at the second transmission terminal is stopped. When this is done, a transmission means for transmitting information indicating a communication state of image data captured by the second image pickup means to the first transmission terminal, and
Equipped with
The first transmission terminal is
A receiving means for receiving information indicating a communication state of image data captured by the second imaging means transmitted by the transmitting means, and a receiving means for receiving information indicating a communication state.
It is possible to execute the first display control for displaying the first state indicating that the communication of the image data captured by the first image pickup means has been started on the display screen by the first icon, and the first display control is possible. In response to the reception of information indicating the communication state of the image data captured by the second image pickup means in the state where the display control of the above is executed, the image data captured by the second image pickup means received. Based on the information indicating the communication state, the display screen displays the second state indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped. Display control means that can execute the second display control,
Have,
The display control means displays a third state on the display screen by a second icon, indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped. Display,
By accepting the pressing of the second icon, the first transmission terminal starts communication of image data captured by the first imaging means, and from the second icon to the first icon. In a state where the display is changed and the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is displayed. A transmission system characterized by transmitting to the second transmission terminal by the session. The transmission system according to claim 1 .
The display control means displays a second state on the display screen by displaying a message indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped. A transmission system featuring. The transmission system according to claim 1 or 2 .
The display control means is characterized in that a first state indicating that the first transmission terminal is communicating image data captured by the first image pickup means is displayed on a display screen by displaying a message. Transmission system. The transmission system according to any one of claims 1 to 3 , wherein the image data communication is image data communication used in a video conference. Between a first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A session management means for managing a session for communicating established image data, and the second transmission terminal when the first transmission terminal and the second transmission terminal transmit and receive image data by the session. When the communication of the image data captured by the second imaging means is stopped in the transmission terminal, the information indicating the communication state of the image data captured by the second imaging means is transmitted to the first transmission terminal. A first transmission terminal connected via a network to a management system having a transmission means for transmitting.
A receiving means for receiving information indicating a communication state of image data captured by the second imaging means transmitted by the transmitting means, and a receiving means for receiving information indicating a communication state.
It is possible to execute the first display control for displaying the first state indicating that the communication of the image data captured by the first image pickup means has been started on the display screen by the first icon, and the first display control is possible. In response to the reception of information indicating the communication state of the image data captured by the second image pickup means in the state where the display control of the above is executed, the image data captured by the second image pickup means received. Based on the information indicating the communication state, the display screen displays the second state indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped. Display control means that can execute the second display control,
Have,
The display control means displays a third state on the display screen by a second icon, indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped. Display,
By accepting the pressing of the second icon, the first transmission terminal starts communication of image data captured by the first imaging means, and from the second icon to the first icon. In a state where the display is changed and the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is displayed. A first transmission terminal characterized by transmitting to the second transmission terminal by the session. A first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A transmission method executed by a transmission system including a plurality of transmission terminals including a management system for managing a session for communicating image data.
The management system is
It has a session management means for managing the session established between the first transmission terminal and the second transmission terminal.
When the first transmission terminal and the second transmission terminal transmit and receive image data by the session, the communication of the image data captured by the second image pickup means at the second transmission terminal is stopped. If so, a transmission step of transmitting information indicating the communication state of the image data captured by the second imaging means to the first transmission terminal is executed.
The first transmission terminal is
A reception step for receiving information indicating a communication state of image data captured by the second image pickup means transmitted by the transmission step, and a reception step for receiving information indicating a communication state.
The first display control step of displaying the first state indicating that the communication of the image data captured by the first image pickup means has started on the display screen by the first icon, and the first display control step.
In response to receiving information indicating the communication state of the image data captured by the second image pickup means in the state where the first display control step is executed, the image is captured by the received second image pickup means. A second state indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped based on the information indicating the communication state of the image data is displayed on the display screen. The second display control step to be displayed on the
In the state where the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is obtained by the session. The transmission step to send to the second transmission terminal,
And run
A third state indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped is displayed on the display screen by the second icon.
By accepting the pressing of the second icon, the communication of the image data captured by the first imaging means is started, and the display is changed from the second icon to the first icon. Transmission method. Between a first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A session management means for managing a session for communicating established image data, and the second transmission terminal when the first transmission terminal and the second transmission terminal transmit and receive image data by the session. When the communication of the image data captured by the second imaging means is stopped in the transmission terminal, the information indicating the communication state of the image data captured by the second imaging means is transmitted to the first transmission terminal. A display method executed by the first transmission terminal connected via a network to a management system having a transmission means for transmission.
A reception step for receiving information indicating a communication state of image data captured by the second image pickup means transmitted by the transmission means, and a reception step for receiving information indicating a communication state.
The first display control step of displaying the first state indicating that the communication of the image data captured by the first image pickup means has started on the display screen by the first icon, and the first display control step.
In response to receiving information indicating the communication state of the image data captured by the second image pickup means in the state where the first display control step is executed, the image is captured by the received second image pickup means. A second state indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped based on the information indicating the communication state of the image data is displayed on the display screen. The second display control step to be displayed on the
In the state where the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is obtained by the session. The transmission step to send to the second transmission terminal,
And run
A third state indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped is displayed on the display screen by the second icon.
By accepting the pressing of the second icon, the communication of the image data captured by the first imaging means is started, and the display is changed from the second icon to the first icon. Display method to be displayed. Between a first transmission terminal capable of communicating image data and audio data captured by the first imaging means and a second transmission terminal capable of communicating image data and audio data captured by the second imaging means. A session management means for managing a session for communicating established image data, and the second transmission terminal when the first transmission terminal and the second transmission terminal transmit and receive image data by the session. When the communication of the image data captured by the second imaging means is stopped in the transmission terminal, the information indicating the communication state of the image data captured by the second imaging means is transmitted to the first transmission terminal. The first transmission terminal connected via a network to a management system having a transmission means for transmitting.
A receiving means for receiving information indicating a communication state of image data captured by the second imaging means transmitted by the transmitting means,
It is possible to execute the first display control for displaying the first state indicating that the communication of the image data captured by the first image pickup means has been started on the display screen by the first icon, and the first display control is possible. In response to the reception of information indicating the communication state of the image data captured by the second image pickup means in the state where the display control of the above is executed, the image data captured by the second image pickup means received. Based on the information indicating the communication state, the display screen displays the second state indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped. Display control means that can execute the second display control,
In the state where the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is obtained by the session. Transmission means to transmit to the second transmission terminal,
It is a program that functions as
The display means displays on the display screen a third state indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped by the second icon. Let me
By accepting the pressing of the second icon to the first transmission terminal, communication of the image data captured by the first imaging means is started, and the second icon is changed to the first icon. A program that changes the display . The program according to claim 8 .
The display control means displays a second state on the display screen by displaying a message indicating that the communication of the image data captured by the second image pickup means from the second transmission terminal is stopped. A program featuring. The program according to claim 8 or 9 .
The display control means is characterized in that a first state indicating that the first transmission terminal is communicating image data captured by the first image pickup means is displayed on a display screen by displaying a message. Program to do. The program according to any one of claims 8 to 10 , wherein the communication of the image data is the communication of the image data used in the video conference. Established between a first base capable of communicating image data and audio data captured by the first imaging means and a second base capable of communicating image data and audio data captured by the second imaging means. Session management means for managing sessions for communicating image data,
When the first base and the second base are transmitting and receiving image data by the session, and the communication of the image data captured by the second imaging means is stopped at the second base. In addition, a transmission means for transmitting information indicating a communication state of image data captured by the second image pickup means to the first base.
A computer connected via a network to a management system that has
A receiving means for receiving information indicating a communication state of image data captured by the second imaging means transmitted by the transmitting means,
It is possible to execute the first display control for displaying the first state indicating that the communication of the image data captured by the first image pickup means has been started on the display screen by the first icon, and the first display control is possible. In response to the reception of information indicating the communication state of the image data captured by the second image pickup means in the state where the display control of the above is executed, the image data captured by the second image pickup means received. A second state indicating that the communication of the image data captured by the second image pickup means from the second base is stopped based on the information indicating the communication state is displayed on the display screen. Display control means that can execute display control of
In the state where the second display control is executed, when the first state is displayed by the first display control, the image data captured by the first image pickup means is obtained by the session. Transmission means to transmit to the second transmission terminal,
It is a program that functions as
The display control means displays a third state on the display screen by a second icon, indicating that the first image pickup means is turned off and the communication of the image data captured by the first image pickup means is stopped. Display,
By accepting the pressing of the second icon to the first transmission terminal, communication of the image data captured by the first imaging means is started, and the second icon is changed to the first icon. A program that changes the display . The program according to claim 12 .
The display control means displays a second state on the display screen by displaying a message indicating that the communication of the image data captured by the second image pickup means from the second base is stopped. Characterized program. The program according to claim 12 or 13 .
The display control means is characterized in that a first state indicating that the first base is communicating image data captured by the first image pickup means is displayed on a display screen by displaying a message. program. The program according to any one of claims 12 to 14 , wherein the communication of the image data is the communication of the image data used in the video conference.

Images