JP2023109747A - Transmission system, first transmission terminal, transmission method, display method, and program - Google Patents

Abstract

To provide a transmission system capable of solving a problem that, conventionally, only the image communication or voice communication was restricted in consideration of the surrounding environment of the destination terminal, and the image communication was not restricted in consideration of the image communication state of the destination terminal itself; and therefore, in a conventional request source terminal, when the destination terminal cannot support image communication, useless image communication is made.SOLUTION: A management system 50 creates a predetermined communication control message for each terminal to control image communication based on the image communication status information indicating the image communication status of each terminal, and a communication control message is sent to the terminal that controls image communication. With this, since a transmission management system can perform image communication according to the image communication status of the terminal on the other side to a given terminal, useless image communication can be prevented.SELECTED DRAWING: Figure 2

Description

The present invention relates to managing communication of content data for conversation between a plurality of transmission terminals.

An example of a transmission system that transmits and receives content data between a plurality of transmission terminals via a relay device is a video conference system that conducts a video conference via a communication network such as the Internet. Such a teleconferencing system is becoming more and more necessary as travel expenses and travel time are reduced in recent years. In such a video conference system, a plurality of video conference terminals, which are examples of transmission terminals, are used. A video conference can be realized by transmitting and receiving image data and audio data between these video conference terminals.

However, there are environments that are inappropriate for video conferences, such as private places such as homes. As a countermeasure for holding a video conference in such an environment, a transmission terminal has been proposed that obtains the environmental conditions in which the transmission terminal of the other party is placed and restricts image communication or voice communication (Patent Document 1). reference).

However, the transmission terminal disclosed in Patent Document 1 only regulates image communication or voice communication in consideration of the surrounding environment of the transmission terminal of the other party. It does not mean that image communication is regulated in consideration of the image communication state. That is, in the transmission terminal disclosed in Patent Document 1, the transmission terminal on the opposite side has a voice communication function but does not have an image communication function, or even if it has an image communication function, the image communication function is not provided. The image communication is not restricted in consideration of the image communication state such as no communication. Therefore, for example, even if the transmission terminal on the other side performs image communication when the transmission terminal on the other side is in the image communication state as described above, the transmission terminal on the other side cannot support image communication, so the transmission terminal does not perform image communication in vain. A problem arises as to what to do.

The invention according to claim 1 is a transmission management system for managing image communication in a plurality of transmission terminals capable of at least voice communication, wherein terminal identification information for identifying each of the plurality of transmission terminals is used for each transmission. terminal management means for managing image communication state information indicating the image communication state of the terminal; and creation means for creating a predetermined communication control message for controlling image communication by a predetermined transmission terminal based on the image communication state information and transmitting means for transmitting the created communication control message to a transmission terminal, among the plurality of transmission terminals, that controls the image communication.

The invention according to claim 6 is a transmission system comprising the transmission management system according to claim 2 or 3 and the transmission terminal, wherein the transmission terminal receives a communication control message transmitted from the transmission management system. and display control means for causing a predetermined display means to display a display message based on the received communication control message.

The invention according to claim 10 is a program characterized by causing the transmission management system to function as each means according to any one of claims 1 to 5.

The invention according to claim 11 is a program providing system that provides the program according to claim 10 to the transmission management system via a communication network.

According to a twelfth aspect of the invention, there is provided a maintenance system for performing maintenance on the transmission management system according to any one of the first to fifth aspects.

As described above, according to the present invention, the transmission management system generates a predetermined communication control message for controlling image communication by a predetermined transmission terminal based on the image communication state information indicating the image communication state of each transmission terminal. and transmits a communication control message to the transmission terminal that controls the image communication. As a result, the transmission management system can cause a predetermined transmission terminal to perform image communication according to the image communication state of the transmission terminal on the opposite side, thereby preventing unnecessary image communication. play.

1 is a schematic diagram of a transmission system according to a first embodiment of the present invention; FIG. FIG. 2 is a conceptual diagram showing the state of transmission and reception of image data, audio data, and various types of management information in a transmission system; FIG. 2 is a conceptual diagram for explaining image quality of image data; 1 is an external view of a terminal according to this embodiment; FIG. 3 is a hardware configuration diagram of a terminal according to the embodiment; FIG. 1 is a hardware configuration diagram of a management system, a relay device, or a program providing system according to this embodiment; FIG. 3 is a functional block diagram of each terminal, device and system that constitute the transmission system according to the present embodiment; FIG. FIG. 4 is a functional configuration diagram of a final narrowing-down unit; 4 is a functional configuration diagram of a primary narrowing-down unit; FIG. FIG. 4 is a conceptual diagram showing a change quality control table; 4 is a conceptual diagram showing a relay device management table; FIG. 4 is a conceptual diagram showing a terminal authentication management table; FIG. 4 is a conceptual diagram showing a terminal management table; FIG. 4 is a conceptual diagram showing a destination list management table; FIG. 4 is a conceptual diagram showing a session management table; FIG. 4 is a conceptual diagram showing an address priority management table; FIG. 4 is a conceptual diagram showing a transmission speed priority management table; FIG. 4 is a conceptual diagram showing a quality control table; FIG. FIG. 10 is a sequence diagram showing a process of managing state information indicating an operating state of each relay device; FIG. 10 is a sequence diagram showing processing in a preparatory stage for starting remote communication between terminals; FIG. 10 is a sequence diagram showing a process of narrowing down relay devices; FIG. 10 is a processing flow diagram showing processing for narrowing down relay devices; FIG. 10 is a diagram showing a calculation state of priority points when narrowing down relay devices; FIG. 10 is a sequence diagram showing a process in which a transmission terminal selects a relay device according to the first embodiment of the present invention; FIG. 10 is a processing flow diagram showing processing for selecting a relay device in a transmission terminal; FIG. 10 is a sequence diagram showing processing for creating, transmitting and receiving a communication control message; FIG. 4 is a flowchart mainly showing processing for creating a communication control message; FIG. 4 is a flow diagram showing in detail part of the process of creating a communication control message; FIG. 4 is a diagram showing an example of a teleconference screen displayed on a display of a transmission terminal; FIG. 4 is a diagram showing an example of a teleconference screen displayed on a display of a transmission terminal; FIG. 4 is a diagram showing an example of a teleconference screen displayed on a display of a transmission terminal; FIG. 4 is a diagram showing an example of a teleconference screen displayed on a display of a transmission terminal; FIG. 4 is a sequence diagram showing processing for transmitting and receiving image data and audio data between transmission terminals;

An embodiment of the present invention will be described below with reference to FIGS. 1 to 33. FIG. In this embodiment, the transmission terminal performs voice communication by transmitting voice data to the transmission terminal of the other party, and performs image communication by transmitting image data.

<<Overall configuration of the embodiment>>
FIG. 1 is a schematic diagram of a transmission system 1 according to this embodiment of the invention. FIG. 2 is a conceptual diagram showing the state of transmission and reception of image data, audio data, and various types of management information in a transmission system. FIG. 3 is a conceptual diagram for explaining image quality of image data.

Further, the transmission system includes a data providing system that unidirectionally transmits content data from one transmission terminal to another transmission terminal via a transmission management system, and a data transmission system that transmits information and data between a plurality of transmission terminals via a transmission management system. It includes a communication system that mutually conveys emotions and the like. This communication system is a system for mutually transmitting information and emotions between multiple communication terminals (equivalent to "transmission terminals") via a communication management system (equivalent to "transmission management system"). Examples include conference systems and videophone systems.

In this embodiment, assuming a video conference system as an example of a communication system, a video conference management system as an example of a communication management system, and a video conference terminal as an example of a communication terminal, a transmission system, a transmission management system, and a transmission terminal. That is, the transmission terminal and transmission management system of the present invention are applied not only to a video conference system, but also to a communication system or a transmission system. In the present embodiment, "teleconference" is described, but it is also called "video conference", and both have the same content.

First, the transmission system 1 shown in FIG. 1 includes a plurality of transmission terminals (10aa, 10ab, . . . ) and displays (120aa, 120ab, . , a plurality of relay devices (30a, 30b, 30c, 30d), a transmission management system 50, a program providing system 90, and a maintenance system 100. FIG.

A plurality of terminals 10 transmit and receive image data and audio data as an example of content data.

In the following description, "transmission terminal" is simply referred to as "terminal", and "transmission management system" is simply referred to as "management system". An arbitrary terminal among the plurality of terminals (10aa, 10ab, . . . ) is denoted as “terminal 10”, and an arbitrary display among the plurality of displays (120aa, 120ab, . , an arbitrary relay device among the plurality of relay devices (30a, 30b, 30c, 30d) is represented as "relay device 30". Further, a terminal as a request source requesting the start of a video conference is represented as a "request source terminal", and a terminal as a destination (relay destination) of the request is represented as a "destination terminal".

Also, as shown in FIG. 2, in the transmission system 1, a management information session sei for transmitting and receiving various kinds of management information is established between the request source terminal and the destination terminal via the management system 50. established. Also, between the request source terminal and the destination terminal, through the relay device 30, four types of data, ie, high-resolution image data, medium-resolution image data, low-resolution image data, and audio data are transmitted and received. are established. Here, these four sessions are collectively shown as an image/audio data session sed.

Here, the image resolution of the image data handled in this embodiment will be described. As shown in FIG. 3(a), a low-resolution image consisting of 160 horizontal pixels and 120 vertical pixels, serving as a base image, and a low-resolution image having horizontal pixels as shown in FIG. There is a medium resolution image consisting of 320 pixels by 240 pixels vertically and a high resolution image consisting of 640 pixels horizontally by 480 pixels vertically as shown in FIG. 3(c). Of these, when passing through the narrow band path, low image quality image data consisting of only low resolution image data as a base image is relayed. When the band is relatively wide, intermediate image quality image data composed of low resolution image data serving as a base image and intermediate resolution image data are relayed. Also, when the band is very wide, high-quality image data composed of low-resolution image data serving as the base image quality, medium-resolution image data, and high-resolution image data is relayed.

A relay device 30 shown in FIG. 1 relays content data between a plurality of terminals 10 . The management system 50 centrally manages login authentication from the terminal 10, management of the call status of the terminal 10, management of the destination list, etc., and management of the communication status of the relay device 30 and the like. An image of image data may be a moving image, a still image, or both a moving image and a still image.

A plurality of routers (70a, 70b, 70c, 70d, 70ab, 70cd) selects the optimum route for image data and audio data. An arbitrary router among the routers (70a, 70b, 70c, 70d, 70ab, 70cd) is hereinafter referred to as "router 70".

The program providing system 90 includes an HD (Hard Disk) 204, which will be described later, and stores terminal programs for realizing various functions in the terminal 10 (or causing the terminal 10 to function as various means). can send terminal programs to The HD 204 of the program providing system 90 also stores a relay device program for realizing various functions in the relay device 30 (or causing the relay device 30 to function as various means). A relay device program can be transmitted. Furthermore, the HD 204 of the program providing system 90 also stores a transmission management program for realizing various functions of the management system 50 (or causing the management system 50 to function as various means). A transmission management program can be sent.

The maintenance system 100 is a computer for maintaining, managing, or maintaining at least one of the terminal 10 , the relay device 30 , the management system 50 , and the program providing system 90 . For example, if the maintenance system 100 is installed in Japan and the terminal 10, relay device 30, management system 50, or program providing system 90 is installed outside the country, the maintenance system 100 can be remotely operated via the communication network 2. , the terminal 10 , the relay device 30 , the management system 50 , and the program providing system 90 . In addition, the maintenance system 100 can, without going through the communication network 2, store the model number, manufacturing number, sales destination, maintenance inspection, Or perform maintenance such as management of failure history.

By the way, the terminals (10aa, 10ab, 10ac, . . . ), the relay device 30a, and the router 70a are communicably connected by the LAN 2a. Terminals (10ba, 10bb, 10bc, . . . ), relay device 30b, and router 70b are communicably connected by LAN 2b. Also, the LAN 2a and the LAN 2b are communicably connected by a dedicated line 2ab including a router 70ab, and constructed within a predetermined area A. For example, region A is Japan, LAN 2a is built in the office in Tokyo, and LAN 2b is built in the office in Osaka.

On the other hand, the terminals (10ca, 10cb, 10cc, . A terminal 10d (a, 10db, 10dc, . Also, the LAN2c and LAN2d are communicably connected by a dedicated line 2cd including a router 70cd, and constructed within a predetermined area B. FIG. For example, Region B is the United States, LAN2c is built in the New York office, and LAN2d is in Washington DC. C. built within the company's premises. Region A and region B are communicably connected via the Internet 2i from routers (70ab, 70cd).

Also, the management system 50 and the program providing system 90 are communicably connected to the terminal 10 and the relay device 30 via the Internet 2i. The management system 50 and the program providing system 90 may be installed in area A or area B, or may be installed in areas other than these.

In this embodiment, the communication network 2 of this embodiment is constructed by LAN2a, LAN2b, dedicated line 2ab, Internet 2i, dedicated line 2cd, LAN2c, and LAN2d. In this communication network 2, there may be places where wireless communication such as WiFi (Wireless Fidelity) and Bluetooth (registered trademark) is performed in addition to wired communication.

In addition, in FIG. 1, four sets of numbers shown under each terminal 10, each relay device 30, management system 50, each router 70, and program providing system 90 are simple IP addresses in general IPv4. clearly shown. For example, the IP address of the terminal lOaa is "1.2.1.3". Also, IPv6 may be used instead of IPv4, but IPv4 will be used to simplify the description.

Each terminal 10 can be used not only for calls between multiple offices and between different rooms in the same office, but also for calls within the same room, outdoor-to-indoor or outdoor-to-outdoor calls. may be used. When each terminal 10 is used outdoors, wireless communication such as a mobile phone communication network is performed.

<<Hardware Configuration of Embodiment>>
Next, the hardware configuration of this embodiment will be described. FIG. 4 is an external view of the terminal 10 according to this embodiment. Hereinafter, the longitudinal direction of the terminal 10 will be described as the X-axis direction, the direction orthogonal to the X-axis direction in the horizontal plane as the Y-axis direction, and the direction orthogonal to the X-axis direction and the Y-axis direction (vertical direction) as the Z-axis direction.

As shown in FIG. 4, the terminal 10 comprises a housing 1100, an arm 1200, and a camera housing 1300. As shown in FIG. Of these, the front wall surface 1110 of the housing 1100 is provided with an air intake surface (not shown) formed by a plurality of air intake holes, and the rear wall surface 1120 of the housing 1100 is formed with a plurality of exhaust holes. An exhaust surface 1121 is provided. As a result, by driving the cooling fan built into the housing 1100 , the outside air behind the terminal 10 can be taken in via the intake surface (not shown) and exhausted to the rear of the terminal 10 via the exhaust surface 1121 . A sound collection hole 1131 is formed in the right side wall surface 1130 of the housing 1100, and sounds such as voice, noise, and noise can be collected by a built-in microphone 114, which will be described later.

An operation panel 1150 is formed on the right wall surface 1130 side of the housing 1100 . This operation panel 1150 is provided with a plurality of operation buttons (108a to 108e) described later, a power switch 109 described later, and an alarm lamp 119 described later, and outputs sound from a built-in speaker 115 described later. A sound output surface 1151 is formed by a plurality of sound output holes for passage. Further, on the left wall surface 1140 side of the housing 1100, an accommodating portion 1160 is formed as a concave portion for accommodating the arm 1200 and the camera housing 1300. As shown in FIG. A right wall surface 1130 of the housing 1100 is provided with a plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device connection I/F 118, which will be described later. On the other hand, left side wall surface 1140 of housing 1100 is provided with a connection port (not shown) for electrically connecting cable 120c for display 120 to external device connection I/F 118, which will be described later.

In the following description, "operation button 108" is used to indicate an arbitrary operation button among the operation buttons (108a to 108e), and " A connection port 1132" will be used for explanation.

Next, the arm 1200 is attached to the housing 1100 via a torque hinge 1210, and the arm 1200 is configured to be vertically rotatable with respect to the housing 1100 within a tilt angle θ1 of 135 degrees. ing. FIG. 4 shows a state where the tilt angle θ1 is 90 degrees.

The camera housing 1300 is provided with a built-in camera 112, which will be described later, and can capture images of users, documents, rooms, and the like. A torque hinge 1310 is also formed on the camera housing 1300 . Camera housing 1300 is attached to arm 1200 via torque hinge 1310 . The camera housing 1300 is attached to the arm 1200 via a torque hinge 1310, and the camera housing 1300 is attached to the arm 1200 at a pan angle θ2 of ±180 degrees with the state shown in FIG. and a tilt angle θ3 of ±45 degrees in the vertical and horizontal directions.

Note that the relay device 30, the management system 50, and the program providing system 90 have the same external appearance as that of a general server/computer, so description of the external appearance will be omitted.

FIG. 5 is a hardware configuration diagram of the terminal 10 according to this embodiment of the present invention. As shown in FIG. 5, the terminal 10 of the present embodiment has a CPU (Central Processing Unit) 101 that controls the overall operation of the terminal 10, and a program used to drive the CPU 101 such as an IPL (Initial Program Loader). A stored ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103 used as a work area for the CPU 101, a flash memory 104 storing various data such as terminal programs, image data, and audio data, and control of the CPU 101 SSD (Solid State Drive) 105 that controls reading or writing of various data to the flash memory 104 according to the above, media drive 107 that controls reading or writing (storage) of data to the recording medium 106 such as a flash memory, destination of the terminal 10 , a power switch 109 for switching ON/OFF the power of the terminal 10, and a network I/F (Interface) 111 for data transmission using the communication network 2. It has

The terminal 10 also includes a built-in camera 112 that captures an image of a subject and obtains image data under the control of the CPU 101, an image sensor I/F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs sound, a sound , an audio input/output I/F 116 that processes input/output of audio signals between the microphone 114 and the speaker 115 under the control of the CPU 101, and an external display 120 under the control of the CPU 101. A display I/F 117 for transmission, an external device connection I/F 118 for connecting various external devices, an alarm lamp 119 for notifying an abnormality of various functions of the terminal 10, and each of the above components are shown in FIG. It has a bus line 110 such as an address bus and a data bus for electrically connecting to the . Note that some terminals 10 do not have the camera 112 and the imaging element I/F 113 depending on the model.

The display 120 is a display unit configured by liquid crystal or organic EL for displaying an image of a subject, icons for operation, and the like. Also, the display 120 is connected to the display I/F 117 by a cable 120c. This cable 120c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for HDMI (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signals. cable.

The camera 112 includes a lens and a solid-state imaging device that converts light into electric charge and digitizes an image (video) of a subject. etc. are used.

External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I/F 118 via a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven according to the control of the CPU 101 prior to the built-in camera 112 . Similarly, when an external microphone is connected or when an external speaker is connected, according to the control of the CPU 101, the external microphone and speaker are prioritized over the built-in microphone 114 and built-in speaker 115 respectively. Powered by an external speaker.

Note that the recording medium 106 is detachably attached to the terminal 10 . In addition, as long as it is a non-volatile memory from which data is read or written under the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

Further, the terminal program may be recorded in a computer-readable recording medium such as the recording medium 106 as an installable or executable file for distribution. Also, the terminal program may be stored in the ROM 102 instead of the flash memory 104 .

FIG. 6 is a hardware configuration diagram of the management system according to this embodiment of the present invention. The management system 50 stores various data such as a CPU 201 that controls the overall operation of the management system 50, a ROM 202 that stores programs used to drive the CPU 201 such as IPL, a RAM 203 that is used as a work area for the CPU 201, and transmission management programs. A hard disk drive (HDD) 205 that controls reading or writing of various data to or from the HD 204 under the control of the CPU 201, a media drive 207 that controls reading or writing (storage) of data to or from a recording medium 206 such as a flash memory. , a display 208 for displaying various information such as cursors, menus, windows, characters, or images; a network I/F 209 for transmitting data using the communication network 2; a keyboard 211 having a plurality of keys, a mouse 212 for selecting and executing various instructions, selecting an object to be processed, moving a cursor, etc., and a CD-ROM (Compact Disc Read Only Memory) as an example of a removable recording medium. 213, a CD-ROM drive 214 for controlling reading and writing of various data, and bus lines 210 such as an address bus and a data bus for electrically connecting the above components as shown in FIG. It has

The transmission management program may be recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 in an installable format or an executable format and distributed. . Also, the transmission management program may be stored in the ROM 202 instead of the HD 204 .

Also, since the relay device 30 has the same hardware configuration as the management system 50, the description thereof will be omitted. However, a relay device program for controlling the relay device 30 is recorded in the HD 204 . In this case as well, the relay device program may be recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 in an installable or executable format and distributed. good. Also, the relay device program may be stored in the ROM 202 instead of the HD 204 .

Also, since the program providing system 90 and the maintenance system 100 have the same hardware configuration as the management system 50, the description thereof will be omitted. However, the HD 204 stores a program providing program for controlling the program providing system 90 . In this case as well, the program providing program may be recorded in a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 in an installable format or executable format and distributed. good. Also, the program for the program providing system may be stored in the ROM 202 instead of the HD 204 .

As another example of the detachable recording medium, a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), and a Blu-ray disc is configured to be recorded and provided. You may

<<Function Configuration of Embodiment>>
Next, the functional configuration of this embodiment will be described. In this embodiment, when there is a delay in receiving the image data at the terminal 10 as the destination (relay destination), the image resolution of the image data is changed by the relay device 30, and then the image data is sent to the terminal 10 as the relay destination. A case of transmitting image data will be described.

FIG. 7 is a functional block diagram of each terminal, device and system that constitute the transmission system 1 of this embodiment. In FIG. 7, the terminal 10, the relay device 30, and the management system 50 are connected via the communication network 2 so as to be able to perform data communication. The program providing system 90 shown in FIG. 1 is omitted in FIG. 7 because it is not directly related to teleconference communications.

<Functional configuration of terminal>
The terminal 10 includes a transmission/reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14, an audio input unit 15a, an audio output unit 15b, a final selection unit 16, a display control unit 17, a delay detection unit 18, a storage · It has a reading processing unit 19 and a message creating unit 20 . Each of these units is a function realized by any one of the components shown in FIG. It is a means that works. The terminal 10 also has a storage unit 1000 constructed by the RAM 103 shown in FIG. 5 and the flash memory 104 shown in FIG.

(Each functional configuration of the terminal)
Next, each functional configuration of the terminal 10 will be described in detail with reference to FIGS. 5 and 7. FIG. In the following description of each functional configuration of the terminal 10, the relationship with the main components for realizing each functional configuration of the terminal 10 among the components shown in FIG. 5 will also be described. .

The transmitting/receiving unit 11 of the terminal 10 shown in FIG. 7 is realized by commands from the CPU 101 shown in FIG. Sends and receives various data (or information) to and from terminals, devices, or systems. The transmitting/receiving unit 11 starts receiving status information indicating the status of each terminal as a destination candidate from the management system 50 before starting a call with a desired destination terminal. This state information includes not only the operating state (online or offline state) of each terminal 10, but also whether it is possible to make a call even if it is online, whether the user is on a call, whether the user is away from the seat, and so on. Shows the detailed status of This state information is not only the operating state of each terminal 10, but also the cable 120c is disconnected from the terminal 10, the terminal 10 outputs sound but does not output an image, and the state information does not output sound. (MUTE) and various other states. In the following, as an example, a case where the state information indicates the operating state will be described.

The operation input reception unit 12 is implemented by commands from the CPU 101 shown in FIG. 5, and the operation buttons 108 and power switch 109 shown in FIG. 5, and receives various inputs from the user. For example, when the user turns on the power switch 109 shown in FIG. 5, the operation input reception unit 12 shown in FIG. 7 receives the power ON and turns on the power.

The login request unit 13 is realized by a command from the CPU 101 shown in FIG. and the current IP address of the requesting terminal are automatically transmitted. Further, when the user turns off the power switch 109 from the ON state, the transmission/reception unit 11 transmits state information indicating that the power is to be turned off to the management system 50, and then the operation input reception unit 12 completely turns off the power. do. As a result, the management system 50 side can grasp that the power of the terminal 10 has changed from ON to OFF.

The imaging unit 14 is realized by the commands from the CPU 101 shown in FIG. 5, the camera 112 and the imaging device I/F 113 shown in FIG. Output data. It should be noted that some models of the terminal 10 do not include the imaging unit 14 .

The voice input unit 15a is realized by commands from the CPU 101 shown in FIG. 5 and the voice input/output I/F 116 shown in FIG. After that, audio data related to this audio signal is input. The audio output unit 15b is implemented by commands from the CPU 101 shown in FIG. 5 and the audio input/output I/F 116 shown in FIG. output sound from

The final narrowing-down unit 16 performs the final narrowing-down process of finally narrowing down the plurality of relay devices 30 to one relay device 30. In order to perform the final narrowing-down processing, the CPU 101 shown in FIG. A measuring unit 16a, a calculating unit 16b, and a final selecting unit 16c are implemented.

Of these, the measurement unit 16 a measures the reception date and time when the prior transmission information is received by the transmission/reception unit 11 for each piece of prior transmission information (described later) received by the transmission/reception unit 11 . For each piece of pre-transmission information whose reception date and time are measured by the measurement unit 16a, the calculation unit 16b calculates the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. Calculate the required time (T) from transmission to reception of The final selection unit 16c finally selects one relay device by selecting the relay device 30 to which the pre-transmission information that took the shortest required time among the required times calculated by the calculation unit 16b was relayed. .

The display control unit 17 is realized by commands from the CPU 101 shown in FIG. 5 and the display I/F 117 shown in FIG. It controls transmission of this combined image data to the display 120 . Further, the display control unit 17 can transmit the information of the destination list received from the management system 50 to the display 120 and display the destination list on the display 120 . Further, the display control unit 17 causes the display 120 to display screens as shown in FIGS. The screens of FIGS. 29 to 32 will be described later.

The delay detector 18 is implemented by a command from the CPU 101 shown in FIG. 5, and detects the delay time (ms) of image data or audio data sent from another terminal 10 via the relay device 30. .

The storage/read processing unit 19 is executed by commands from the CPU 101 shown in FIG. 5 and the SSD 105 shown in FIG. A process of reading various data stored in the unit 1000 is performed. The storage unit 1000 stores a terminal ID (Identification) for identifying the terminal 10, a password, and the like.

The message creation unit 20 is implemented by a command from the CPU 101 shown in FIG. Create a message (m). The image communication status information and image communication status message (m) will be described later.

The storage unit 1000 also stores image communication state information, which will be described later. Further, the storage unit 1000 overwrites and stores image data and audio data received during a call with a destination terminal each time they are received. Among them, an image is displayed on the display 120 by the image data before being overwritten, and sound is output from the speaker 115 by the audio data before being overwritten.

Note that the terminal ID in this embodiment and the relay device ID described later indicate identification information such as language, characters, symbols, or various marks used to uniquely identify the terminal 10 and the relay device 30, respectively. Also, the terminal ID and the relay device ID may be identification information in which at least two of the language, characters, symbols, and various signs are combined.

<Functional configuration of relay device>
The relay device 30 has a transmission/reception section 31 , a state detection section 32 , a data quality confirmation section 33 , a change quality control section 34 , a data quality change section 35 and a storage/read processing section 39 . Each of these units has a function realized or functions by any of the components shown in FIG. It is a means. Further, the relay device 30 has a storage unit 3000 configured by the RAM 203 shown in FIG. 6 and/or the HD 204 shown in FIG. Note that FIG. 10 is a conceptual diagram showing a change quality management table.

(change quality control table)
In the storage unit 3000, a change quality management DB (Data Base) 3001 configured by a change quality management table as shown in FIG. 10 is constructed. In the change quality management table, the IP address of the terminal 10 as the relay destination (destination) of the image data and the image quality of the image data relayed by the relay device 30 are associated with this relay destination and managed.

(Each functional configuration of the relay device)
Next, each functional configuration of the relay device 30 will be described in detail. In describing each functional configuration of the relay device 30, the relationship with the main components for realizing each functional configuration of the relay device 30 among the components shown in FIG. explain.

The transmission/reception unit 31 of the relay device 30 shown in FIG. 7 is implemented by commands from the CPU 201 shown in FIG. 6 and the network I/F 209 shown in FIG. It transmits and receives various data (or information) to and from other terminals, devices, or systems.

The state detector 32 is implemented by a command from the CPU 201 shown in FIG. 6, and detects the operating state of the relay device 30 having the state detector 32 . The operating status includes "online", "offline", "busy", and "suspended".

The data quality confirmation unit 33 is realized by a command from the CPU 201 shown in FIG. 6, searches the change quality management DB 3001 (see FIG. 10) using the IP address of the destination terminal as a search key, and receives a corresponding relay. By extracting the image quality of the image data, the image quality of the relayed image data is confirmed.

The change quality control unit 34 is realized by a command from the CPU 201 shown in FIG. For example, a video conference can be held by transmitting and receiving high-quality image data between a request source terminal (terminal 10aa) with a terminal ID of "01aa" and a destination terminal (terminal 10db) with a terminal ID of "01db". While the conference is being held, the request source terminal (terminal 10bb) and the destination terminal (terminal 10ca), which conduct another video conference, start a video conference via the communication network 2, etc., so that the destination terminal (terminal 10db) When a delay occurs in receiving the image data, the relay device 30 lowers the image quality of the image data that has been relayed so far from high image quality to medium image quality. In such a case, the content of the change quality management DB 3001 is changed so that the image quality of the image data relayed by the relay device 30 is lowered from high image quality to intermediate image quality based on quality information indicating intermediate image quality.

The data quality changing unit 35 is implemented by a command from the CPU 201 shown in FIG. change.

The storage/read processing unit 39 is realized by commands from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. process to read out various data.

<Functional configuration of management system>
The management system 50 includes a transmission/reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state extraction unit 55, a primary screening unit 56, a session management unit 57, a quality determination unit 58, and storage/read processing. It has a section 59 , a delay time management section 60 and a message creation section 61 . Each of these units is implemented or functions by any of the constituent elements shown in FIG. It is a means. The management system 50 also has a storage unit 5000 constructed by the HD 204 shown in FIG.

Note that FIG. 8 is a functional configuration diagram of the final screening section. FIG. 9 is a functional configuration diagram of the primary screening unit. FIG. 11 is a conceptual diagram showing a relay device management table. FIG. 12 is a conceptual diagram showing a terminal authentication management table. FIG. 13 is a conceptual diagram showing a terminal management table. FIG. 14 is a conceptual diagram showing a destination list management table. FIG. 15 is a conceptual diagram showing a session management table. FIG. 16 is a conceptual diagram showing an address priority management table. FIG. 17 is a conceptual diagram showing a transmission speed priority management table. FIG. 18 is a conceptual diagram showing a quality control table.

(relay device management table)
A relay device management DB 5001 configured by a relay device management table as shown in FIG. 11 is constructed in the storage unit 5000 . In this relay device management table, for each relay device ID of each relay device 30, the operation status of each relay device 30, the reception date and time when the status information indicating the operation status was received by the management system 50, the IP address of the relay device 30 , and the maximum data transmission speed (Mbps) in the relay device 30 are associated and managed. For example, in the relay device management table shown in FIG. 11, the relay device 30a with the relay device ID of "111a" is in the operating state of "online" and the date and time when the state information was received by the management system 50 is "2009. 13:00 on Nov. 10, 2009", the IP address of this relay device 30a is "1.2.1.2" and the maximum data transmission rate of this relay device 30a is 100 Mbps. there is

(Terminal authentication management table)
Furthermore, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table as shown in FIG. 12 is constructed. In this terminal authentication management table, each password is associated with each terminal ID of all the terminals 10 managed by the management system 50 and managed. For example, the terminal authentication management table shown in FIG. 12 indicates that the terminal ID of the terminal 10aa is "01aa" and the password is "aaaa".

(terminal management table)
A terminal management DB 5003 configured by a terminal management table as shown in FIG. 13 is constructed in the storage unit 5000 . In this terminal management table, for each terminal ID of each terminal 10, the management system 50 stores the destination name when each terminal 10 is the destination, the operation status of each terminal 10, the image communication status described later, and the login request information described later. The received date and time and the IP address of the terminal 10 are associated and managed. For example, in the terminal management table shown in FIG. 13, a terminal 10aa with a terminal ID of "01aa" has a terminal name of "Japan Tokyo office AA terminal" and an operating status of "online (transmitting)". The image communication status is "ON", the date and time when the login request information was received by the management system 50 is "13:40 on November 10, 2009", and the IP address of this terminal 10aa is "1.2.1.3". ” is shown.

Here, the image communication state information will be explained. The image communication state information indicates the state of image communication of the terminal 10, and has the following four patterns.
ON: State in which the terminal 10 is in image communication OFF: State in which the terminal 10 is in suspension of image communication (However, the terminal 10 has an image communication function, which is an imaging unit 14 for performing image communication.)
NA: A state in which the terminal 10 does not have an image communication function (the terminal 10 does not have an image communication function, which is the imaging unit 14 for performing image communication.)
DC: state in which the image communication state is unknown (a state in which the image communication state has not been sent from the terminal 10 because the terminal 10 is offline or the like, or an initial state).
Of these, the image communication state information indicating DC is initially stored in the storage unit 1000, and the imaging unit 14 overwrites the image communication state information indicating ON or OFF from DC. Image communication state information indicating NA is stored in the storage unit 1000 in advance when the terminal 10 is shipped from the factory.

(Address list management table)
Furthermore, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table as shown in FIG. 14 is constructed. In this destination list management table, all the terminal IDs of the destination terminals registered as destination terminal candidates are associated with the terminal ID of the request source terminal requesting the start of a call in the video conference and managed. For example, in the destination list management table shown in FIG. The candidates are the terminal 10ab with the terminal ID "01ab", the terminal 10ba with the terminal ID "01ba", and the terminal 10bb with the terminal ID "01bb". This destination terminal candidate is updated by being added or deleted in response to an addition or deletion request to the management system 50 from any request source terminal.

(session management table)
A session management DB 5005 configured by a session management table as shown in FIG. 15 is constructed in this storage unit 5000 . In this session management table, for each selection session ID used to execute a session for selecting the relay device 30, the relay device ID of the relay device 30 used for relaying image data and audio data, the request source terminal A terminal ID, a terminal ID of a destination terminal, a reception delay time (ms) when image data is received at the destination terminal, and delay information indicating the delay time are sent from the destination terminal to the management system. The reception date and time received at 50 are associated and managed. For example, in the session management table shown in FIG. 15, the relay device 30a (relay device ID “111a”) selected in the session executed using the selection session ID “se1” has the terminal ID “01aa”. and the destination terminal (terminal 10db) with a terminal ID of "01db". It is shown that the delay time of the image data at "14:00 on November 10" is 200 (ms). When a teleconference is held between two terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal instead of the destination terminal. However, when a teleconference is held between three or more terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the terminal 10 on the receiving side of the image data and audio data.

(Address Priority Management Table)
Further, in the storage unit 5000, a priority management DB 5006 configured by an address priority management table as shown in FIG. 16 is constructed. In this address priority management table, in an arbitrary terminal 10 and an arbitrary relay device 30, the more "same" the four sets of dot addresses in the general IPv4 IP address, the higher the address priority. Dot address similarity and address priority are associated and managed so that the degree of point increases. The "same" means that the dot address portions are the same, and the "different" means that the dot address portions are different. For example, in the address priority management table shown in FIG. 16, when the IP address has the same three values from the top to the bottom of the dot address, the point of the address priority is "5". If the IP address has the same two values from the upper to the lower positions of the dot address, the point of address priority is "3". In this case, whether the lowest dot address values are the same or not is irrelevant to the priority. In the case of IP addresses having the same highest value of the dot address and different second highest value, the address priority point is "1". In this case, whether or not the third highest dot address and the lowest dot address have the same value is irrelevant to the priority. In the case of IP addresses with different highest dot addresses, the point of address priority is "0". In this case, whether or not the values of the second, third, and lowest dot addresses are the same is irrelevant to the priority.

(Transmission speed priority management table)
The priority management DB 5006 constructed in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. In this transmission speed priority management table, the maximum data transmission speed and the transmission speed priority are set so that the higher the maximum data transmission speed (Mbps) value in the relay device 30, the higher the transmission speed priority point. Associated and managed. For example, in the transmission speed priority management table shown in FIG. 17, when the maximum data transmission speed in the relay device 30 is 1000 Mbps or higher, the transmission speed priority point is "5". When the maximum data transmission speed in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission speed priority point is "3". When the maximum data transmission speed in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission speed priority point is "1". When the maximum data transmission speed in the relay device 30 is less than 10 Mbps, the transmission speed priority point is "0".

(quality control table)
Further, in the storage unit 5000, a quality control DB 5007 is built up with a quality control table as shown in FIG. In this quality control table, the delay time of image data and the image quality of image data are set so that the longer the delay time (ms) of the image data in the request source terminal or the destination terminal, the lower the image quality of the image data relayed by the relay device 30 . (image quality) are associated and managed.

(Each functional configuration of the management system)
Next, each functional configuration of the management system 50 will be described in detail. In describing each functional configuration of the management system 50, the relationship with the main components for realizing each functional configuration of the management system 50 among the components shown in FIG. explain.

The transmission/reception unit 51 is executed by commands from the CPU 201 shown in FIG. 6 and the network I/F 209 shown in FIG. or information).

Terminal authentication unit 52 is implemented by commands from CPU 201 shown in FIG. terminal authentication management DB 5002 and determines whether the same terminal ID and password are managed in the terminal authentication management DB 5002, thereby performing terminal authentication.

The state management unit 53 is implemented by commands from the CPU 201 shown in FIG. 6, and stores the request source terminal in the terminal management DB 5003 (see FIG. 13) in order to manage the operating state of the request source terminal that has requested login. , the operation status of the requesting terminal, the date and time when the login request information was received by the management system 50, and the IP address of the requesting terminal are stored and managed in association with each other. In addition, the state management unit 53 performs terminal management based on the state information sent from the terminal 10 indicating that the power is to be turned off when the user turns the power switch 109 of the terminal 10 from ON to OFF. The operating state indicating online of the DB 5003 (see FIG. 13) is changed to offline.

The terminal extracting unit 54 is implemented by a command from the CPU 201 shown in FIG. 6, and searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the requesting terminal that requested the login as a key, and extracts the requesting terminal. The terminal ID is extracted by reading out the terminal ID of the candidate of the destination terminal that can be called with. The terminal extracting unit 54 also searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the request source terminal that has made the login request as a key, and registers the terminal ID of the request source terminal as a destination terminal candidate. It also extracts the terminal IDs of other requesting terminals.

Furthermore, the terminal extracting unit 54 searches the destination list management DB 5004 using the terminal ID of the terminal that has transmitted the image communication status message (m) as the terminal ID of the requesting terminal, and extracts all the terminal IDs of the corresponding destination terminals. Extract.

The terminal state extraction unit 55 is implemented by a command from the CPU 201 shown in FIG. ), and read out the operating state for each terminal ID extracted by the terminal extraction unit 54 . As a result, the terminal state extracting unit 55 can acquire the operating state of the destination terminal candidates that can communicate with the request source terminal that has made the login request. In addition, the terminal state extraction unit 55 searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and acquires the operating state of the request source terminal that has made the login request.

Furthermore, the terminal state extraction unit 55 extracts the corresponding image communication state information by searching the terminal management DB 5003 (see FIG. 13) using the terminal ID of the destination terminal as a search key. In addition, the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and determines the IP address of the terminal 10 in image communication, the terminal 10 in which image communication is stopped, or the terminal 10 having the image communication function. Extract addresses.

The primary narrowing-down unit 56 is implemented by a command from the CPU 201 shown in FIG. Perform the previous primary narrowing down process. For this purpose, the primary narrowing-down unit 56 is configured, as shown in FIG. 9, by a command from the CPU 201 shown in FIG. A unit 56c and a priority determination unit 56d are realized.

Among these, the selection session ID generating unit 56 a generates a selection session ID used for executing a session for selecting the relay device 30 . The terminal IP address extraction unit 56b extracts the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the requesting terminal and the terminal ID of the destination terminal included in the start request information sent from the requesting terminal. By searching, the IP address of each corresponding terminal 10 is extracted. The primary selection unit 56c selects the relay device ID of the relay device 30 whose operation status is "online" among the relay devices 30 managed by the relay device management DB 5001 (see FIG. 11). A device 30 is selected.

Further, the primary selection unit 56c searches the relay device management DB 5001 (see FIG. 11) based on the IP address of the request source terminal and the IP address of the destination terminal extracted by the terminal IP address extraction unit 56b. , for each dot address of the IP address of the selected relay device 30, it is investigated whether it is the same as or different from each dot address in each of the IP addresses of the request source terminal and the destination terminal. Further, for each relay device, the primary selection unit 56c selects the two relay devices with the highest points among the integrated points obtained by integrating the higher address priority points for the terminal 10 and the transmission speed priority points. By selecting 30, a further selection of relay devices 30 is made. In the present embodiment, the top two relay devices 30 with the highest points are selected, but the present invention is not limited to this. Three or more relay devices 30 may be selected.

The priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16) and determines the address priority points for each of the relay devices 30 investigated by the primary selection unit 56c. Also, the priority determining unit 56d searches the priority management DB 5006 (see FIG. 17) based on the maximum data transmission rate of each relay device 30 managed in the relay device management DB 5001 (see FIG. 11). , the point of transmission speed priority is determined for each relay device 30 narrowed down by the first narrowing process by the primary selector 56c.

Subsequently, the session management unit 57 is realized by commands from the CPU 201 shown in FIG. session ID, the terminal ID of the requesting terminal, and the terminal ID of the destination terminal are stored and managed in association with each other. In addition, the session management unit 57 stores the relay device of the relay device 30 finally selected as one by the final selection unit 16c of the terminal 10 for each selection session ID in the session management DB 5005 (see FIG. 15). Store and manage IDs.

The quality determination unit 58 searches the quality control DB 5007 (see FIG. 18) using the delay time as a search key, extracts the image quality of the corresponding image data, and determines the image quality of the image data to be relayed by the relay device 30. do.

The storage/read processing unit 59 is executed by the instructions from the CPU 201 shown in FIG. 6 and the HDD 205 shown in FIG. process to read out various data.

The delay time management unit 60 is implemented by commands from the CPU 201 shown in FIG. 6, and searches the terminal management DB 5003 (see FIG. 13) using the IP address of the destination terminal as a search key to find the corresponding terminal. ID is extracted, and in the session management table of the session management DB 5005 (see FIG. 15), the delay time indicated by the delay information is stored in the delay time field of the record containing the extracted terminal ID. and manage.

The message creation unit 61 creates a communication control message according to the image communication state of each terminal 30 . The process of creating this communication control message will be described later in detail with reference to FIG. Also, the message creating unit 61 acquires the image communication state information relating to the terminal 10 as the transmission source and the IP address of the terminal 10 from the image communication state message (m) sent from the terminal 10 . Further, the message creation unit 61 adds the IP address obtained from the image communication status message (m) to the attribute portion of the image communication status corresponding to the IP address acquired from the image communication status message (m) in the terminal management DB 5003 (see FIG. 13). Overwrite and rewrite the acquired image communication state information.

Also, the message creating unit 61 extracts the terminal ID corresponding to the IP address obtained from the image communication status message (m) from the terminal management DB 5003 (see FIG. 13). Furthermore, the message creation unit 61 refers to the image communication status information extracted from the terminal management DB 5003 (see FIG. 13), and determines whether the request source terminal and all destination terminals each have an image communication function. do.

<<Processing or operation of the embodiment>>
Next, a processing method in the transmission system 1 according to this embodiment will be described with reference to FIGS. 19 to 33. FIG. FIG. 19 is a sequence diagram showing a process of managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50. As shown in FIG. FIG. 20 is a sequence diagram showing preparatory processing for starting a call between a plurality of terminals 10. As shown in FIG. FIG. 21 is a sequence diagram showing the process of narrowing down the relay devices 30. As shown in FIG. FIG. 22 is a processing flow diagram showing processing for narrowing down the relay device 30 . FIG. 23 is a diagram showing the calculation state of the priority points when narrowing down the relay device 30. As shown in FIG. FIG. 24 is a sequence diagram showing the process of selecting the relay device 30 by the terminal 10. As shown in FIG. FIG. 25 is a processing flow diagram showing processing for selecting the relay device 30 at the terminal. FIG. 33 is a sequence diagram showing processing for transmitting and receiving image data and audio data between terminals.

First, with reference to FIG. 19, processing for managing status information indicating the status of each relay device 30 transmitted from each relay device 30 to the management system 50 will be described. First, in each relay device 30, the state detector 32 shown in FIG. 7 periodically detects the operating state of the relay device 30 (steps S1-1 to S1-4). In order for the management system 50 to manage the operating state of each relay device 30 in real time, the transmission/reception unit 31 of each relay device 30 periodically transmits each state information to the management system 50 via the communication network 2. (Steps S2-1 to 4). Each piece of status information includes the relay device ID of each relay device 30 and the operating state detected by the state detector 32 of the relay device 30 associated with each relay device ID. In this embodiment, the relay devices (30a, 30b, and 30d) are operating normally and are "online", while the relay device 30c is in operation, but the relay operation of the relay device 30c is being performed. A case where the program to be executed is "offline" due to some trouble is shown.

Next, in the management system 50, the transmission/reception unit 51 receives each state information sent from each relay device 30, and the relay device management DB 5001 (see FIG. ) stores and manages the status information for each relay device ID (steps S3-1 to S3-4). As a result, the operating status of "online", "offline", or "out of order" is stored and managed for each relay device ID in the relay device management table as shown in FIG. At this time, the date and time when the status information was received by the management system 50 is also stored and managed for each relay device ID. If the status information is not sent from the relay device 30, either the operating status field and the reception date and time field in each record of the relay device management table shown in FIG. 11 are blank, or The operation status and reception date and time at the time of the previous reception are indicated respectively.

Next, with reference to FIG. 20, transmission/reception processing of each piece of management information in a preparatory stage before starting a call between the terminal 10aa and the terminal 10db will be described. 20, various types of management information are transmitted and received by the management information session sei shown in FIG.

First, when the user turns on the power switch 109 shown in FIG. 5, the operation input reception unit 12 shown in FIG. 7 receives the power ON and turns on the power (step S21). Triggered by the reception of the power ON, the login request unit 13 automatically transmits login request information indicating a login request from the transmission/reception unit 11 to the management system 50 via the communication network 2 (step S22). This login request information includes a terminal ID and a password for identifying the terminal 10aa, which is the request source. These terminal ID and password are data read out from the storage section 1000 via the storage/readout processing section 19 and sent to the transmission/reception section 11 . When the login request information is transmitted from the terminal 10aa to the management system 50, the management system 50 on the receiving side can grasp the IP address of the terminal 10ab on the transmitting side.

Next, the terminal authentication unit 52 of the management system 50 retrieves the terminal authentication management DB 5002 (see FIG. 12) of the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission/reception unit 51 as search keys. ) and determines whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). Since the same terminal ID and password are managed by this terminal authentication unit 52, when it is determined that the login request is from a terminal 10 having valid use authority, the state management unit 53 The DB 5003 (see FIG. 13) stores the terminal ID of the terminal 10aa, the operation status, the date and time when the login request information was received, and the IP address of the terminal 10aa in association with each other (step S24). As a result, the terminal management table shown in FIG. 13 contains the terminal ID "01aa", the operating state "online", the received date and time "2009.11.10.13:40", and the IP address of the terminal 10aa "1". .2.1.3” will be associated and managed.

Then, the transmitting/receiving unit 51 of the management system 50 transmits the authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the request source terminal (terminal 10aa) requesting the login via the communication network 2. (step S25). In the present embodiment, the case where the terminal authentication unit 52 determines that the terminal has a legitimate usage right will be described below.

The terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the request source terminal (terminal 10aa) that requested the login as a search key. ) are extracted by reading terminal IDs of candidate destination terminals (step S26). Here, the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminals (terminals 10ab, 10ba, 10db) corresponding to the terminal ID “01aa” of the request source terminal (terminal 10aa) are extracted. will be

Next, the terminal state extracting unit 55 extracts terminal IDs (“01ab”, “01ba”, “01db”) of destination terminal candidates extracted by the terminal extracting unit 54 as search keys, and searches the terminal management DB 5003 (FIG. 13). ), and read out the operating status (“offline”, “online”, “online”) for each terminal ID extracted by the terminal extracting unit 54, so that each operation of the terminal (10ab, 10ba, 10db) The state is acquired (step S27).

Next, the transmitting/receiving unit 51 retrieves the terminal IDs (“01ab”, “01ba”, “01db”) as the search keys used in step S27 and the corresponding destination terminals (terminals 10ab, 10ba, 10db). destination status information including the operation status (“offline”, “online”, “online”) of each of the above is transmitted to the requesting terminal (terminal 10aa) via the communication network 2 (step S28). As a result, the request source terminal (terminal 10aa) is notified of the current operating states ("offline , "online", and "online").

Further, the terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID "01aa" of the request source terminal (terminal 10aa) that has made the login request as a search key, and The terminal IDs of other requesting terminals that have registered the terminal ID "01aa" of the terminal (terminal 10aa) as a destination terminal candidate are extracted (step S29). In the destination list management table shown in FIG. 14, the extracted terminal IDs of other requesting terminals are "01ab", "01ba", and "01db".

Next, the terminal state extraction unit 55 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the terminal ID "01aa" of the request source terminal (terminal 10aa) that has sent the login request as a search key. , the operation status of the request source terminal (terminal 10aa) that has requested the login is acquired (step S30).

Then, the transmission/reception unit 51 selects the terminal management DB 5003 (FIG. 13) among the terminals (10ab, 10ba, 10db) associated with the terminal IDs ("01ab", "01ba", "01db") extracted in step S29. ), the terminals (10ba, 10db) whose operating status is "online" include the terminal ID "01aa" of the requesting terminal (terminal 10aa) acquired in step S30 and the operating status "online". Destination status information is transmitted (steps S31-1, 2). When the transmission/reception unit 51 transmits the destination status information to the terminals (10ba, 10db), management is performed using the terminal management table shown in FIG. Refers to the IP address of the terminal that is As a result, each of the other destination terminals (terminals 10db, 10ba) that can communicate with the request source terminal (terminal 10aa) that has requested the login as a destination receives the request source terminal (terminal 10aa) that has requested the login. Terminal ID "01aa" and operating status "online" can be conveyed.

On the other hand, in the other terminal 10 as well, when the user turns on the power switch 109 shown in FIG. 6 in the same manner as in step 21, the operation input reception unit 12 shown in FIG. , the same processing as the processing in steps S22 to S31-1 and S31-2 is performed, so the description thereof will be omitted.

Next, processing for narrowing down the relay device 30 will be described with reference to FIG. 21 . In addition, in FIG. 21, various types of management information are transmitted and received by the management information session sei shown in FIG. In the present embodiment, the request source terminal (terminal 10aa) selects terminals (10ba, 10db ) can make a call with at least one of Therefore, the case where the user of the request source terminal (terminal 10aa) selects to start a call with the destination terminal (terminal 10db) will be described below.

First, when the user presses the operation button 108 shown in FIG. 5 to select the terminal 10db, the operation input reception unit 12 shown in FIG. 7 starts a call with the destination terminal (terminal 10db). Receives a request to do so (step S41). Then, the transmitting/receiving unit 11 of the request source terminal (terminal 10aa) sends a start request including the terminal ID "01aa" of the terminal 10aa and the terminal ID "01db" of the destination terminal (terminal 10db) and indicating that the call is to be started. The information is transmitted to the management system 50 (step S42). As a result, the transmitting/receiving unit 51 of the management system 50 can receive the start request information and recognize the IP address "1.2.1.3" of the request source terminal (terminal 10aa) that is the transmission source. .

Based on the terminal ID "01aa" of the request source terminal (terminal 10aa) and the terminal ID "01db" of the destination terminal (terminal 10db) included in the start request information, the state management unit 53 sets the terminal management DB 5003 (see FIG. 13). ) in the terminal management table, the operating state fields of the records containing the terminal IDs "01aa" and "01db" are both changed to "busy" (step S43). In this state, the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) have not started a call, but are in a state of being in a call. When trying to talk with (terminal 10db), a notification sound or display indicating a so-called busy state is output.

Next, processing for executing a session for selecting the relay device 30 will be described. First, the selection session ID generator 56a shown in FIG. 9 generates a selection session ID used for executing a session for selecting the relay device 30 (step S44). Then, the session management unit 57 stores the session management DB 5005 (see FIG. 15) of the storage unit 5000 with the selection session ID “se1” generated in step S44 and the terminal ID “01aa” of the request source terminal (terminal 10aa). , and the terminal ID "01db" of the destination terminal (terminal 10db) are stored and managed in association with each other (step S45).

Next, the primary screening unit 56 of the management system 50 shown in FIG. (Terminal 10db) is primarily narrowed down to the relay device 30 for relaying the call (step S46).

Here, the processing in step S46 will be described in more detail with reference to FIGS. 9 and 22. FIG. First, the terminal IP address extraction unit 56b shown in FIG. 9 extracts the terminal ID "01aa" of the requesting terminal (terminal 10aa) included in the start communication information sent from the requesting terminal (terminal 10aa). , and the terminal ID "01db" of the destination terminal (terminal 10db), the IP addresses ("1.2 .1.3”, “1.3.2.4”) are extracted (step S46-1).

Next, the primary selection unit 56c selects each of the relay devices (30a, 30b, 30d) that are "online" among the operating states of the relay devices 30 managed by the relay device management DB 5001 (see FIG. 11). A relay device ID (111a, 111b, 111d) is selected (step S46-2). Further, the primary selection unit 56c selects the IP address "1.2.1.3" of the request source terminal (terminal 10aa) and the IP address "1. 3.2.4”, each IP address (“ 1.2.1.2.”, “1.2.2.2”, and “1.3.2.2”), the request source terminal (terminal 10aa) and the destination terminal (terminal 10db ) are the same as or different from each dot address in each IP address (“1.2.1.3”, “1.3.2.4”) (step S46-3).

Next, the priority determining unit 56d refers to the priority management DB 5006 (see FIG. 16), and determines the address priority points for each relay device (30a, 30b, 30d) investigated in step 46-3. is determined (step S46-4). If the result of this decision processing is expressed in a table, it will be in a state as shown in FIG. FIG. 23 is a diagram showing the calculation state of the priority points when narrowing down the relay device 30. As shown in FIG. In FIG. 23, address priority points, transmission speed priority points, and integration points are shown for each relay device ID. In addition, the points of the address priority further indicate the points of each relay device 30 for the request source terminal (terminal 10aa) and the destination terminal (terminal 10db). The integrated point is the sum of the higher of the two address priority points and the transmission speed priority point.

In the present embodiment, the IP address "1.2.1.2" of the relay device 30a corresponds to the IP address "1.2.1.3" of the request source terminal (terminal 10aa). .Different", the address priority point is "5", as shown in FIG. Also, as shown in FIG. 1, the IP address "1.2.1.2" of the relay device 30a corresponds to the IP address "1.3.2.4" of the destination terminal (terminal 10db). As shown in FIG. 16, since it is "same. different. different. different", the address priority point becomes "1" as shown in FIG. Also, the IP address "1.2.2.2" of the relay device 30b is "same. Therefore, the address priority point is "3". The IP address "1.2.2.2" of the relay device 30b is "same.different.same.different" with respect to the IP address "1.3.2.4" of the destination terminal (terminal 10db). Therefore, the address priority point is "1". Further, the IP address "1.3.2.2" of the relay device 30d is "same. Therefore, the address priority point is "1". The IP address "1.3.2.2" of the relay device 30d is "same.same.same.different" to the IP address "1.3.2.4" of the destination terminal (terminal 10db). Therefore, the address priority point is "5".

Next, returning to FIG. 22, the priority determining unit 56d determines the priority management DB 5006 (FIG. 17) based on the maximum data transmission rate of each relay device 30 managed in the relay device management DB 5001 (see FIG. 11). (see step S46-5). In this embodiment, as shown in FIG. 11, the maximum data transmission speed of the relay device 30a is 100 (Mbps). It becomes 3 points. Similarly, the maximum data transmission speed of the relay device 30b is calculated to be 1000 (Mbps), so the transmission speed priority is 5 points. Similarly, since the maximum data transmission speed of the relay device 30d is calculated to be 10 (Mbps), the transmission speed priority is 1 point.

Next, the primary selection unit 56c integrates the higher one of the points of address priority among the terminals (10aa, 10db) and the point of transmission speed priority for each relay device (30a, 30b, 30d). Among the integrated points, the top two relay devices 30 with the highest points are selected (step 46-6). In this embodiment, as shown in FIG. 23, the relay device IDs (111a, 111b, 111d) have integration points "8", "8", and "6", respectively, so that the relay device ID " 111a" and the relay apparatus 30b associated with the relay apparatus ID "111b" are selected.

When the narrowing down process in step S46 ends, the transmitting/receiving unit 51 shown in FIG. The relay device narrowing-down information for the purpose is transmitted (step S47). This relay device screening information includes the number of relay devices 30 narrowed down in step 46 "2", the terminal ID "01aa" of the request source terminal (terminal 10aa), and the selection session ID "se1". is As a result, the terminal 10db can grasp how many relay devices 30 exist and from which terminal 10 the request to start a call has been issued in the execution of the session with the selection session ID “se1”. At the same time, the IP address "1.1.1.2" of the management system 50, which is the transmission source of the relay narrowing-down information, can be grasped.

Then, the terminal lOdb transmits reception completion information indicating completion of reception of the relay device narrowing-down information from the transmission/reception unit 11 to the management system 50 via the communication network 2 (step S48). This reception completion information includes the session ID “se1”. As a result, the management system 50 notifies that the transmission of the number of relay devices being executed with the session ID "se1" has been completed, and sends the IP address "1.3.2.4" of the destination terminal (terminal 10db) that is the transmission source. ” can be grasped.

Next, a process in which the destination terminal (terminal 10db) selects the relay device 30 will be described with reference to FIGS. 24 and 25. FIG. In FIG. 24, various types of management information are transmitted and received by the management information session sei shown in FIG.

First, before starting a teleconference call, the management system 50 sends advance relay request information to each of the relay devices (30a, 30b) narrowed down in step S46 to request relay in advance. Send (step S61-1, 2). This preliminary relay request information includes a session ID “se1”, the IP address “1.2.1.3” of the request source terminal (terminal 10aa), and the IP address “1.3.2” of the destination terminal (terminal 10db). .4” is included. As a result, each relay device (30a, 30b) can grasp which session for selection belongs to, what the request source terminal is, and what the destination terminal is. The IP address "1.1.1.2" of the management system 50, which is the source of the transmission, can be grasped.

Next, each of the relay devices (30a, 30b) transmits from the transmission/reception unit 31 via the communication network 2 to the request source terminal (terminal 10aa) grasped in steps S61-1 and S61-2 before starting the call. Pre-transmission request information indicating that pre-transmission information including ping (Packet Internet Groper), which will be described later, is to be transmitted is transmitted to each relay device (30a, 30b) as its own device (steps S62-1, 2). . This advance transmission information includes the session ID “se1”. As a result, the request source terminal (terminal 10aa) recognizes that the pre-transmission information will be transmitted to each of the relay devices (30a and 30b) in the relay device 30 selection process executed with the session ID "se1". , IP addresses (“1.2.1.2” and “1.2.2.2”) of the relay devices (30a, 30b) that are the sources of the advance transmission request information.

Note that the management system 50 does not directly notify the request source terminal of the IP address of the destination terminal, but instead notifies the IP address of the destination terminal to the relay apparatus 30a as in step S61-1 above. As in step S61-2, the relay device 30a requests the request source terminal to transmit the advance transmission request information to its own device (relay device 30a) because each terminal 10 can This is to ensure security by not disclosing the IP address of 10.

Next, the requesting terminal (terminal 10aa) transmits advance transmission information from the transmitting/receiving section 11 to each relay device (30a, 30b) via the communication network 2 (steps S63-1, 2). Prior to the transmission of the image data and audio data, this advance transmission information is transmitted to the destination terminal (terminal 10db) via each relay device (30a, 30b) in place of the image data and audio data, This information is used to measure the time required from transmission by the request source terminal (terminal 10aa) to reception by the destination terminal (terminal 10db). In addition, the pre-transmission information includes a ping for confirming that the request source terminal (terminal 10aa), the relay devices (30a and 30b), and the destination terminal (terminal 10db) are communicably connected, The date and time when the pre-transmission information was transmitted from the terminal (terminal 10aa) and the session ID "se1" are included. As a result, each relay device (30a, 30b) can recognize that pre-transmission information has been sent in executing the session with the selection session ID "se1", and is the source of this pre-transmission information. The IP address "1.2.1.3" of the request source terminal (terminal 10aa) can be grasped.

Next, each relay device (30a, 30b) receives the IP address "1.3.2.4 ' (step S64-1, 2). As a result, the destination terminal (terminal 10db) can recognize that the pre-transmission information has been sent in executing the session with the session ID "se1", and is the source (relay source) of this pre-transmission information. The IP addresses (“1.2.1.2”, “1.2.2.2”) of the relay devices (30a, 30b) can be grasped.

Next, the final narrowing-down unit 16 of the destination terminal (terminal 10db) finally narrows down the destination terminal (terminal 10db) to one relay device 30 that relays the image data and the voice data in the teleconference call based on the pre-transmission information (step S65). ).

Here, the processing in step S65 will be described in more detail with reference to FIGS. 8 and 25. FIG. First, the measurement unit 16a of the final narrowing unit 16 shown in FIG. The date and time are measured (step S65-1). Next, for each piece of pre-transmission information for which the reception time has been measured, the calculation unit 16b calculates, based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information, The required time until reception is calculated (step S65-2). Next, the final selection unit 16c determines whether or not all pieces of pre-transmission information corresponding to the number of relay devices 30 scheduled to be relayed, “2”, have been received in the execution of the session with the session ID “se1” (step S65). -3). Then, if all of them have not been received (NO), the final selection unit 16c determines whether a predetermined time (here, one minute) has passed since the pre-transmission information was received by the terminal 10db (step S65-4). ). Further, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, if all have been received in step S65-3 (YES), or if the predetermined time has elapsed in step S65-4 (YES), the final selection unit 16c determines that the calculation unit 16b has calculated One of the relay devices 30 that relayed the pre-transmission information requiring the shortest required time is selected (step S65-5). In this embodiment, the relay device 30a is selected on the assumption that the pre-transmission information relayed by the relay device 30a has a shorter time from transmission to reception than the pre-transmission information relayed by the relay device 30b. is shown.

Next, the destination terminal (terminal 10db) transmits selection information indicating that the relay device 30a has been selected from the transmission/reception unit 11 to the management system 50 via the communication network 2 (step S66). This selection information includes the session ID "se1" and the relay device ID "111a" of the selected relay device 30a. As a result, the management system 50 can recognize that the relay device 30a has been selected in the execution of the session with the session ID "se1", and the IP address "1" of the destination terminal (terminal 10db) that is the transmission source of the selection information. .3.2.4” can be grasped.

Next, in the session management table of the session management DB 5005 (see FIG. 15), the session management unit 57 of the management system 50 adds 1 to the relay device ID field of the record containing the session ID "se1". The relay device ID "111a" of the relay device 30a selected as one is stored and managed (step S67). Then, the transmission/reception unit 51 of the management system 50 transmits relay start request information indicating a request to start relay to the relay device 30a via the communication network 2 (step S68). This relay start request information contains the IP addresses (“1.2.1.3”, “1.3.2.4”) of the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) to be relayed. It is included. As a result, the relay device 30a can recognize that the relay device 30a, which is the self-device, has been selected. Also, a session is established for communicating voice data (step S69). Therefore, the terminals (10aa, 10db) can start a teleconference call.

In step S47, the management system 50 transmits the relay device selection information to the destination terminal (terminal 10db). Although the device selection processing (step S65) is performed, the present invention is not limited to this. Up to S64-1 and S64-2, the sender and receiver of each information may be switched between the requesting terminal (terminal 10aa) and the destination terminal (terminal 10db). As a result, the request source terminal (terminal 10aa) can perform relay device selection processing instead of step S65, and can also transmit selection information instead of step S66.

26 to 28, when a video conference is held at another site (a plurality of terminals 10), after a communication control message is created and transmitted to each terminal 10, each terminal 10 performs communication control. Processing up to processing based on a message will be described. FIG. 26 shows a case where a teleconference is held using three terminals 10aa, 10ba, and 10db. FIG. 26 is a sequence diagram showing processing for creating, transmitting and receiving a communication control message. FIG. 27 is a flowchart mainly showing processing for creating a communication control message. FIG. 28 is a flow chart showing in detail a part of the process of creating a communication control message. In this embodiment, a case where three terminals (10aa, 10ba, 10db) as multiple bases hold a conference will be described.

First, as shown in FIG. 26, the message creation unit 20 of each terminal 10 (here, the terminal 10aa) reads out pre-stored image communication state information from the storage unit 1000, and performs this image communication. An image communication status message (m) containing status information is created (step S71). This image communication status message (m) contains the IP address "1.2.1.3" of this terminal 10aa. Note that the destination name may be included instead of the IP address. This image communication status message (m) can be created, for example, in XML (eXtensible Markup Language) format conforming to RFC3921 XMPP (eXtensible Messaging and Presence Protocol). A communication control message and a display control message, which will be described later, are also created in the same format.

Next, the transmission/reception unit 11 of the terminal lOaa transmits the image communication status message (m) created in step S71 to the management system 50 via the communication network 2 (step S72). As a result, the transmission/reception unit 51 of the management system 50 receives the image communication status message (m).

Next, the message creation unit 61 of the management system 50 creates a communication control message according to the image communication status of each terminal 30 (step S73). This step S73 will be described in detail with reference to FIG.

First, from the image communication status message (m) received by the transmission/reception part 51, the message creation unit 61 extracts the image communication status information related to the terminal 10aa as the transmission source and the IP address "1.2.1.3" of the terminal 10aa. ” is obtained (step S73-1). Then, in the terminal management DB 5003 (see FIG. 13), the message creation unit 61 adds the image acquired in step S73-1 to the attribute portion of the image communication state corresponding to the IP address acquired in step S73-1. The communication state information is overwritten and rewritten (step S73-2).

Next, the message creation unit 61 extracts the terminal ID corresponding to the IP address obtained in step S73-1 from the terminal management DB 5003 (see FIG. 13) (step S73-3).

Next, the terminal extraction unit 54 searches the destination list management DB 5004 using the terminal ID extracted in step S73-3 as the terminal ID of the request source terminal, thereby extracting all the terminal IDs of the corresponding destination terminals. (Step S73-4). Then, the terminal state extraction unit 55 searches the terminal management DB 5003 (see FIG. 13) based on the terminal ID of the destination terminal to extract the corresponding image communication state information (step S73-5).

Next, the message creating unit 61 refers to the image communication state information extracted in step S73-5 above, and determines whether the request source terminal and all of the destination terminals each have an image communication function ( step S73-6). If neither the request source terminal nor any of the destination terminals have the image communication function (NO), the process of step S73 ends. In this case, message transmission processing (see FIGS. 26 and 27) in step S74, which will be described later, is not performed.

On the other hand, if one of the requesting terminal and all of the destination terminals has an image communication function (YES), the message creation unit 61 generates a communication control message (described later) according to the image communication state of each terminal 10. (step S73-7).

Next, with reference to FIG. 28, step S73-7 will be described in more detail. First, the message creation unit 61 refers to the image communication state extracted in step S73-5 above, and selects two or more of the plurality of terminals (10aa, 10ba, 10db) in conversation at multiple sites. It is determined whether the terminal 10 is performing image communication (step S101). If it is determined in step S101 that a plurality of terminals 10 are in image communication (YES), the message creating unit 61 further determines whether there is a terminal 10 in which image communication is stopped among the remaining terminals 10. (step S102). If it is determined in step S103 that there is a terminal 10 that has stopped image communication (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), The IP address of the terminal 10 for which the image communication is stopped is extracted (step S103). Then, the message creation unit 61 creates a communication control message (M3) to the effect that the plurality of terminals are performing image communication (step S104). Specifically, the communication control message (M3) includes the IP address of the transmission destination extracted in step S103 and the IP address of the management system 50, which is the transmission source, in addition to the message "MESSAGE_IMAGE_ON".

On the other hand, if it is determined in step S101 that a plurality of terminals 10 are not in image communication (NO), the message creation unit 61 refers to the image communication state extracted in step S73-5, It is determined whether only a single terminal 10 is performing image communication (step S105). If it is determined at step S105 that only a single terminal 10 is performing image communication (YES), the message creating unit 61 further refers to the image communication state extracted at step S73-5. Then, it is determined whether or not all the terminals 10 other than the terminal 10 currently performing the image communication have the image communication function (step S106). If it is determined in step S106 that none of the terminals 10 other than the terminal 10 currently performing image communication has the image communication function (YES), the terminal state extraction unit 55 stores the information in the terminal management DB 5003 (FIG. 13). ), and extracts the IP address of the single terminal 10 determined to be in image communication at step S105 (step S107). Then, the message creating unit 61 creates a communication control message (M4) for forcibly stopping the image communication (step S108). Specifically, the communication control message (M4) includes the IP address of the transmission destination extracted in step S107 and the IP address of the management system 50, which is the transmission source, in addition to the "DATA_IMAGE_OFF" message.

On the other hand, if it is not determined in step S106 that none of the terminals 10 other than the terminal 10 currently performing the image communication has the image communication function (NO), that is, if the terminal 10 other than the terminal 10 currently performing the image communication If it is determined that at least one of the terminals 10 has the image communication function, the message creation unit 61 further refers to the image communication state extracted in step S73-5 to set the image communication function. It is determined whether or not there is a terminal 10 that has stopped image communication (step S109). If it is determined in step S109 that there is a terminal 10 that has stopped image communication (YES), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), It extracts the IP address of the terminal 10 that is in image communication determined to be (step S110). Then, the message creating unit 61 creates a communication control message (M1) to the effect that the other terminals are suspending image communication (step S104). Specifically, the communication control message (M1) includes the IP address of the transmission destination extracted in step S110 and the IP address of the management system 50, which is the transmission source, in addition to the message "MESSAGE_IMAGE_OFF".

Further, the terminal state extracting unit 55 refers to the terminal management DB 5003 (see FIG. 13), and extracts the IP address of the terminal 10 for which image communication is stopped, which is determined to exist in step S109 (step S112). Then, the message creation unit 61 creates a communication control message (M2) to the effect that another single terminal is in image communication (step S113). Specifically, the communication control message (M2) includes the IP address of the transmission destination extracted in step S112 and the IP address of the management system 50, which is the transmission source, in addition to the message "MESSAGE_IMAGE_ON".

On the other hand, (1) if it is not determined in step S102 that there is a terminal 10 that has stopped image communication among the remaining terminals 10 (NO), (2) only a single terminal 10 is not in image communication (NO), that is, there is no terminal 10 in image communication, or (3) it is determined in step S109 that there is no terminal 10 in image communication stop. If yes (NO), the terminal state extraction unit 55 refers to the terminal management DB 5003 (see FIG. 13), and extracts the IP address of the terminal 10 determined to have the image communication function in step S73-6. Extract (step S114). Then, the message creation unit 61 creates a display control message (M5) for deleting a predetermined display message displayed on each display 120 of the plurality of terminals 10 (step S115). Specifically, the display control message (M5) includes the IP address of the transmission destination extracted in step S114 and the IP address of the management system 50, which is the transmission source, in addition to the message "MESSAGE_IMAGE_ON".

As described above, the communication control message (M1, M2, M3, or M4) or the display control message (M5) is created by the message creating section 61. FIG.

Next, the transmitting/receiving unit 51 of the management system 50 transmits the image communication state of each of these terminals and the image of the terminal of the conversation partner to the three terminals (10aa, 10ba, 10db) at the multipoints via the communication network 2. A communication control message (M1, M2, M3, or M4) or a display control message (M5) created according to the communication state is transmitted (step S74). 26, in step S74, transmission from the management system 50 to the terminal ba is step S74-1, transmission from the management system 50 to the terminal db is step S74-2, and transmission from the management system 50 to the terminal aa is step S74-1. is represented by step S74-3.

Specifically, when step S103 is executed, the transmission/reception unit 51 of the management system 50 sends the communication control message (M3) created in step S104 to the IP address extracted in step S103. ). Further, when step S107 is executed, the transmission/reception unit 51 transmits the communication control message (M4) created in step S108 to the IP address extracted in step S107. Further, when step S110 is executed, the transmitting/receiving section 51 transmits the communication control message (M1) created in step S111 to the extracted IP address. Further, when step S112 is executed, the transmission/reception unit 51 transmits the communication control message (M2) created in step S113 to the IP address extracted in step S112. Further, when step S114 is executed, the transmission/reception unit 51 transmits the communication control message (M5) created in step S115 to the IP address extracted in step S114.

As a result, each terminal (10aa, 10ba, 10db) performs processing based on the contents of the communication control message (M1, M2, M3, or M4) or the display control message (M5) (steps S75-1 to 3). .

Next, the processing performed in steps S75-1 to S75-3 will be described with reference to FIGS. 7 and 29 to 32. FIG. 29 to 32 are diagrams showing examples of teleconference screens displayed on the display 120 of the terminal 10 side. Of these, the teleconference screen P1 displayed on the display 120 includes a main screen P2, a sub-screen P3, and a sub-screen P4. Two sub-screens below P4 and three more sub-screens to the left of the bottom sub-screen are included. At the bottom of the teleconference screen P1, there is a message display area P5 for displaying a display message. ' button P6 and whether the camera 112 of the terminal is turned on and image communication is being performed from the terminal side, or whether the power of the camera 112 of the terminal is turned off and image communication is not being performed from the terminal side. is displayed. When an "x" is displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is turned on, indicating that the user can turn off the power. On the other hand, when "x" is not displayed on the camera icon P7, it means that the power of the camera 112 of the own terminal is turned off, and it is displayed that the user can turn on the power.

First, in step S104, when the communication control message (M3) to the effect that other terminals are suspending image communication, the display control unit 17 of the terminal 10, as shown in FIG. causes the message display area P5 of the display 120 to display a display message such as "Images from other bases are being received. The camera of the terminal itself is off." The main screen P2 displays the main image of the other party, and the sub-screen P3 originally displays the image of the own terminal, but it is not displayed here. In P4, a sub image of the other party is displayed.

As a result, the user can see the image P1 for teleconference shown in FIG. It can be recognized that it is in a state where If the user wishes to transmit image data from his/her own terminal, he/she can start transmitting the image data by pressing the icon P7. When the camera icon P7 is pressed, an "x" is displayed on the camera icon P7.

Further, in step S108, when the communication control message (M4) for forcibly stopping the image communication is created, according to the command from the CPU 101, the imaging unit 14 of the terminal 10 stops imaging, and the transmitting/receiving unit 11 stops transmission of image data (stops image communication). Further, as shown in FIG. 30, the display control unit 17 hides all the screens P2, P3, P4, etc. of the teleconference screen P1, and deletes the "x" from the camera icon P7. . Note that the imaging unit 14 includes not only the camera 112 shown in FIG. 5, but also an external camera connected to an external device connection I/F.

As a result, by viewing the teleconference image P1 shown in FIG. 30, the user is in a state in which image data is not being transmitted from the own terminal, and the camera 112 of the own terminal has stopped imaging. It can be recognized that the terminal is in a state in which no image data has been sent from the other terminal.

Furthermore, in step S111 above, when the communication control message (M1) indicating that all other terminals are suspending image communication is created, the display control unit 17 of the terminal 10 displays a message as shown in FIG. Then, in the message display area P5 of the display 120, a display message is displayed, for example, "Images from other bases have not been received. The camera of the terminal is on." In FIG. 31, nothing is displayed on the main screen P2 and the sub-screen P4, and an image of the terminal itself is displayed on the sub-screen P3.

As a result, by viewing the teleconference screen P1 shown in FIG. 31, the user can see that only the own terminal is transmitting image data, and that image data is being transmitted from all of the other terminals. It can be recognized that it is in a state where it has not been received.

Furthermore, in the above step S113, when the communication control message (M2) indicating that another single terminal is performing image communication is created, the display control unit 17 of the terminal 10 is shown in FIG. In the message display area P5, for example, a display message is displayed that reads, "An image is being received from another location. The camera of the terminal itself is off." In FIG. 32, the main screen P2 displays one image of the other party, and the sub screens P3 and P4 do not display anything.

As a result, by viewing the teleconference screen P1 shown in FIG. 32, the user can see that the user is in a state in which image data is not being transmitted from his/her own terminal, and that one of the terminals of the other party is It can be recognized that the image data is being sent only from the location.

31 is displayed on one terminal 10 side and the image of FIG. 32 is displayed on the other terminal 10, each user should match the state of image communication on both sides. , you can choose to perform image communication with each other or not to perform image communication with each other.

Further, when the display control message (M5) to delete the display message is created in step S115, the display control unit 17 of the terminal 10 deletes the display message displayed in the message display area P5.

Note that the terminal 10 having the image communication function can perform image communication not only by the processing in step S72 of FIG. 26 but also by pressing the camera icon P7 shown in FIG. Status messages can be sent. Specifically, when the user selects the camera icon P7 in which "x" is displayed as shown in FIG. Creates an image communication status message to the effect that it is in progress. Also, when the user selects the camera icon P7 with no "x" displayed as shown in FIG. Create an image communication status message to that effect.

Next, a process of transmitting and receiving image data and audio data for a teleconference call between a request source terminal and a destination terminal will be described with reference to FIGS. 7 and 33. FIG. It should be noted that transmission and reception of image data and audio data are performed in one-way processing of transmitting image data and audio data from the terminal lOaa to the terminal lOdb and reverse processing of transmitting image data and audio data from the terminal lOdb to the terminal lOaa. , detection of a delay time, etc., which will be described later, are the same processing, so the one-way communication will be described and the reverse communication will be omitted.

First, the request source terminal (terminal 10aa) receives the image data of the subject captured by the imaging unit 14a and the voice input by the voice input unit 15a by the image/audio data session sed shown in FIG. Voice data is transmitted from the transmission/reception unit 11 to the relay device 30a via the communication network 2 (step S81). It should be noted that in this embodiment, high-quality image data consisting of low resolution, medium resolution, and high resolution shown in FIG. 3 and audio data are transmitted. Accordingly, in the relay device 30a, the transmitting/receiving section 31 receives the image data and the audio data of the three resolutions. Then, the data quality confirmation unit 33 searches the change quality management DB 3001 (see FIG. 10) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key, and retrieves the corresponding image to be relayed. The image quality of the image data to be relayed is confirmed by extracting the image quality of the data (step S82). In the present embodiment, the image quality of the confirmed image data is "high image quality", which is the same as the image quality of the image data received by the transmission/reception unit 31. Therefore, the relay device 30a uses the image/audio data session sed to The image data with the same image quality and the voice data with the same sound quality are transferred to the destination terminal (terminal 10db) (step S83). As a result, the destination terminal (terminal 10db) uses the transmitting/receiving unit 11 to receive high-quality image data and audio data consisting of low resolution, medium resolution, and high resolution. Then, the display control unit 17 can combine the image data of the three image qualities to display an image on the display 120, and the audio output unit 15b can output audio based on the audio data.

Next, the delay detection unit 18 of the terminal 10db detects the reception delay time of the image data received by the transmission/reception unit 11 at regular time intervals (for example, at intervals of 1 second) (step S84). In this embodiment, the case where the delay time is 200 (ms) will be described below.

The transmission/reception unit 11 of the destination terminal (terminal 10db) transmits delay information indicating a delay time of "200 (ms)" to the management system 50 via the communication network 2 by the management information session sei shown in FIG. Send (step S85). As a result, the management system 50 can grasp the delay time and the IP address "1.3.2.4" of the terminal lOdb, which is the source of the delay information.

Next, the delay time management unit 60 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key. , the corresponding terminal ID "01db" is extracted, and further, in the session management table of the session management DB 5005 (see FIG. 15), the delay time field portion of the record of the terminal ID "01db" is indicated by the delay information. The set delay time "200 (ms)" is stored and managed (step S86).

Next, the quality determination unit 58 searches the quality control DB 5007 (see FIG. 18) using the delay time “200 (ms)” as a search key, and extracts the image quality “medium image quality” of the corresponding image data. , the image quality is set to "medium image quality" (step S87).

Next, in the session management table of the session management DB (see FIG. 15), the transmission/reception unit 51 searches the relay device management DB 5001 (see FIG. 15) using the relay device ID “111a” associated with the terminal ID “01db” as a search key. 11), and extracts the IP address "1.2.1.2" of the corresponding relay device 30a (step S88). Then, the transmitting/receiving unit 51 transmits the image quality indicating the image quality "middle image quality" of the image data determined in step 87 to the relay device 30a via the communication network 2 by the management information session sei shown in FIG. Information is transmitted (step S89). This quality information includes the IP address "1.3.2.4" of the destination terminal (terminal 10db) used as a search key in step S86. As a result, in the relay device 30a, the change quality management unit 34 stores the IP address "1.3.2.4" of the destination terminal 10 (here, the terminal 10db) in the change quality management DB 3001 (see FIG. 10). , and the image quality "medium image quality" of the image data to be relayed are stored and managed in association with each other (step S90).

Next, similarly to step S81, the terminal 10aa sends high-quality image data consisting of low resolution, medium resolution, and high resolution to the relay device 30a by the image/audio data session sed, and Voice data is transmitted (step S91). As a result, in the relay device 30a, similarly to step S82, the data quality confirmation unit 33 uses the IP address "1.3.2.4" of the destination terminal (terminal 10db) as a search key to change the quality management DB 3001 ( 10) and extracting the corresponding image quality of the image data to be relayed, "medium image quality", to confirm the image quality of the image data to be relayed (step S92). In this embodiment, the image quality of the confirmed image data is “medium image quality”, which is lower than the image quality of the image data received by the transmission/reception unit 31, which is “high image quality”. is changed from "high image quality" to "middle image quality" (step S93).

Then, the transmitting/receiving unit 31 transmits the image data whose image quality has been changed to "medium image quality" and the sound quality of the voice to the terminal 10db via the communication network 2 by the image/audio data session sed. The voice data not stored is transmitted (step S94). As a result, the destination terminal (terminal 10db) uses the transmitting/receiving unit 11 to receive medium image quality image data consisting of low resolution and medium resolution and audio data. Then, the display control unit 17 can combine the image data of the two resolutions to display an image on the display 120, and the audio output unit 15b can output audio based on the audio data.

In this way, when a delay occurs in reception at the destination terminal (terminal 10db) that receives the image data, the relay device 30a changes the quality of the image so that the participants in the video conference feel uncomfortable. can be prevented.

<<main effects of the present embodiment>>
As described above, according to the present embodiment, the management system 50 performs predetermined communication control for causing predetermined terminals 10 to control image communication based on the image communication state information indicating the image communication state of each terminal 10. A message is created, and the communication control message is transmitted to the terminal 10 that controls image communication. As a result, the transmission management system can cause a predetermined transmission terminal to perform image communication according to the image communication state of the terminal 10 of the other party, thereby preventing unnecessary image communication. play. This also has the effect of avoiding wasteful power consumption, communication costs, and the like.

Even if the environment of the LAN 2, such as the IP address of the relay device 30, of the communication network 2 can be grasped, it is difficult to grasp the environment of the entire Internet 2i. , to two or more out of 30 relay devices that relay image data and audio data. Then, before actually transmitting/receiving image data and audio data between a plurality of terminals 10 next time, by transmitting/receiving pre-transmission information in place of image data and audio data, the earliest pre-transmission is performed. There is an effect that it is possible to narrow down to one relay device 30 that can relay information.

That is, by selecting the relay device 30 to which two or more IP addresses close to any of the IP addresses of the terminal 10 are assigned, two or more candidates for the relay device 30 to be finally used are left. be able to. As a result, after that, by actually transmitting/receiving the advance transmission information between the request source terminal and the destination terminal via each of the candidate relay devices 30, It is possible to narrow down to the relay device 30 that relayed the pre-transmission information that required the shortest time required for the transmission. Therefore, there is an effect that it is possible to realize transmission and reception of image data or audio data with the highest possible quality under the environment of the current communication network 2 .

Further, in the present embodiment, when narrowing down the relay devices 30, not only the relay devices 30 with IP addresses close to the IP addresses of the terminals 10 that are to be held in the video conference are preferentially selected, but also the maximum data transmission rate in each relay device 30 Two or more repeaters 30 are selected in consideration of speed as well. As a result, it is possible to narrow down the candidates for the relay device 30 that match the actual environment of the communication network 2 .

Furthermore, in the present embodiment, when narrowing down the relay devices 30, since the relay devices 30 whose operating status is online are narrowed down, it is possible to narrow down the relay device 30 candidates that are more suited to the actual environment of the communication network 2. can.

[Supplement to the embodiment]
The relay device 30, the management system 50, the program providing system 90, and the maintenance system 100 in the above embodiments may be constructed by a single computer, or each part (function or means) may be divided and arbitrarily assigned. It may be constructed by a plurality of computers. Further, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may or may not be divided into a plurality of modules and transmitted. You may make it transmit. Furthermore, when the program providing system 90 is constructed by a plurality of computers, the plurality of modules may be divided and transmitted from each computer.

In addition, a recording medium such as a CD-ROM storing the terminal program, the relay device program, or the transmission management program of each of the above embodiments, the HD 204 storing these programs, and the program provided with this HD 204 The provision system 90 is used when the terminal program, the relay device program, and the transmission management program are provided domestically or internationally to users or the like as program products.

Furthermore, in the above embodiment, the modified quality control table shown in FIG. 10 and the quality control table shown in FIG. Although managed by focusing on the resolution of the image of the data, it is not limited to this, and other examples of quality include the depth of image quality of image data, the sampling frequency of audio of audio data, and the bit length of audio of audio data. You may pay attention to etc. and may manage. Also, the audio data may be divided into data of three types of resolution (high resolution, medium resolution, and low resolution) and transmitted/received.

In addition, in FIGS. 11, 13, and 15, the reception date and time are managed, but the present invention is not limited to this, and at least the reception time of the reception date and time may be managed.

Furthermore, in the above embodiment, the IP address of the relay device is managed in FIG. 11 and the IP address of the terminal is managed in FIG. or terminal identification information for identifying the terminal 10 on the communication network 2, each FQDN (Fully Qualified Domain Name) may be managed. In this case, the IP address corresponding to the FQDN is obtained by a well-known DNS (Domain Name System) server. In addition to the "relay device specifying information for specifying the relay device 30 on the communication network 2", "relay device connection destination information indicating the connection destination to the relay device 30 on the communication network 2" or "communication It may also be expressed as "relay device destination information indicating the destination to the relay device 30 on the network 2". Similarly, not only "terminal specifying information for specifying the terminal 10 on the communication network 2", but also "terminal connection destination information indicating the connection destination to the terminal 10 on the communication network 2", or "on the communication network 2 It may be expressed as "terminal destination information indicating the destination to the terminal 10".

Further, in the above embodiment, the case of the video conference system was explained as an example of the transmission system 1, but the present invention is not limited to this, and a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone may be used. . Also, the transmission system 1 may be a car navigation system. In this case, for example, one of the terminals 10 corresponds to a car navigation device mounted on a car, and the other of the terminals 10 is a management terminal or a management server of a management center that manages car navigation, or a car navigation device mounted on another car. It corresponds to a car navigation device in which the Also, the transmission system 1 may be a communication system for mobile phones. In this case, for example, the terminal 10 corresponds to a mobile phone.

Also, in the above embodiment, image data and audio data have been described as examples of content data, but the content data is not limited to this, and may be touch data. In this case, the sensation of the user touching one terminal is transmitted to the other terminal. Further, the content data may be smell data. In this case, the smell (smell) on one terminal side is transmitted to the other terminal side. Also, the content data may be at least one of image data, audio data, tactile data, and olfactory data.

Further, in the above embodiment, the case of holding a teleconference using the transmission system 1 has been described, but the present invention is not limited to this. May be used for presentation.

1 transmission system 10 transmission terminal 11 transmission/reception unit (an example of receiving means, an example of communication stopping means)
14 imaging unit (an example of imaging means)
16 final selection unit 16a measurement unit 16b calculation unit 16c final selection unit 17 display control unit (an example of display control means)
18 delay detection unit 20 message creation unit (an example of creation means)
30 relay device 31 transmission/reception unit 32 state detection unit 33 data quality confirmation unit 34 change quality control unit 35 data quality change unit 50 transmission management system 51 transmission/reception unit (an example of transmission means)
52 terminal authentication unit 53 state management unit 54 terminal extraction unit 55 terminal state extraction unit 56 primary screening unit 56a selection session ID generation unit 56b terminal IP address extraction unit 56c primary selection unit 56c
56d priority determination unit 57 session management unit 58 quality determination unit 60 delay time management unit 61 message creation unit (an example of creation means)
70 router 90 program providing system 100 maintenance system 101 CPU (an example of control means)
120 display (an example of display means)
1000 storage unit 3000 storage unit 3001 change quality control DB
5000 Storage unit 5001 Relay device management DB
5002 Terminal authentication management DB
5003 Terminal management DB (an example of terminal management means)
5004 Destination list management DB (an example of destination list management means)
5005 Session management DB
5006 Priority management DB
5007 Quality control database

JP 2008-131412 A

The invention according to claim 1 is a transmission system including a plurality of transmission terminals including a first transmission terminal and a second transmission terminal, and a management system, wherein the management system comprises: and image data acquired by the imaging means of the second transmission terminal when the imaging means of the second transmission terminal is turned off while the second transmission terminal is transmitting and receiving image data transmitting means for transmitting information indicating the communication state of the first transmission terminal to the first transmission terminal, the first transmission terminal transmitting image data acquired by the imaging means of the first transmission terminal and the acquired image data receiving means for receiving information indicating the communication state of the image data transmitted by the transmitting means when a first icon indicating that the communication has started is displayed on the display screen; display on the display screen a state indicating that the communication of the image data captured by the imaging means of the second transmission terminal is stopped based on the information shown, and the imaging means of the first transmission terminal is turned off display control means for displaying on a display screen a second icon indicating that communication of the image data captured by the image capturing means of the first transmission terminal is stopped when the accepts an operation on the second icon to start communication of captured image data, while changing the display from the second icon to the first icon on the display screen. It is a transmission system that

Claims (12)

A transmission management system for managing image communication in a plurality of transmission terminals capable of at least voice communication,
terminal management means for managing image communication state information indicating an image communication state of each transmission terminal for each terminal identification information for identifying each of the plurality of transmission terminals;
creating means for creating a predetermined communication control message for causing a predetermined transmission terminal to control image communication based on the image communication state information;
transmitting means for transmitting the created communication control message to a transmission terminal that controls the image communication among the plurality of transmission terminals;
A transmission management system characterized by comprising: In a first case, the image communication state information indicates that, among the plurality of transmission terminals, a single transmission terminal is performing image communication, and the other transmission terminals are suspending image communication. teeth,
The creating means creates a first communication control message indicating that the other transmission terminal is suspending image communication and a second communication control message indicating that the single transmission terminal is performing image communication,
The transmission means transmits the first communication control message to the single transmission terminal during the image communication, and transmits the second communication control message to the transmission terminal during the suspension of the image communication. The transmission management system according to claim 1, characterized by: In a second case, the image communication status information indicates that, among the plurality of transmission terminals, a plurality of transmission terminals are performing image communication, and the other transmission terminals are suspending image communication. ,
the creation means creates a third communication control message indicating that the plurality of transmission terminals are performing image communication;
2. The transmission management system according to claim 1, wherein said transmission means transmits said third communication control message to said transmission terminal for which image communication is stopped. A third aspect of the present invention, wherein the image communication state information indicates that a single transmission terminal among the plurality of transmission terminals is performing image communication, and that the other transmission terminals do not have an image communication function. in case of,
The creation means creates a fourth communication control message for forcibly stopping image communication,
2. The transmission management system according to claim 1, wherein said transmission means transmits said fourth communication control message to a single transmission terminal during said image communication. In a fourth case where the image communication state information does not correspond to any of the first to third cases, the creating means deletes a predetermined display message displayed on the plurality of transmission terminal sides. Create a display control message to the effect that
2. The transmission management system according to claim 1, wherein said transmission means transmits said display control message to said plurality of transmission terminals. A transmission system comprising the transmission management system according to claim 2 or 3 and the transmission terminal,
The transmission terminal is
receiving means for receiving a communication control message transmitted from the transmission management system;
display control means for displaying a display message based on the received communication control message on a predetermined display means;
A transmission system comprising: A transmission system comprising the transmission management system according to claim 4 and the transmission terminal,
The transmission terminal is
receiving means for receiving a communication control message transmitted from the transmission management system;
communication stop means for stopping image communication from the terminal based on the received communication control message;
A transmission system comprising: A transmission system according to claim 7, wherein
The transmission terminal is
A transmission characterized by comprising control means for stopping said imaging based on a communication control message received by said receiving means, with respect to said imaging means for imaging a subject and obtaining image data to be used in said image communication. system. A transmission system comprising the transmission management system according to claim 5 and the transmission terminal,
The transmission terminal is
receiving means for receiving a display control message transmitted from the transmission management system;
display control means for erasing the predetermined display message displayed on predetermined display means based on the received display control message;
A transmission system comprising: A program that causes the transmission management system to function as each means according to any one of claims 1 to 5. 11. A program providing system that provides the program according to claim 10 to said transmission management system via a communication network. A maintenance system for performing maintenance on the transmission management system according to any one of claims 1 to 5.