JP5821469B2 - Communication terminal and communication program - Google Patents

Description

  The present invention relates to a communication terminal that transmits image data to a plurality of other communication terminals.

  2. Description of the Related Art In recent years, a communication system for performing a video conference or the like via a communication network such as the Internet has become widespread along with a request for reducing business trip expenses and business trip time. In such a communication system, a video conference can be realized by transmitting and receiving image data and audio data among a plurality of communication terminals. In addition, with the recent enhancement of the broadband environment, it has become possible to send and receive high-quality image data and high-quality audio data, making it easier to understand the status of the other party in the video conference and enhancing the level of communication through conversation. It became possible to improve.

  However, when image data and audio data are received using a narrowband route on the communication network, there is a problem that reception of the image data and audio data is delayed. Must be used. Therefore, when the video quality may be low, a technique is known that changes the quality of the image data in accordance with the usage state of the receiving communication terminal.

  For example, in Patent Document 1, the receiving device determines the image quality of the moving image based on the display state, informs the determined image quality to the transmitting device, and the transmitting device encodes the band value encoding speed. A system for transmitting moving images is disclosed.

  However, according to the conventional technique, when a video conference is performed at three or more locations, the image quality of the transmitting device is notified from a plurality of receiving devices. In this case, if the image quality of image data to be transmitted is determined based on any one of the image quality known from a plurality of receiving-side devices, image data with suitable image quality can be sent to one receiving-side device. In other receiving devices, there is a problem that the display quality when the image data is displayed becomes inappropriate.

The invention according to claim 1 is a communication terminal that transmits image data to a plurality of other communication terminals, and shows a state in which an image based on the image data is displayed on each of the plurality of other communication terminals. Display state information receiving means for receiving display state information, display state information storage means for storing a plurality of display state information received by the display state information receiving means, and a plurality of information stored in the display state information storage means Determining means for determining the image quality of the image data based on the display state information, and setting the image quality of the image data to the image quality determined by the determining means to the plurality of other communication terminals. A display area calculated from the height, width, and display rate of the image based on the image data displayed on the plurality of other communication terminals. Certain communication terminal characterized in that it comprises a mean value.

  As described above, according to the present invention, a device that transmits image data receives display states from a plurality of devices that receive image data, and determines the image quality of image data to be transmitted based on the display states. Thus, image data with good display quality can be transmitted by any device.

1 is a schematic diagram of a communication system according to an embodiment of the present invention. It is an external view of the communication terminal which concerns on this embodiment. It is a hardware block diagram of the communication terminal which concerns on one Embodiment of this invention. It is a hardware block diagram of the communication management system, relay apparatus, or program provision server which concerns on one Embodiment of this invention. It is a functional block diagram of each terminal, apparatus, and system which comprise the communication system which concerns on one Embodiment of this invention. It is a conceptual diagram explaining the image quality of image data. It is a conceptual diagram which shows a change quality management table. It is a conceptual diagram which shows a relay apparatus management table. It is a conceptual diagram which shows a terminal authentication management table. It is a conceptual diagram which shows a terminal management table. It is a conceptual diagram which shows a destination list management table. It is a conceptual diagram which shows a session management table. It is a conceptual diagram which shows an address priority management table. It is a conceptual diagram which shows a transmission rate priority management table. It is a conceptual diagram which shows a quality management table. It is the sequence figure which showed the process which manages the status information which shows the operating state of each relay apparatus. It is the sequence diagram which showed the process of the preparation stage which starts communication between communication terminals. It is the sequence diagram which showed the process which narrows down a relay apparatus. FIG. 10 is a process flow diagram illustrating a process of narrowing down relay devices. It is the figure which showed the calculation state of the point of priority at the time of performing the narrowing-down process of a relay apparatus. It is the sequence diagram which showed the process in which a communication terminal selects a relay apparatus. It is the processing flowchart which showed the process which selects a relay apparatus with a communication terminal. It is the sequence diagram which showed the process which transmits / receives image data and audio | voice data between communication terminals. It is a conceptual diagram showing an image quality management table. It is a conceptual diagram showing a display state management table. It is a conceptual diagram showing the display of a communication terminal and the image currently displayed on the display. It is the sequence diagram which showed the process which the communication terminal of a transmission side transmits image data based on each display state of the some communication terminal of a receiving side. It is a processing flowchart for determining an image quality.

<< Overall Configuration of Embodiment >>
Hereinafter, an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a schematic diagram of a communication system 1 according to an embodiment of the present invention. First, the outline of the present embodiment will be described with reference to FIG.

  The communication system 1 shown in FIG. 1 includes a plurality of communication terminals (10aa, 10ab,..., 10db), a display (11aa, 11ab,..., 11db) for each communication terminal (10aa, 10ab,. The relay apparatus (30a, 30b, 30c, 30d) and the communication management system 50 are constructed.

  In the present embodiment, “communication terminal 10” is used to indicate an arbitrary communication terminal among the communication terminals (10aa, 10ab,..., 10db), and any of the displays (11aa, 11ab,..., 11db). "Display 11" is used to indicate the display of "5", and "relay device 30" is used to indicate any of the relay devices (30a, 30b, 30c, 30d).

  The communication terminal 10 transmits / receives image data, audio data, and the like to / from other communication terminals 10. In the present embodiment, the case where the image of the image data is a moving image will be described, but it may be a still image as well as a moving image. The image of the image data may include both a moving image and a still image. The relay device 30 relays image data and audio data between the plurality of communication terminals 10. The communication management system 50 manages the communication terminal 10 and the relay device 30 in an integrated manner.

  Also, the plurality of routers (70a, 70b,..., 70g) shown in FIG. 1 perform selection of optimal paths for image data and audio data. In the present embodiment, “router 70” is used to indicate an arbitrary router among the routers (70a, 70b,..., 70g). The program providing system 90 includes an HD (Hard Disk) (not shown) in which a communication terminal program for causing the communication terminal 10 to realize various functions or various means is stored. You can send the program. The HD of the program providing system 90 also stores a relay device program for causing the relay device 30 to realize various functions or various means, and can transmit the relay device program to the relay device 30. . Further, the HD of the program providing system 90 also stores a communication management program for causing the remote communication management system 50 to realize various functions or various means, and transmits the communication management program to the communication management system 50. be able to.

  The communication terminal 10aa, the communication terminal 10ab, the relay device 30a, and the router 70a are communicably connected via the LAN 2a. The communication terminal 10ba, the communication terminal 10bb, the relay device 30b, and the router 70b are communicably connected via the LAN 2b. The LAN 2a and the LAN 2b are communicably connected via a dedicated line 2ab including the router 70c, and are constructed in a predetermined area A. For example, the region A is Japan, the LAN 2a is constructed in a Tokyo office, and the LAN 2b is constructed in an Osaka office.

  On the other hand, the communication terminal 10ca, the communication terminal 10cb, the relay device 30c, and the router 70c are communicably connected via the LAN 2c. The communication terminal 10da, the communication terminal 10db, the relay device 30d, and the router 70d are connected to be communicable via the LAN 2d. The LAN 2c and the LAN 2d are communicably connected by a dedicated line 2cd including the router 70e, and are constructed in a predetermined area B. For example, region B is the United States, LAN 2c is built in a New York office, and LAN 2d is Washington D.C. C. Is built in the office. Area A and area B are connected to each other via routers (70c, 70e) via the Internet 2i.

  In addition, the communication management system 50 and the program providing system 90 are communicably connected to the communication terminal 10 and the relay device 30 via the Internet 2i. The communication management system 50 and the program providing system 90 may be installed in the region A or the region B, or may be installed in a region other than these.

  In the present embodiment, the communication network 2 of the present embodiment is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d.

Further, in FIG. 1, four sets of numbers shown under each communication terminal 10, each relay device 30, communication management system 50, each router 70, and program providing system 90 are IP addresses in general IPv4. Is simply shown. For example, the IP address of the communication terminal 10aa is “1.2.1.3”. Further, IPv6 may be used instead of IPv4. In order to simplify the description, description will be made using IPv4.
<< Hardware Configuration of Embodiment >>
Next, the hardware configuration of this embodiment will be described. In the present embodiment, when there is a delay in the reception of image data at the communication terminal 10 as the relay destination, the image resolution of the image data is changed by the relay device 30 and then the image is sent to the communication terminal 10 as the relay destination. A case where data is transmitted will be described.

  FIG. 2 is an external view of the communication terminal according to the present embodiment. Hereinafter, the longitudinal direction of the communication terminal 10 will be described as the X-axis direction, the direction orthogonal to the X-axis direction in the horizontal plane as the Y-axis direction, and the direction orthogonal to the X-axis direction and the Y-axis direction (vertical direction) as the Z-axis direction. .

  As illustrated in FIG. 2, the communication terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. Among them, the front side wall surface 1110 of the housing 1100 is provided with an air intake surface (not shown) formed by a plurality of air intake holes, and the rear side wall surface 1120 of the housing 1100 is formed with a plurality of exhaust holes. An exhaust surface 1121 is provided. Thus, by driving a cooling fan built in the housing 1100, outside air behind the communication terminal 10 can be taken in via an intake surface (not shown) and exhausted to the rear of the communication terminal 10 via an exhaust surface 1121. it can. A sound collecting hole 1131 is formed in the right wall surface 1130 of the housing 1100, and sounds such as voice, sound, noise can be collected by a built-in microphone 114 described later.

    An operation panel 1150 is formed on the right wall surface 1130 side of the housing 1100. The operation panel 1150 is provided with a plurality of operation buttons (108a to 108e) to be described later, a power switch 109 to be described later, and an alarm lamp 119 to be described later, and outputs sound from a built-in speaker 115 to be described later. A sound output surface 1151 formed by a plurality of sound output holes for passing through is formed. Further, a housing 1160 as a recess for housing the arm 1200 and the camera housing 1300 is formed on the left wall surface 1140 side of the housing 1100. The right side wall surface 1130 of the housing 1100 is provided with a plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device I / F 118 described later. On the other hand, the left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting a cable 120c for the display 120 to an external device I / F 118 described later.

  In the following description, “operation button 108” is used when an arbitrary operation button is indicated among the operation buttons (108a to 108e), and “an arbitrary connection port is indicated among the connection ports (1132a to 1132c)”. This will be described using the connection port 1132 ”.

  Next, the arm 1200 is attached to the housing 1100 via a torque hinge 1210, and the arm 1200 is configured to be able to rotate in the vertical direction with respect to the housing 1100 within a tilt angle θ1 of 135 degrees. ing. FIG. 2 shows a state where the tilt angle θ1 is 90 degrees.

  The camera housing 1300 is provided with a built-in camera 1021 that can capture images of users, documents, rooms, and the like. A torque hinge 1310 is formed in the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via a torque hinge 1310. The camera housing 1300 is attached to the arm 1200 via a torque hinge 1310. The pan angle θ2 of ± 180 degrees is assumed with respect to the arm 1200, with the state shown in FIG. And a tilt angle θ3 of ± 45 degrees can be rotated in the vertical and horizontal directions. The relay device 30, the communication management system 50, and the program providing system 90 have the same external appearance as a general server computer, and thus the description of the external appearance is omitted.

  FIG. 3 is a hardware configuration diagram of the communication terminal 10 according to the embodiment of the present invention. As shown in FIG. 3, the communication terminal 10 of this embodiment includes a CPU (Central Processing Unit) 101 that controls the operation of the entire communication terminal 10 and a ROM (Read Only Memory) 102 that stores a communication terminal program. , A RAM (Random Access Memory) 103 used as a work area of the CPU 101, a flash memory 104 for storing various data such as image data and audio data, and control of reading or writing of various data to the flash memory 104 according to the control of the CPU 101 An SSD (Solid State Drive) 105, a media drive 107 that controls reading or writing (storage) of data to a recording medium 106 such as a flash memory, an operation button 108 that is operated when a destination of the communication terminal 10 is selected, Switch on / off the power of the communication terminal 10 A power switch 109 for capturing data, a network I / F 111 for performing data communication using a communication network 2 to be described later, a camera 112 that captures an image of a subject according to the control of the CPU 101, and controls the driving of the camera 112. The image sensor I / F 113, the microphone 114 for inputting sound, the speaker 115 for outputting sound, the sound input / output I / F 116 for processing input / output of sound signals between the microphone 114 and the speaker 115 according to the control of the CPU 101, and the CPU 101 A display I / F 117 that transmits / receives image data to / from an external display 100 according to control, an external device I / F 118 that transmits / receives various data to / from an external device, an alarm lamp 119 that notifies abnormality of various functions of the terminal 10, and each of the above components As shown in FIG. And an address bus or a data bus line 110 such as a bus for connecting.

  The recording medium 106 is detachable from the communication terminal 10. Further, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used. Further, the camera 112 is a solid-state imaging device that converts light into electric charges and digitizes an image (video) of a subject. A CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like may be used. Further, the display 100 is configured by a liquid crystal or an organic EL that displays an image of a subject, an operation icon, and the like.

  Further, the communication terminal program may be installed in an installable or executable format, recorded on a computer-readable recording medium such as the recording medium 106, and distributed.

  FIG. 4 is a hardware configuration diagram of the communication management system according to the embodiment of the present invention. The communication management system 50 includes a CPU 201 that controls the overall operation of the communication management system 50, a ROM 202 that stores a communication management program, a RAM 203 that is used as a work area for the CPU 201, an HD (Hard Disk) 204 that stores various data, and a CPU 201. A HDD (Hard Disk Drive) 205 that controls the reading or writing of various data with respect to the HD 204 in accordance with the above control, a media drive 207 that controls the reading or writing (storage) of data with respect to a recording medium 206 such as a flash memory, a cursor, a menu, A display 208 that displays various information such as windows, characters, or images, a network I / F 209 for performing data communication using a communication network 2 described later, and a plurality of characters, numbers, and various instructions for inputting various instructions. Ki Data for a keyboard 211 with a keyboard, a mouse 212 for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like, and a CD-ROM (Compact Disc Read Only Memory) 213 as an example of a removable recording medium 4 includes a CD-ROM drive 214 for controlling reading or writing of data, and a bus line 210 such as an address bus or a data bus for electrically connecting the above components as shown in FIG. .

  The communication management program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213. .

  Further, since the relay device 30 has the same hardware configuration as that of the communication management system 50, the description thereof is omitted. However, the ROM 202 stores a relay device program for controlling the relay device 30. Also in this case, the relay device program is a file in an installable or executable format, and is recorded on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 for distribution. Good.

  Furthermore, since the program providing system 90 has the same hardware configuration as the communication management system 50, the description thereof is omitted. However, a program providing program for controlling the program providing system 90 is recorded in the ROM 202. Also in this case, the program providing program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Good.

As another example of the detachable recording medium, the recording medium is provided by being recorded on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray Disc. May be.
<< Functional Configuration of Embodiment >>
Next, the functional configuration of this embodiment will be described. FIG. 5 is a functional block diagram of each terminal, device, and system constituting the communication system 1 of the present embodiment. In FIG. 5, the communication terminal 10, the relay device 30, and the communication management system 50 are connected so that data communication can be performed via the communication network 2. Further, the program providing system 90 shown in FIG. 1 is omitted in FIG. 5 because it is not directly related to the video conference communication.

<Functional configuration of communication terminal>
The communication terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14a, an image display control unit 14b, an audio input unit 15a, an audio output unit 15b, a final narrowing unit 16, and a delay detection unit 17. And a storage / reading processing unit 19, an image layout processing unit 20, a display state acquisition unit 21, a display area ratio calculation unit 22, an image quality determination unit 23, and an image quality change unit 24. Each of these units is a function or means realized by any one of the components shown in FIG. 3 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the communication terminal 10 has a storage unit 1000 constructed by the SSD 105 or the RAM 103 shown in FIG.

(Image quality management table)
In the storage unit 1000, an image quality management DB (Data Base) 1001 configured by an image quality management table as shown in FIG. The image quality management table is managed by associating the image quality with the average display area ratio that is the average value of the display area ratios of the communication terminals 10 participating in the conference. Since the larger the image display, the higher the image quality is required. Therefore, a high image quality is associated with a high average display area ratio and a low image quality is associated with a low average display area ratio. The display area ratio is a ratio of the area of the image displayed by the image display control unit 14b to the entire area of the display 100 of the communication terminal 10.

For example, in the image quality management table shown in FIG. 24, the image quality of the image data displayed on the communication terminal when the display area ratio of the communication terminal 10 is less than 30% is stored in association with “low image quality”. . Further, when the display area ratio of the communication terminal 10 is 30% or more and less than 70%, “medium image quality” is associated as the image quality of the image data displayed on the communication terminal, and the display area ratio of the communication terminal 10 is 70% or more. For example, “high image quality” is stored in association with the image quality of image data to be displayed on the communication terminal.
Here, the low image quality indicates the quality of an image composed of only a low-resolution image that is the base image whose image quality is shown in FIG. The medium image quality refers to the quality of an image composed of low-resolution image data and medium-resolution image data as the base image whose image quality is shown in FIGS. 6 (a) and 6 (b). . Furthermore, high image quality refers to low-resolution image data, medium-resolution image data, and high image quality, which are the base images shown in FIGS. 6 (a), 6 (b), and (c). Indicates the quality of an image composed of resolution image data.

(Display status management table)
In the storage unit 1000, a display state management DB 1002 configured by a display state management table as shown in FIG. In the display state management table, the terminal ID of the communication terminal 10 participating in the conference and the display area ratio of the display data displayed on the display 100 by the communication terminal 10 are associated and managed. For example, the image display control unit 14b with the terminal ID “10bb” causes the display 100bb to display an image with a display area ratio of 80%, and the image display control unit 14b with the terminal ID “10cb” has a display area ratio of 65% on the display 100cb. It is managed that the image is displayed with.

<Each functional unit of communication terminal>
Next, each part of the communication terminal will be described in detail. The transmission / reception unit 11 of the communication terminal 10 is realized by the network I / F 111 illustrated in FIG. 3, and transmits / receives various data (information) to / from other terminals, devices, or systems via the communication network 2. The operation input receiving unit 12 is realized by the operation button 107 and the power switch 108 shown in FIG. 3 and receives various inputs by the user. For example, when the user turns on the power switch 109 shown in FIG. 3, the operation input receiving unit 12 shown in FIG. 5 receives the power ON and turns on the power. The login request unit 13 is realized by a command from the CPU 101 illustrated in FIG. 3, and requests login from the transmission / reception unit 11 to the communication management system 50 via the communication network 2 when the power ON is received. The login request information indicating the fact and the current IP address of the communication terminal 10ab are automatically transmitted.

  The image capturing unit 14a is realized by the CCD 112 and the image sensor I / F 113 shown in FIG. 3, images a subject, and outputs image data obtained by the image capturing. The image display control unit 14 b is realized by the display I / F 117 illustrated in FIG. 3, and performs control for displaying an image based on the image data on the display 100. The voice input unit 15a is realized by the microphone 114 and the voice input / output I / F 116 shown in FIG. 3. The voice input unit 15a inputs the voice of the user and converts the voice into a voice signal. Output audio data. The audio output unit 15b is realized by the speaker 115 and the audio input / output I / F 116 shown in FIG. 3, and converts an audio signal related to the audio data into audio and outputs the audio.

  The final narrowing unit 16 performs a final narrowing process that finally narrows down from a plurality of relay devices 30 to one relay device 30, so that a measurement unit 16 a and a calculation unit 16 b are executed according to a command from the CPU 101 illustrated in FIG. 3. , And the final selection unit 16c. Among these, the measurement unit 16 a measures the reception date and time when the pre-transmission information is received by the transmission / reception unit 11 for each pre-transmission information described later received by the transmission / reception unit 11. For each pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b determines the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The time required from transmission to reception is calculated. The final selection unit 16c finally selects one relay device by selecting the relay device 30 to which the pre-transmission information that required the shortest required time is relayed among the required times calculated by the calculation unit 16b. .

  The delay detection unit 17 is realized by a command from the CPU 101 shown in FIG. 3, and detects a delay time (ms) of image data or audio data sent from another communication terminal 10 via the relay device 30. To do. The storage / read processing unit 19 is executed by the SSD 105 shown in FIG. 3, and stores various data in the storage unit 1000 and reads various data stored in the storage unit 1000. The storage unit 1000 stores a terminal ID (Identification) for identifying the communication terminal 10, a password, image data, audio data, and the like.

  The image layout processing unit 20 generates a screen layout to be displayed on the display 100. The display state detection unit 21 detects information related to the display state of the image in the layout generated by the image layout processing unit 20. The display area ratio calculation unit 22 calculates the ratio of the area occupied by the image displayed on the display 100 to the area of the entire screen based on the display state as the display area ratio.

  In order to determine the image quality of image data to be transmitted to the relay device 30, the image quality determination unit 23 determines a change detection unit 23a, an average area ratio calculation unit 23b, and a determination according to a command from the CPU 101 illustrated in FIG. The unit 23c is realized. Among these, the change detection unit 23a detects that the display area ratio of another communication terminal managed by the display state management DB 1002 has changed. The average display area ratio calculation unit 23b calculates an average display area ratio that is an average value of the display area ratios managed by the display state management DB 1002. The determination unit 23c determines the image quality based on the average display area ratio calculated by the average display area ratio calculation unit 23b.

  The image quality changing unit 24 sets the image quality of the image data to the image quality determined by the determining unit 23c.

  It should be noted that the terminal ID of this embodiment and the relay device ID described later are identification information such as language, characters, symbols, or various signs used to uniquely identify the communication terminal 10 and the relay device 30, respectively. Show. Further, the terminal ID and the relay device ID may be identification information in which at least two of the language, characters, symbols, and various signs are combined. In the following description, the communication terminal 10 as a request source requesting the start of a video conference is described as “request source terminal 10A”, and the communication terminal 10 as a request destination is described as “destination terminal 10B”.

(Functional configuration of relay device)
Next, functions or means of the relay device 30 will be described. The relay device 30 includes a transmission / reception unit 31, a state detection unit 32, a data quality confirmation unit 33, a change quality management unit 34, a data quality change unit 35, and a storage / read processing unit 39. Each of these units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the relay device 30 has a storage unit 3000 constructed by the HD 204 shown in FIG.

(Change quality control table)
In the storage unit 3000, a change quality management DB 3001 configured by a change quality management table as shown in FIG. In the change quality management table, the IP address of the communication terminal 30 as the relay destination of the image data and the image quality of the image data relayed by the relay device 30 are associated with the relay destination and managed.

  Here, the resolution of the image data handled in the present embodiment will be described. As shown in FIG. 6 (a), the horizontal image is composed of 160 pixels and the vertical image is 120 pixels, and a low-resolution image serving as a base image and a horizontal image as illustrated in FIG. 6 (b). There are a medium resolution image composed of 320 pixels and 240 pixels vertically, and a high resolution image composed of 640 pixels horizontally and 480 pixels vertically as shown in FIG. 6C. Among these, when passing through a narrow band route, low-quality image data consisting only of low-resolution image data serving as a base image is relayed. When the band is relatively wide, low-resolution image data serving as a base image and medium-quality image data composed of medium-resolution image data are relayed. When the bandwidth is very wide, high-resolution image data composed of low-resolution image data serving as base image quality, intermediate-resolution image data, and high-resolution image data is relayed. For example, in the modified quality management table shown in FIG. 7, when the relay device 30 relays image data to the destination terminal 10db having the IP address “1.3.2.4”, the relay is performed. The image quality of the image data (image quality) is “high image quality”.

<Each functional unit of the relay device>
Next, each functional configuration of the relay device 30 will be described in detail. In the following description, each component of the relay device 30 will be described in relation to main components for realizing each component of the relay device 30 among the components illustrated in FIG. 3.

  The transmission / reception unit 31 of the relay device 30 shown in FIG. 5 is realized by the network I / F 209 shown in FIG. 4, and communicates with other terminals, devices, or systems via the communication network 2 and various data (information ). The state detection unit 32 is realized by a command from the CPU 201 illustrated in FIG. 4, and detects the operating state of the relay device 30 having the state detection unit 32. The operating state includes “ON line”, “OFF line”, and “failing” state.

  The data quality confirmation unit 33 is realized by a command from the CPU 201 shown in FIG. 4, and a change quality management DB 3001 in which a change quality table (see FIG. 7) is configured using the IP address of the destination terminal 10B as a search key. And the image quality of the corresponding relayed image data is extracted to check the image quality of the relayed image data. The change quality management unit 34 is realized by a command from the CPU 201 shown in FIG. 4 and changes the contents of the change quality management DB 3001 based on quality information to be described later sent from the communication management system 50. For example, during a video conference by transmitting / receiving high-quality image data between the request source terminal 10aa having the terminal ID “01aa” and the destination terminal 10db having the terminal ID “01db”. When a delay in receiving image data occurs at the destination terminal 10db due to the request source terminal bb and the destination terminal ca performing another video conference starting the video conference via the communication network 2, etc., the relay device No. 30 needs to lower the image quality of image data relayed up to now from high image quality to medium image quality. In such a case, the content of the change quality management DB 3001 is changed so as to lower the image quality of the image data relayed by the relay device 30 from the high image quality to the medium image quality based on the quality information indicating the medium image quality.

  The data quality changing unit 35 is realized by an instruction from the CPU 201 shown in FIG. 4, and the image quality of the image data sent from the transmission source terminal 10 is based on the contents of the changed change quality management DB 3001. To change. The storage / reading processing unit 39 is realized by the HDD 205 shown in FIG. 4, and stores various data in the storage unit 3000 and reads various data stored in the storage unit 3000.

<Functional configuration of communication management system>
Next, functions or means of the communication management system 50 will be described. The communication management system 50 includes a transmission / reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state acquisition unit 55, a primary narrowing unit 56, a session management unit 57, a quality determination unit 58, a storage / readout A processing unit 59 and a delay time management unit 60 are included. Each of these units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the communication management system 50 has a storage unit 5000 constructed by the HD 204 shown in FIG.

(Relay device management table)
In the storage unit 5000, a relay device management DB 5001 configured by a relay device management table as shown in FIG. 8 is constructed. In this relay device management table, for each relay device ID of each relay device 30, the operation status of each relay device 30, the date and time when the communication management system 50 received the status information indicating the operation status, the IP of the relay device 30 The address and the maximum data transmission rate (Mbps) in the relay device 30 are managed in association with each other. For example, in the relay device management table shown in FIG. 8, the relay device 30a with the relay device ID “111a” has an operation state of “ON line” and the date and time when the communication management system 50 received the status information. “November 10, 2009, 13:00” indicates that the IP address of the relay device 30a is “1.2.1.2” and the maximum data transmission speed in the relay device 30a is 100 Mbps. Has been.

(Terminal authentication management table)
Furthermore, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table as shown in FIG. In this terminal authentication management table, each password is associated with each terminal ID of all communication terminals 10 managed by the communication management system 50 and managed. For example, the terminal authentication management table shown in FIG. 9 indicates that the terminal ID of the communication terminal 10aa is “01aa” and the password is “aaaa”.

(Terminal management table)
Further, in the storage unit 5000, a terminal management DB 5003 configured by a terminal management table as shown in FIG. 10 is constructed. In this terminal management table, for each terminal ID of each communication terminal 10, the operation status of each communication terminal 10, the reception date and time when login request information described later is received by the communication management system 50, and the IP address of the communication terminal 10 are associated. Managed. For example, in the terminal management table shown in FIG. 10, the communication terminal 10aa with the terminal ID “01aa” has an operation state of “ON line” and the date and time when the login request information is received by the communication management system 50 is “ "November 10, 2009, 13:40" indicates that the IP address of the communication terminal 10aa is "1.2.1.3".

(Destination list management table)
Further, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table as shown in FIG. In this destination list management table, all terminal IDs of destination terminals 10B registered as candidates for destination terminal 10B are managed in association with terminal IDs of request source terminal 10A requesting the start of a video conference. For example, in the destination list management table shown in FIG. 11, a candidate for the destination terminal 10B that can request the start of the video conference from the request source terminal 10aa having the terminal ID “01aa” has the terminal ID “01ab”. ", The communication terminal 10ab having the terminal ID" 01ba "and the communication terminal 10db having the terminal ID" 01db ". The destination terminal 10B candidates are updated by addition or deletion in response to a request for addition or deletion from the request source terminal 10A to the communication management system 50.

(Session management table)
Further, in this storage unit 5000, a session management DB 5005 configured by a session management table as shown in FIG. 12 is constructed. In this session management table, for each selection session ID used to execute a session for selecting the relay device 30, the relay device ID of the relay device 30 used for relaying image data and audio data, the request source terminal 10A Is sent from the destination terminal 10B, the terminal ID of the destination terminal 10B, the terminal ID of the destination terminal 10B, the reception delay time (ms) when the destination terminal 10B receives image data, and the delay information indicated by this delay time. The reception date and time received and received by the communication management system 50 is associated and managed. For example, in the session management table shown in FIG. 12, the relay device 30a (relay device ID “111a”) selected in the session executed using the selection session ID “se1” has the terminal ID “01aa”. "And the destination terminal 10db with the terminal ID" 01db "are relayed image data and audio data, and the destination terminal 10db" November 10, 2009 14:00 It is shown that the delay time of the image data at the time is 200 (ms). When a video conference is performed between the two communication terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal 10A instead of the destination terminal 10B. . However, when a video conference is performed between three or more communication terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the communication terminal 10 on the image data and audio data receiving side. To do.

(Address priority management table)
Further, in the storage unit 5000, a priority management DB 5006 configured by an address priority management table as shown in FIG. 13 is constructed. In this address priority management table, in accordance with the difference in the four dot address (Dot Address) portions of the general IPv4 IP addresses, the address priority points are associated and managed so as to increase. The For example, in the address priority management table shown in FIG. 13, when the three values are the same IP address from the top to the bottom of the dot address, the address priority point is “5”. When the two values are the same IP address from the top to the bottom of the dot address, the address priority point is “3”. In this case, whether the value of the lowest dot address is the same or not is not related to the priority. In the case of IP addresses having the same highest value of dot addresses and different second values from the top, the address priority point is “1”. In this case, whether or not the values of the third and lowest dot addresses from the top are the same has no relation to the priority. In the case of IP addresses having different uppermost values of dot addresses, the address priority point is “0”. In this case, whether or not the values of the second, third, and lowest dot addresses from the top are the same has no relation to the priority.

(Transmission speed priority management table)
The priority management DB 5006 built in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. In this transmission rate priority management table, the transmission rate priority points are associated and managed in accordance with the value of the maximum data transmission rate (Mbps) in the relay device 30 so as to increase. For example, in the transmission rate priority management table shown in FIG. 14, when the maximum data transmission rate in the relay device 30 is 1000 Mbps or more, the transmission rate priority point is “5”. When the maximum data transmission rate in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission rate priority point is “3”. When the maximum data transmission rate in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission rate priority point is “1”. When the maximum data transmission rate in the relay device 30 is less than 10 Mbps, the transmission rate priority point is “0”.

(Quality control table)
Furthermore, in the storage unit 5000, a quality management DB 5007 configured by a quality management table as shown in FIG. In this quality management table, the image quality (image quality) of the image data relayed by the relay device 30 is associated and managed according to the delay time (ms) of the image data at the request source terminal 10A or the destination terminal 10B.

(Each functional part of the communication management system)
Next, each functional unit of the communication management system 50 will be described in detail. In the following, in describing each part of the communication management system 50, a relationship with main constituent elements for realizing each part of the communication management system 50 among the constituent elements shown in FIG. 4 will also be described. .

  The transmission / reception unit 51 is executed by the network I / F 209 illustrated in FIG. 3, and transmits / receives various data (information) to / from other terminals, devices, or systems via the communication network 2. The terminal authentication unit 52 searches the terminal authentication management DB 5002 in the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and is identical to the terminal authentication management DB 5002. Terminal authentication is performed by determining whether or not the terminal ID and password are managed. The state management unit 53 stores the terminal ID and request of the request source terminal 10A in the terminal management DB 5003 in which the terminal management table (see FIG. 10) is configured in order to manage the operating state of the request source terminal 10A that has requested login. The operating state of the original terminal 10A, the reception date and time when the login request information is received by the communication management system 50, and the IP address of the requesting terminal 10 are stored and managed in association with each other.

  The terminal extraction unit 54 searches the destination list management DB 5004 in which the destination list management table (see FIG. 11) is configured using the terminal ID of the request source terminal 10A that requested the login as a key, and communicates with the request source terminal 10A. The terminal ID is extracted by reading the candidate terminal ID of the destination terminal 10B capable of receiving the information. Further, the terminal extraction unit 54 searches the destination list management DB 5004 in which the destination list management table (see FIG. 11) is configured using the terminal ID of the request source terminal 10A that has requested the login as a key, and the request source terminal 10A. The terminal IDs of other request source terminals 10A in which the terminal IDs of the request terminals 10B are registered as candidates for the destination terminal 10B are also extracted.

  The terminal state acquisition unit 55 searches the terminal management DB 5003 in which the terminal management table (see FIG. 10) is configured using the candidate terminal ID of the destination terminal 10B extracted by the terminal extraction unit 54 as a search key, and The operating state is read for each terminal ID extracted by the terminal extracting unit 54. Thereby, the terminal state acquisition unit 55 can acquire the operation state of the candidate of the destination terminal 10B that can communicate with the request source terminal 10A that has requested the login. Further, the terminal state acquisition unit 55 searches the terminal state management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and also acquires the operating state of the request source terminal 10A that has requested login.

  The primary narrowing-down unit 56 performs a primary narrowing-down process before the final narrowing-down process in order to support a final narrowing-down process that finally narrows down from a plurality of relay apparatuses 30 to one relay apparatus 30. Therefore, the selection session ID generation unit 56a A terminal IP address extraction unit 56b, a primary selection unit 56c, and a priority determination unit 56d. Among these, the selection session ID generation unit 56 a generates a selection session ID used for executing a session for selecting the relay device 30. Based on the terminal ID of the request source terminal 10A and the terminal ID of the destination terminal 10B included in the start request information sent from the request source terminal 10A, the terminal IP address extracting unit 56b The IP address of each corresponding communication terminal 10 is extracted by searching the terminal management DB 5003 in which the reference is configured. The primary selection unit 56c selects the relay device 30 whose operation state is “ON line” among the relay devices 30 managed by the relay device management DB 5001 in which the relay device management table (see FIG. 8) is configured. The relay device 30 is selected by selecting the relay device ID.

  Further, the primary selection unit 56c is configured by a relay device management table (see FIG. 8) based on the IP address of the request source terminal 10A and the IP address of the destination terminal 10B extracted by the terminal IP address extraction unit 56b. By searching the relay device management DB 5001 that has been selected, the dot addresses of the IP addresses of the request source terminal 10A and the destination terminal 10B are the same for each dot address of the IP address of the selected relay device 30. Investigate whether or not. Further, the primary selection unit 56c, for each relay device, the upper two relays having the higher points among the integrated points obtained by integrating the higher points with respect to the communication terminal 10 at the address priority points and the transmission speed priority points. By selecting the device 30, the relay device 30 is further selected.

  In the present embodiment, the top two relay devices 30 with high points are selected. However, the present invention is not limited to this. If one relay device 30 can be narrowed down as many as possible, the top points with high points are selected. Three or more relay devices 30 may be selected.

  The priority determination unit 56d refers to the priority management DB 5006 in which the priority management table (see FIG. 13) is configured, and points the address priority for each relay device 30 investigated by the primary selection unit 56c. To decide. Further, the priority determination unit 56d determines the priority management table (based on the maximum data transmission rate of each relay device 30 managed by the relay device management DB 5001 in which the relay device management table (see FIG. 8) is configured. By searching the priority management DB 5006 configured (see FIG. 13), a transmission rate priority point is determined for each relay device 30 narrowed down by the first narrowing-down process by the primary selection unit 56c.

  The session management unit 57 adds the session ID for selection generated by the selection session ID generation unit 56a and the terminal ID of the request source terminal to the session management DB 5005 in which the session management table (see FIG. 12) of the storage unit 5000 is configured. , And the terminal ID of the destination terminal are stored and managed in association with each other. In addition, the session management unit 57 finally adds one to the session management DB 5005 in which the session management table (see FIG. 11) is configured for each selection session ID by the final selection unit 16c of the communication terminal 10. The relay device ID of the selected relay device 30 is stored and managed.

The quality determining unit 58 searches the quality management DB 5007 (see FIG. 15) using the delay time as a search key and extracts the image quality of the corresponding image data, thereby determining the image quality of the image data to be relayed to the relay device 30. To do. The storage / read processing unit 59 is executed by the HDD 205 shown in FIG. 4, and stores various data in the storage unit 5000 or reads various data stored in the storage unit 5000. The delay time management unit 60 extracts the corresponding terminal ID by searching the terminal management DB 5003 (see FIG. 10) using the IP address of the destination terminal 10B as a search key, and further, the session management table of the session management DB 5005 In (see FIG. 12), the delay time indicated by the delay information is stored and managed in the field of delay time in the record including the extracted terminal ID.
<< Processing and Operation of Embodiment >>
The above is the description of the configuration and function (or means) of the communication system 1 according to the present embodiment, and subsequently, the processing method in the communication system 1 according to the present embodiment is described with reference to FIGS. 16 to 22. . FIG. 16 is a sequence diagram showing processing for managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the communication management system 1. FIG. 17 is a sequence diagram illustrating processing in a preparation stage for starting communication between a plurality of communication terminals 10. FIG. 18 is a sequence diagram illustrating processing for narrowing down the relay devices 30. FIG. 19 is a process flow diagram illustrating a process of narrowing down the relay devices 30. FIG. 20 is a diagram illustrating a point calculation state when the relay device 30 performs the narrowing-down process. FIG. 21 is a sequence diagram illustrating processing in which the communication terminal 10 selects the relay device 30. FIG. 22 is a process flow diagram illustrating a process of selecting the relay device 30 at the communication terminal. FIG. 23 is a sequence diagram illustrating processing for transmitting and receiving image data and audio data between communication terminals.

  First, a process for managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the communication management system 50 will be described with reference to FIG. First, in each relay device 30, the state detection unit 32 shown in FIG. 5 periodically detects the operating state of the relay device 30 as its own device (S1-1 to S1-4). In order to manage the operation state of each relay device 30 in real time on the communication management system 50 side, the transmission / reception unit 31 of each relay device 30 periodically transmits each status information to the communication management system 50 via the communication network 2. Transmit (steps S2-1 to S2-4). Each state information includes the relay device ID for each relay device 30 and the operating state detected by the state detection unit 32 of the relay device 30 related to each relay device ID. In the present embodiment, the relay devices (30a, 30b, 30d) operate normally and are “ON line”, while the relay device 30c is in operation, but the relay operation of the relay device 30c. The case where some trouble has occurred in the program for executing the “OFF line” is shown.

  Next, in the communication management system 50, the transmission / reception unit 51 receives each status information sent from each relay device 30, and the relay device status management DB 5001 (see FIG. 8), state information is stored and managed for each relay device ID (steps S3-1 to S3-4). As a result, the operating status of “ON line”, “OFF line”, or “failed” is stored and managed for each relay device ID in the relay device management table as shown in FIG. The At this time, the reception date and time when the status information is received by the communication management system 50 is also stored and managed for each relay device ID. If the status information is not sent from the relay device 30, the field portion of the operating state and the field portion of the reception date and time in each record of the relay device management table shown in FIG. The operation status and the reception date and time at the previous reception are shown.

  Next, processing in a preparatory stage before starting communication between the communication terminal 10aa and the communication terminal 10db will be described using FIG. First, when the user turns on the power switch 109 shown in FIG. 3, the operation input receiving unit 12 shown in FIG. 5 receives the power ON and turns on the power (step S21). The login request unit 13 automatically transmits login request information indicating a login request from the transmission / reception unit 11 to the communication management system 50 via the communication network 2 in response to the reception of the power ON (step S22). . This login request information includes a terminal ID and a password for identifying the communication terminal 10aa that is the own device as the request source. These terminal ID and password are data read from the storage unit 1000 via the storage / read processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the communication terminal 10aa to the communication management system 50, the communication management system 50 that is the reception side can grasp the IP address of the communication terminal 10ab that is the transmission side.

  Next, the terminal authentication unit 52 of the communication management system 50 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as a search key, and the terminal authentication management DB 5002 of the storage unit 5000 (FIG. 9). Terminal authentication is performed by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). Since the same terminal ID and password are managed by the terminal authentication unit 52, when it is determined that the login request is from the communication terminal 10 having a valid use authority, the state management unit 53 In the management DB 5003 (see FIG. 10), the terminal ID of the communication terminal 10aa, the operating state, the reception date and time when the login request information is received, and the IP address of the communication terminal 10aa are stored in association with each other (step S24). Accordingly, in the terminal management table shown in FIG. 10, the communication terminal ID “01aa”, the operation state “ON line”, the reception date and time “2009.11.10.13:40”, and the terminal IP address “1” are displayed. .2.1.3 "is managed in association with each other.

  Then, the transmission / reception unit 51 of the communication management system 50 transmits the authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the request source terminal 10aa that has made the login request via the communication network 2. (Step S25). In the present embodiment, the case where the terminal authentication unit 52 determines that the terminal has a valid usage right will be described below.

  The terminal extraction unit 54 of the communication management system 50 searches the destination list management DB 5004 (see FIG. 11) using the terminal ID “01aa” of the request source terminal 10aa that requested the login as a search key, and communicates with the request source terminal 10aa. It extracts by reading the terminal ID of the candidate destination terminal 10B (step S26). Here, the terminal IDs “01ab”, “01ba”, and “01db” of the destination terminals (10ab, 10ba, 10db) corresponding to the terminal ID “01aa” of the request source terminal 10aa are extracted.

  Next, the terminal state acquisition unit 55 uses the terminal IDs (“01ab”, “01ba”, “01db”) of the candidate destination terminal 10B extracted by the terminal extraction unit 54 as a search key, as a terminal management DB 5003 (FIG. 10) and by reading out the operating state (“OFF line”, “ON line”, “ON line”) for each terminal ID extracted by the terminal extraction unit 54, the communication terminals (10ab, 10ba, 10db) is acquired (step S27).

  Next, the transmission / reception unit 51 operates the terminal ID (“01ab”, “01ba”, “01db”) as the search key used in step S27 and the operating state of the corresponding destination terminal (10ab, 10ba, 10db). The destination state information including (“OFF line”, “ON line”, “ON line”) is transmitted to the request source terminal 10aa via the communication network 2 (step S28). As a result, the request source terminal 10aa can operate the communication terminals (10ab, 10ba, 10db) that are candidates for the destination terminal 10B that can communicate with the request source terminal 10aa at the current operation state (“OFF line”, “ "ON line", "ON line").

  Further, the terminal extraction unit 54 of the communication management system 50 searches the destination list management DB 5004 (see FIG. 11) using the terminal ID “01aa” of the request source terminal 10aa that requested the login as a search key, and the request source terminal 10aa. The terminal ID of the other request source terminal 10A in which the terminal ID “01aa” is registered as a candidate for the destination terminal 10B is extracted (step S29). In the destination list management table shown in FIG. 11, the terminal IDs of the other request source terminals 10A to be extracted are “01ab”, “01ba”, and “01db”.

  Next, the terminal state acquisition unit 55 of the communication management system 50 searches the terminal state management DB 5003 (see FIG. 10) using the terminal ID “01aa” of the request source terminal 10aa that has made the login request as a search key. The operating state of the request source terminal 10aa that has requested the login is acquired (step S30).

  Then, the transmission / reception unit 51 includes the terminal management DB 5003 (FIG. 10) among the communication terminals (10ab, 10ba, 10db) related to the terminal IDs (“01ab”, “01ba”, and “01db”) extracted in step S29. The communication terminal (10ba, 10db) whose operation state is “ON line” in the reference) includes the terminal ID “01aa” and the operation state “ON line” of the request source terminal 10aa acquired in step S30. The destination state information is transmitted (steps S31-1, S31-2). When the transmission / reception unit 51 transmits the destination state information to the communication terminals (10ba, 10db), the terminal management table shown in FIG. 10 is used based on each terminal ID (“01ba”, “01db”). Refers to the IP address of the managed terminal. Thus, the terminal ID “01aa” of the request source terminal 10aa that requested the login is sent to each of the other destination terminals (10db, 10ba) that can communicate with the request source terminal 10aa that requested the login, and The operating status “ON line” can be communicated.

  On the other hand, in the other communication terminals 10 as well, when the user turns on the power switch 109 shown in FIG. 4, the operation input receiving unit 12 shown in FIG. Since it receives and performs the same process as the process of said step S22-S31-1,2, the description is abbreviate | omitted.

  Next, processing for narrowing down the relay devices 30 will be described with reference to FIG. In the present embodiment, the request source terminal 10aa is a communication terminal (10ba, 10db) whose operation state is ON line based on the destination state information received in step S28 among the communication terminals 10 as destination candidates. It is possible to communicate with at least one of the above. Thus, hereinafter, a case will be described in which the user of the request source terminal 10aa selects to start communication with the destination terminal 10db.

  First, when the user presses the operation button 108 shown in FIG. 3 and selects the communication terminal 10db, the operation input accepting unit 12 shown in FIG. 5 requests to start communication with the communication terminal 10db. Is received (step S41). Then, the transmission / reception unit 11 of the communication terminal 10aa includes the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db, and starts communication request management information indicating that communication is to be started. It transmits to the system 50 (step S42). As a result, the transmission / reception unit 51 of the communication management system 50 receives the start request information and grasps the IP address “1.2.1.3” of the request source terminal 10aa that is the transmission source. Then, based on the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db included in the start request information, the state management unit 53 uses the terminal management table of the terminal management DB 5003 (see FIG. 10). The field portion of the operating state of the record including the terminal ID “01aa” and the terminal ID “01db” is changed to “busy” (step S43). In this state, the request source terminal 10aa and the destination terminal 10db have not started communication (call), but are in a call state, and another communication terminal 10 tries to communicate with the request source terminal 10aa or the destination terminal 10db. Then, a voice or display indicating a so-called call state is output.

  Next, processing for executing a session for selecting the relay device 30 in steps S44 to S48 and steps S61-1 to S66 will be described. First, the selection session ID generation unit 56a generates a selection session ID used to execute a session for selecting the relay device 30 (step S44). Then, the session management unit 57 stores the selection session ID “se1” generated in step S44, the terminal ID “01aa” of the request source terminal 10aa, and the destination in the session management DB 5005 (see FIG. 12) of the storage unit 5000. The terminal ID “01db” of the terminal 10db is associated and stored and managed (step S45).

  Next, the primary screening unit 56 of the communication management system 50 relays the communication between the request source terminal 10aa and the destination terminal 10db based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006. The primary narrowing down of the relay device 30 is performed (step S46).

  Here, the process in step S46 will be described in more detail with reference to FIG. First, the terminal IP address extraction unit 56b is based on the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db included in the start communication information transmitted from the request source terminal 10aa. By searching the terminal management DB 5003 (see FIG. 10), the IP address (“1.2.1.3”, “1.3.2.4”) of the corresponding communication terminal (10aa, 10db) is obtained. Extract (step S46-1). Next, the primary selection unit 56c selects the relay device (30a, 30b, 30d) that is “ON line” among the operating states of the relay device 30 managed by the relay device management DB 5001 (see FIG. 8). Each relay device ID (111a, 111b, 111d) is selected (step S46-2). Further, the primary selection unit 56c extracts the IP address “1.2.1.3” of the request source terminal 10aa and the IP address “1.3.2.4” of the destination terminal 10db extracted in step S46-1. ”Is searched for the relay device management DB 5001 (see FIG. 8), and the IP addresses (“ 1.2.1. ”) Of the relay devices (30a, 30b, 30d) selected in step S46-2 are searched. 2. ”,“ 1.2.2.2 ”,“ 1.3.2.2 ”) for each IP address (“ 1.2.1 ”) of the request source terminal 10aa and destination terminal 10db. .3 "," 1.3.2.4 "), it is checked whether it is the same as or different from each dot address (step S46-3).

  Next, the priority determination unit 57c refers to the priority management DB 5006 (see FIG. 13), and determines the point of the address priority for each relay device (30a, 30b, 30d) investigated in step 46-3. Is determined (step S46-4). When the result of this determination process is shown in a table, the state shown in FIG. 20 is obtained. FIG. 20 is a diagram illustrating a calculation state of priority points when the relay device 30 performs the narrowing-down process. In FIG. 20, for each relay device ID, an address priority point, a transmission speed priority point, and an integration point are shown. Further, the address priority points further indicate a point for each requesting terminal 10aa and a point for the destination terminal 10db of each relay device 30. The integration point is the sum of the higher one of the two address priority points and the transmission speed priority point.

  In the present embodiment, the IP address “1.2.1.2” of the relay device 30a is “same as the same” as the IP address “1.2.1.3” of the request source terminal 10aa. Therefore, as shown in FIG. 20, the address priority point is “5”. Further, since the IP address “1.2.1.2” of the relay device 30a is “same.different.different.different” with respect to the IP address “1.3.2.4” of the destination terminal 10db, The address priority point is “1”. Further, the IP address “1.2.2.2” of the relay device 30b is “same, same, different, different” with respect to the IP address “1.2.1.3” of the request source terminal 10aa. The point of the address priority is “3”. Further, since the IP address “1.2.2.2” of the relay device 30b is “same.different.same.different” with respect to the IP address “1.3.2.4” of the destination terminal 10db, The address priority point is “1”. Furthermore, the IP address “1.3.2.2” of the relay device 30d is “same.different.different.different” with respect to the IP address “1.2.1.3” of the request source terminal 10aa. The point of the address priority is “1”. Further, the IP address “1.3.2.2” of the relay device 30d is “same. Same. Same. Different” with respect to the IP address “1.3.2.4” of the destination terminal 10db. The address priority point is “5”.

  Next, returning to FIG. 19, the priority determination unit 57d determines the priority management DB 5006 (FIG. 14) based on the maximum data transmission rate of each relay device 30 managed by the relay device management DB 5001 (see FIG. 8). By searching for (reference), the point of the transmission speed priority is determined for each relay device (30a, 30b, 30d) narrowed down by the first narrowing-down process in step S46-2 (step S46-5). In the present embodiment, as shown in FIG. 8, since the maximum data transmission rate of the relay device 30a is 100 (Mbps), referring to the transmission rate priority shown in FIG. The degree is 3 points. Similarly, if the maximum data transmission rate of the relay device 30b is calculated to be 1000 (Mbps), the transmission rate priority is 5 points. Similarly, if the maximum data transmission rate of the relay device 30d is calculated to be 10 (Mbps), the transmission rate priority is 1 point.

  Next, for each relay device (30a, 30b, 30d), the primary selection unit 56c integrates the higher point of the communication terminals (10aa, 10db) and the transmission speed priority point at the address priority point. Among the integrated points, the top two relay devices 30 with higher points are selected (step S46-6). In the present embodiment, as shown in FIG. 20, the relay device IDs (111a, 111b, 111d) have the integration points “8”, “8”, and “6”, respectively. The relay device 30a related to “111a” and the relay device 30b related to the relay device ID “111b” are selected.

  When the narrowing-down process in step S46 is completed, the transmission / reception unit 51 illustrated in FIG. 5 relays the number of relay devices 30 thus narrowed down to the destination terminal 10db via the communication network 2. Device narrowing information is transmitted (step S47). The relay device narrowing information includes the number “2” of the relay devices 30 narrowed down in the step 46, the terminal ID “01aa” of the request source terminal 10aa, and the selection session ID “se1”. . As a result, the communication terminal 10db grasps the number of the relay devices 30 and the communication terminal 10 from which the request for starting the video conference is made in executing the session with the selection session ID “se1”. In addition, the IP address “1.1.1.2” of the communication management system 50 that is the transmission source of the relay device narrowing information can be grasped.

  Then, the communication terminal 10db transmits reception completion information indicating that reception of the relay device narrowing information is completed from the transmission / reception unit 11 to the communication management system 50 via the communication network 2 (step S48). This reception completion information includes the session ID “se1”. As a result, the communication management system 50 sets the IP address “1.3.2.4” of the destination terminal 10db as the transmission source together with the completion of the transmission of the number of relay devices executed with the session ID “se1”. I can grasp it.

  Next, a process in which the destination terminal 10aa selects the relay device 30 will be described with reference to FIG. First, before starting a video conference, the communication management system 50 transmits pre-relay request information for requesting relay in advance to each of the relay devices (30a, 30b) narrowed down in step S46. (Steps S61-1, 2). This pre-relay request information includes the session ID “se1”, the IP address “1.2.1.3” of the request source terminal 10aa, and the IP address “1.3.2.4” of the destination terminal 10db. ing. As a result, the relay device (30a, 30b) can grasp which selection session it belongs to, what the request source terminal 10A is, and what the destination terminal 10B is, and a prior relay request. It is possible to grasp the IP address “1.1.1.2” of the communication management system 50 that is the information transmission source.

  Next, each of the relay devices (30a, 30b) transmits its own device from the transmission / reception unit 31 to the request source terminal 10aa grasped in steps S61-1, 2 via the communication network 2 before the start of the video conference. Is transmitted to each relay device (30a, 30b) as a prior transmission request information indicating that prior transmission information including ping (Packet Internet Groper) described later is transmitted (steps S62-1, 2). This pre-transmission information includes the session ID “se1”. Accordingly, the request source terminal 10aa grasps that the transmission information is transmitted to each relay device (30a, 30b) in the selection process of the relay device 30 that is executed with the session ID “se1”, and the prior transmission. It is possible to grasp the IP address (“1.2.1.2”, “1.2.2.2”) of the relay device (30a, 30b) that is the transmission source of the request information.

  Note that the communication management system 50 does not notify the request source terminal 10ba of the IP address of the destination terminal 10db directly, and instead of notifying the IP address of the destination terminal 10db to the relay device 10aa as in step 61-1. The communication terminal 10aa sends a request to the request source terminal 10ba to transmit the pre-transmission request information to the own apparatus (relay apparatus 10aa) as in step 61-2. This is because the IP address of the other communication terminal 10 is not notified to ensure security.

  Next, the request source terminal 10aa transmits the pre-transmission information from the transmission / reception unit 11 to the relay device (30a, 30b) via the communication network 2 (steps S63-1, 2). Prior to transmission of image data and audio data, this pre-transmission information is transmitted to the destination terminal 10db via each relay device (30a, 30b) instead of these image data and audio data, so that the request source terminal This is information used to measure the time required from transmission of 10aa to reception of the destination terminal db. In addition, the advance transmission information includes a ping for confirming that the request source terminal 10aa, the relay devices (30a, 30b), and the destination terminal 10db are communicably connected, and the advance transmission information from the request source terminal 10aa. And the session ID “se1” is included. Thereby, each relay device (30a, 30b) can grasp that the pre-transmission information has been sent in the execution of the session with the selection session ID “se1”, and is a transmission source of the pre-transmission information. The IP address “1.2.1.3” of the request source terminal 10aa can be grasped.

  Next, each relay device (30a, 30b) responds to the IP address “1.3.2.4” of the destination terminal 10db included in the pre-relay request information received in steps S61-1, 2 above. The prior transmission information is relayed (steps S64-1, 2). As a result, the destination terminal 10db can grasp that the pre-transmission information has been sent in the execution of the session with the session ID “se1”, and can also determine the relay device (transmission source) that is the transmission source (relay source) of the pre-transmission information. 30a, 30b) IP addresses ("1.2.1.2", "1.2.2.2") can be grasped.

  Next, the final narrowing unit 16 of the destination terminal 10db finally narrows down to one relay device 30 that relays image data and audio data in the video conference based on the pre-transmission information (step S65).

  Here, the process in step S65 will be described in more detail with reference to FIGS. First, the measurement unit 16a of the final narrowing unit 16 illustrated in FIG. 5 receives the pre-transmission information relayed by each relay device (30a, 30b) by the transmission / reception unit 11 of the communication terminal 10db. The reception date and time is measured (step S65-1). Next, the calculation unit 16b determines, for each pre-transmission information whose reception time is measured, from the transmission of each pre-transmission information based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information. The time required until reception is calculated (step S65-2). Next, the final selection unit 16c determines whether or not all pieces of pre-transmission information corresponding to the number “2” of relay devices 30 scheduled to be relayed have been received in the execution of the session with the session ID “se1” (Step S65). -3). And when not having received all (NO), the last selection part 16c judges whether predetermined time (here 1 minute) passed since the prior transmission information was received by communication terminal 10db (step S65-). 4). Further, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, when all the data is received in step S65-3 (YES) or when a predetermined time has elapsed (YES) in step S65-4, the final selection unit 16c has been calculated by the calculation unit 16c so far. One relay device 30 that relays the pre-transmission information that requires the shortest required time is selected (step S65-5). In the present embodiment, the relay device 30a is selected on the assumption that the pre-transmission information relayed by the relay device 30a has a shorter time from transmission to reception than the pre-transmission information relayed by the relay device 30b. Is shown.

  In the above embodiment, the relay device 30a is narrowed down on the destination terminal 10db side. However, the present invention is not limited to this, and the destination terminal 10db receives the transmission of the pre-transmission information to the request source terminal 10aa or the communication management system 50. By transmitting all the required time information indicating the required time until the requesting terminal 10aa side or the communication management system 50 side, one relay device 30a may be finally narrowed down.

  Next, the destination terminal 10db transmits selection information indicating that the relay device 30a has been selected from the transmission / reception unit 11 to the communication management system 50 via the communication network 2 (step S66). This selection information includes the session ID “se1” and the relay device ID “111a” of the selected relay device 30a. As a result, the communication management system 50 can grasp that the relay device 30a has been selected in the execution of the session with the session ID “se1”, and the IP address “1.3” of the communication terminal 10db that is the transmission source of the selection information. .2.4 ".

  Next, the session management unit 56 of the communication management system 50 finally adds the relay device ID field of the record including the session ID “se1” in the session management table of the session management DB 5005 (see FIG. 12). The relay device ID “111a” of the relay device 30a selected as one is stored and managed (step S67). Then, the transmission / reception unit 51 of the communication management system 50 transmits relay start request information indicating a request to start relaying to the relay device 30a via the communication network 2 (step S68). The relay start request information includes the IP addresses (“1.2.1.3” and “1.3.2.4”) of the request source terminal 10aa and the destination terminal 10db to be relayed. As a result, the relay device 30a establishes a session for communicating the low resolution, medium resolution, and high resolution three image data and audio data between the communication terminals (10aa, 10db) (step S69). ). Thereby, the communication terminals (10aa, 10db) can start a video conference.

  In step S47, the communication management system 50 transmits the relay device narrowing information to the destination terminal 10db. As a result, the relay device selection process (steps S48 to S64-1, 2) is performed on the destination terminal 10db side. Step S65) is performed, but the present invention is not limited to this. In step S47, the communication management system 50 transmits the relay device narrowing information to the request source terminal 10aa. The transmission source and the reception source of each information may be switched between the request source terminal 10aa and the destination terminal 10db. As a result, the request source terminal 10aa can perform the relay device selection process instead of step S65, and can also transmit the selection information instead of step S66.

  Next, a process of transmitting and receiving image data and audio data in order to perform a video conference between the request source terminal 10aa and the destination terminal 10db will be described with reference to FIGS. First, the request source terminal 10aa transmits the image data of the subject imaged by the imaging unit 14a and the audio data of the audio input by the audio input unit 15a from the transmission / reception unit 11 to the relay device 30a via the communication network 2. (Step S81). In the present embodiment, the high-quality image data including the low resolution, medium resolution, and high resolution shown in FIG. 6 and the audio data are transmitted. Thereby, in the relay device 30a, the transmission / reception unit 31 receives the image data and the audio data of the three resolutions. Then, the data quality confirmation unit 33 searches the change quality management DB 3001 (see FIG. 7) using the IP address “1.3.2.4” of the destination terminal 10db as a search key, and the image quality of the corresponding relayed image data Is extracted to confirm the image quality of the image data to be relayed (step S82). In the present embodiment, the image quality of the confirmed image data is “high quality” and is the same as the image quality of the image data received by the transmission / reception unit 31. The voice data is transferred to the destination terminal 10db (step S83). As a result, the destination terminal 10db receives the image data and the audio data at the transmission / reception unit 11, the image display control unit 14b causes the display 11 to display an image based on the image data, and the audio output unit 15b based on the audio data. Audio can be output.

  Next, the delay detection unit 17 of the communication terminal 10db detects the reception delay time of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every second) (step S84). In the present embodiment, the following description is continued for a case where the delay time is 200 (ms).

  The transmission / reception unit 11 of the destination terminal 10db transmits delay information indicating the delay time “200 (ms)” to the communication management system 50 via the communication network 2 (step S85). Thereby, the communication management system 50 can grasp the delay time and the IP address “1.3.2.4” of the communication terminal 10db that is the transmission source of the delay information.

  Next, the delay time management unit 60 of the communication management system 50 searches the terminal management DB 5003 (see FIG. 10) using the IP address “1.3.2.4” of the destination terminal 10db as a search key. The corresponding terminal ID “01db” is extracted, and in the session management table of the session management DB 5005 (see FIG. 12), the delay information field portion of the record of the terminal ID “01db” is indicated by the delay information. The stored delay time “200 (ms)” is stored and managed (step S86).

  Next, the quality determination unit 58 searches the quality management DB 5007 (see FIG. 15) using the delay time “200 (ms)” as a search key, and extracts the image quality “medium image quality” of the corresponding image data. The image quality is determined as “medium image quality” (step S87).

  Next, the transmission / reception unit 51 uses the relay device ID “111a” associated with the terminal ID “01db” in the session management table of the session management DB (see FIG. 12) as a search key, the relay device management DB 5001 (FIG. 8) and the IP address “1.2.1.2” of the corresponding relay device 30a is extracted (step S88). Then, the transmission / reception unit 51 transmits quality information indicating the image quality “medium image quality” of the image data determined in step S87 to the relay device 30a via the communication network 2 (step S89). This quality information includes the IP address “1.3.2.4” of the destination terminal 10db used as the search key in step S86. Accordingly, in the relay device 30a, the change quality management unit 34 stores the IP address “1.3.2. Of the destination communication terminal 10 (here, the destination terminal 10db) in the change quality management DB 3001 (see FIG. 7). 4 ”and the image quality“ medium image quality ”of the relayed image data are stored and managed in association with each other (step S90).

  Next, similarly to step S81, the communication terminal 10aa continuously transmits high-quality image data including three images of low image quality, medium image quality, and high image quality, and audio data to the relay apparatus 30a (step S81). S91). As a result, in the relay device 30a, as in step S82, the data quality confirmation unit 33 uses the IP address “1.3.2.4” of the destination terminal 10db as a search key to change the quality management DB 3001 (see FIG. 7). ) And the image quality “medium image quality” of the corresponding image data to be relayed is extracted to check the image quality of the image data to be relayed (step S92). In this embodiment, since the image quality of the confirmed image data is “medium image quality” and lower than the image quality “high image quality” of the image data received by the transmission / reception unit 31, the data quality changing unit 35 The image quality of the image data is changed by suppressing the image quality from “high image quality” to “medium image quality” (step S93). Then, the transmission / reception unit 31 transmits the image data in which the image quality of the image data is changed to “medium image quality” and the audio data in which the sound quality of the audio is not changed to the communication terminal 10 db via the communication network 2 ( Step S94). In this way, when reception delay occurs at the destination terminal 10db that receives the image data, the relay device 30a changes the quality of the image so that the person participating in the video conference does not feel uncomfortable. can do.

  Next, processing in which the transmission-side communication terminal 10aa transmits image data based on the display states of the reception-side communication terminal 10bb and the communication terminal 10cb will be described with reference to FIG.

  First, the operation input receiving unit 12 of the communication terminal 10bb that receives image data during a meeting receives a display instruction for changing the size of an image by a user operation (step S70-1). Next, the image layout processing unit 20 displays an image having a size based on the instruction received in step S70-1 on the display 100bb (step S71-1).

  On the other hand, when the operation input receiving unit 12 receives an instruction to change the size of the image from the user, the display state acquiring unit 21 represents a display state including a display height ratio, a display width ratio, and a display ratio on the display. Display state information is acquired (step S72-1). The display height ratio represents the ratio of the height of the displayed image to the height of the entire screen of the display 100bb of the communication terminal 10, and the display width ratio is the screen of the display 100bb of the communication terminal 10. This represents the ratio of the height of the displayed image to the overall width. Further, the display rate represents a rate at which an image is not covered with another image. For example, as shown in FIG. 26, an image is displayed on the left side of the display 100bb of the communication terminal 10bb based on the image data transmitted from the communication terminal 10aa. In the display state of this image, as shown in FIG. 26, the image is displayed in an area occupying a display width ratio of 80% and a display height ratio of 100%, and the display ratio is 100%.

  When the display state acquisition unit 21 of the communication terminal 10bb acquires the display state including the display height rate, the display width rate, and the display rate in step S72-1, the display area rate calculation unit 22 is transmitted from the communication terminal 10aa. The display area ratio of the displayed image is calculated based on the image data. Specifically, the display area ratio is calculated by multiplying the display height ratio, the display width ratio, and the display ratio (step S73-1). In the example shown in FIG. 26, since the image displayed on the screen of the communication terminal 10bb has a display width ratio of 80%, a display height ratio of 100%, and a display ratio of 100%, the display area ratio is 80% × 100% ×. 100% = 80%.

  When the display area ratio calculation unit 22 calculates the display area ratio bb of the communication terminal 10bb in step S73-1, the transmission / reception unit 11 transmits information representing the display area ratio to the communication management system 50. The transmission / reception unit 51 of the communication management system 50 transmits the display area ratio bb received from the communication terminal 10bb to the communication terminal 10aa (step S74-1). When the transmission / reception unit 11 of the communication terminal 10aa receives the display area ratio 10bb, the storage / reading processing unit 19 stores it in the display state management DB 1002 of the storage unit 1000 (step S75-1).

  On the other hand, the communication terminal 10cb performs the same processing as Step S70-1 to Step S75-1 of the communication terminal 10bb (Step S70-2 to Step S75-2). When the display area rate is transmitted to the communication terminal 10aa in step S75-2, the storage / reading processing unit 19 of the communication terminal 10aa stores the display area rate in the display state management DB 1002 of the storage unit 1000 (step S76).

  Next, the image quality determination unit 23 of the communication terminal 10aa determines the image quality of the image data to be transmitted based on the display area ratio stored in the display state management DB 1002. Specifically, as shown in FIG. 28, first, the change detection unit 23a detects that the display area ratio corresponding to any terminal ID stored in the display state management DB 1002 has changed ( Step S771). When the change detection unit 23a detects a change in the display area rate, the average display area rate calculation unit 23b calculates an average display area rate that is an average of the display area rates corresponding to each terminal ID (step S772). Specifically, as shown in FIG. 25, the display area ratio of the communication terminal 10bb is 80%, the display area ratio of the communication terminal 10cb is 65%, and the display area ratio of the communication terminal 10db is 35%. The display area ratio is (80 (%) + 65 (%) + 35 (%)) / 3 = 60 (%).

  If the change detection unit 23a does not detect a change in the display area ratio in step S771, the process ends.

Then, the determination unit 23c searches the image quality management table of the image quality management DB 1001 shown in FIG. 24 using the average display area ratio as a search key, extracts the associated image quality, and the image quality change unit 24 extracts The image quality of the image data is changed to the image quality (step S77).
The program may determine the image quality based on the flowchart shown in FIG. In this case, first, it is determined whether or not the average display area ratio is 30% or less (step S773). If the average display area ratio is 30% or less, the image quality of the image is determined as “low image quality” (step S774).
If the average display area ratio exceeds 30%, it is determined whether or not the average display area ratio is 70% or less (step S775). If the average display area ratio is 70% or less, the image quality of the image is determined as “medium image quality” (step S776), and if the average display area ratio exceeds 70%, the image quality of the image is determined as high image quality (step S776). ).

  When the image quality of the image data is changed in step S77 (step S77), the transmission / reception unit 11 transmits the image data with the changed image quality to the relay device 30 (step S78). Then, the relay device 30 transmits the received image data to each communication terminal 10 (step S80).

<< Main effects of embodiment >>
As described above, according to this embodiment, a device that transmits image data receives display states from a plurality of devices that receive image data, and determines the image quality of image data to be transmitted based on those display states. By doing so, it is possible to transmit image data with good quality (image quality) of images displayed on any device.

  Further, even if the LAN 2 environment such as the IP address of the relay device 30 in the communication network 2 can be obtained, it is difficult to obtain the environment of the Internet 2i. To two or more of a plurality of relay apparatuses 30 that relay image data and audio data. Then, before actually transmitting / receiving image data and audio data among the plurality of communication terminals 10, it is actually the earliest possible in advance by transmitting / receiving pre-transmission information instead of image data and audio data. There is an effect that the transmission information can be narrowed down to one relay device 30 that can relay the transmission information.

  That is, by selecting the relay device 30 to which the upper two or more IP addresses close to any of the IP addresses of the communication terminals 10 are selected, two or more candidates for the relay device 30 to be finally used are selected. Can leave. As a result, by transmitting and receiving the pre-transmission information between the request source terminal 10A and the destination terminal 10B via each relay device 30 as a candidate in actuality, Therefore, it is possible to narrow down to the relay device 30 that relays the pre-transmission information having the shortest time required for transmission and reception. Therefore, it is possible to realize transmission / reception of image data or audio data with the highest image quality under the environment of the current communication network 2.

  In the present embodiment, when narrowing down the relay devices 30, not only the relay device 30 having an IP address close to the IP address of the communication terminal 10 that conducts the video conference is preferentially selected, but the maximum data in each relay device 30 is also selected. Two or more relay apparatuses 30 are selected in consideration of the transmission speed. As a result, the candidates for the relay device 30 in accordance with the actual environment of the communication network 2 can be narrowed down.

Furthermore, in this embodiment, when narrowing down the relay devices 30, the candidates for the relay devices 30 that further match the actual network 2 environment can be narrowed down to narrow down the relay devices 30 whose operation state is the ON line. it can.
<< Supplement of Embodiment >>
Although an example of a conference between distant locations is shown as a teleconferencing system, remote conferencing is not limited to a distant location, as in a conference room in the same building. Conference rooms close to each other, and between places where the voice is not heard even in the same room. That is, the remote conference means not a conference that is directly face-to-face, but a conference that is performed via a conference device. In addition, the conference includes not only a meeting between a plurality of people but also a meeting held by one person.

  In addition, the display area ratio corresponding to each communication terminal 10 may be set as an average display area ratio by holding a history for a predetermined number of times and calculating an average value of all of the history. In the method of determining the image quality of the image data based on the latest display state described above, when the display state of any one of the communication terminals is greatly changed, the image quality is greatly changed. When the display state for a predetermined number of times is used, it is possible to avoid a sudden change in image quality.

  Moreover, in the said embodiment, the display state detection part 21 of the communication terminal 10 may perform a process, when the image layout process part 20 changes the magnitude | size of an image, The predetermined specific timing, Alternatively, the process may be executed at a specific time.

  Further, the communication management system 50 and the program providing system 90 in each of the above embodiments may be constructed by a single computer, or by a plurality of computers arbitrarily assigned by dividing each unit (function or means). It may be constructed. In addition, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may be transmitted by dividing it into a plurality of modules. It may be transmitted. Further, when the program providing system 90 is constructed by a plurality of computers, a plurality of modules may be divided and transmitted from each computer.

  In addition, the communication terminal program, the relay device program, and the communication management program of the present embodiment, the HD 204 storing these programs, and the program providing system 90 including the HD 204 are stored. These are used when the above-mentioned communication terminal program, relay device program, and communication management program are provided to a user or the like as a program product domestically or abroad.

  Furthermore, in the above-described embodiment, an example of the image quality (image quality) of the image data relayed by the relay device 30 using the changed quality management table shown in FIG. 7 and the quality management table shown in FIG. However, the present invention is not limited to this, and other examples of image quality include depth of image data image quality, sampling frequency of audio data audio, and audio data audio. It may be managed by paying attention to the bit length in

  8, 10, and 12 manage the reception date and time. However, the present invention is not limited to this, and at least the reception time of the reception date and time may be managed.

  Furthermore, in the above embodiment, the IP address of the relay device is managed in FIG. 8 and the IP address of the communication terminal is managed in FIG. 10, but the present invention is not limited to this, and the relay device 30 is specified on the communication network 2. Each FQDN (Fully Qualified Domain Name) may be managed as long as it is relay device specifying information or terminal specifying information for specifying the communication terminal 10 on the communication network 2. In this case, an IP address corresponding to the FQDN is acquired by a known DNS (Domain Name System) server. In addition to “relay device specifying information for specifying the relay device 30 in the communication network 2”, “relay device connection destination information indicating a connection destination to the relay device 30 on the communication network 2” or “communication It may be expressed as “relay device destination information indicating a destination to the relay device 30 on the network 2”. Similarly, not only “terminal specifying information for specifying the communication terminal 10 in the communication network 2” but also “terminal connection destination information indicating a connection destination to the communication terminal 10 on the communication network 2” or “communication network 2”. It may be expressed as “terminal destination information indicating the destination to the communication terminal 10”.

  In the above embodiment, the case of a video conference system has been described as an example of the communication system 1. However, the present invention is not limited to this, and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone. . The communication system 1 may be a car navigation system. In this case, for example, one of the communication terminals 30 corresponds to a car navigation device mounted on a car, and the other of the communication terminals 30 is mounted on a management terminal or management server of a management center that manages car navigation, or another car. It corresponds to the car navigation device that is used.

10 Communication terminal 11 Transmitter / receiver (an example of display state information receiving means, an example of transmitting means)
DESCRIPTION OF SYMBOLS 16 Final narrowing down part 16a Measurement part 16b Calculation part 16c Final selection part 17 Delay detection part 19 Memory | storage / reading process part 20 Image layout display part 21 Display state acquisition part 22 Display area rate calculation part 23 Image quality determination part (an example of a determination means) )
23a change detection unit 23b average display area ratio calculation unit 23c determination unit 24 image quality change unit 31 transmission / reception unit 32 state detection unit 33 data quality confirmation unit 34 change quality management unit 35 data quality change unit 51 transmission / reception unit 52 terminal authentication unit 53 state management Unit 54 terminal extraction unit 55 terminal state acquisition unit 56 primary narrowing unit 56a selection session ID generation unit 56b terminal IP address extraction unit 56c primary selection unit 56d priority determination unit 57 session management unit 58 quality determination unit 60 delay time management unit 1000 storage unit 1001 image quality management DB
1002 Display state management DB
3000 storage unit 3001 change quality management DB
5000 storage unit 5001 relay device management DB
5002 Terminal authentication management DB
5003 Terminal management DB
5004 Destination list management DB
5005 Session management DB
5006 Priority management DB
5007 Quality Management DB

JP 7-236136 A

Claims (4)

A communication terminal that transmits image data to a plurality of other communication terminals,
Display state information receiving means for receiving display state information indicating a state in which an image based on the image data is displayed in each of the plurality of other communication terminals;
Display state information storage means for storing a plurality of the display state information received by the display state information receiving means;
Determining means for determining the image quality of the image data based on a plurality of the display state information stored in the display state information storage means;
A transmission unit configured to transmit the image quality of the image data to the other communication terminals with the image quality determined by the determination unit;
Have
The display state information includes an average value of a display area ratio calculated from a height, a width, and a display ratio of an image based on the image data displayed on the plurality of other communication terminals. Communication terminal. The communication terminal according to claim 1,
The determining means includes
Image quality information storage means for storing the display state information and image quality information indicating the image quality in association with each other;
Extraction means for extracting the image quality information corresponding to the display state information by searching the image quality information storage unit based on the display state information stored in the display state information storage unit;
A communication terminal that determines the image quality represented by the image quality information extracted by the extraction means. A communication program executed by a communication terminal that transmits image data to a plurality of other communication terminals,
The communication terminal is
Display state information storage means for storing display state information indicating a state in which an image based on the image data is displayed in each of the plurality of other communication terminals;
A display state information receiving step of receiving display state information indicating a state in which an image based on the image data is displayed in each of the plurality of other communication terminals;
A display state information storage step of storing a plurality of the display state information received in the display state information reception step in the display state information storage unit;
A determination step of determining an image quality of the image data based on the plurality of display state information stored in the display state information storage step;
Causing the computer to execute a transmission step of transmitting the image data of the image data to the image quality determined in the determination step and transmitting the image data to the other communication terminals,
The display state information includes an average value of a display area ratio calculated from a height, a width, and a display ratio of an image based on the image data displayed on the plurality of other communication terminals. Communication program. The communication program according to claim 3 ,
The communication terminal has image quality information storage means for storing the display state information and image quality information indicating the image quality in association with each other;
The determination step includes
An extraction step of extracting the image quality information corresponding to the display state information by searching the image quality information storage unit based on the display state information stored in the display state information storage unit;
A communication program comprising a determination step of determining an image quality represented by the image quality information extracted in the extraction step .

Images