JP2017069967A - Transmission system, external input device and display resolution conversion program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an operation load of a user when the connection of a display device is changed to another display device, by enabling display data to be shared with another transmission terminal without changing the resolution of an external input device by a user.SOLUTION: A transmission system includes a transmission terminal connected to a display device of which connection can be changed to a different display device. When the connection of the display device is changed to the other display device, the transmission terminal acquires a displayable resolution (S112) of the other display device and a resolution (S111) displayed at present. When a resolution selected from the acquired displayable resolution is different from the resolution which is displayed at present by the other display device (S115, S120), the transmission terminal changes the resolution of the other display device into the selected resolution (S117, S122). The external input device changes the resolution of the display data into a resolution which fits to the other display device of the transmission terminal, on the basis of the changed resolution and the resolution of a display device of the external input device.SELECTED DRAWING: Figure 31

Description

  The present invention relates to a transmission system adapted to transmit display data displayed on an external input device connected to a transmission terminal to another transmission terminal via a communication network.

  In recent years, transmission systems that perform video conferences and the like with remote locations via a communication network such as the Internet have become widespread. In such a transmission system, a video conference can be realized by transmitting and receiving image data and audio data among a plurality of transmission terminals. In addition, the enhancement of broadband environment in recent years has made it possible to send and receive high-quality image data and high-quality audio data, making it easier to understand the status of the other party in the video conference and enhancing the level of communication through conversations. Can be improved.

  In addition, when conducting a video conference using a transmission terminal, the parties to the video conference use an external input device such as their own PC (Personal Computer) separately from the transmission terminal, and the conference material is displayed on the display of the external input device. In some cases, discussions or the like are performed with the other party. In such a case, the conventional external input device transmits the display data representing the image of the conference material or the like displayed on the display of the external input device by the other party of the video conference via the communication network. It was sent to the terminal.

  By doing so, the parties of the video conference can hear the voice of the other party and see the face by the audio data and the image data transmitted and received via the transmission terminal when the conference is being held. At the same time, the display data displayed on the display of the external input device can be shared with the other party of the video conference.

  Further, as another technique related to sharing of display data between terminals, for example, there is a technique described in Patent Document 1. The data conference system disclosed in Patent Document 1 includes a terminal and a remote terminal connected to the terminal, and displays data on a display screen displayed on a display by an application in the terminal as a graphical device. A display screen is shared between the terminal and the remote terminal by transmitting capture data captured by the interface to the remote terminal, and the terminal displays the display on the remote terminal. Resolution information is stored, resolution conversion of the capture data is performed on the entire display screen based on the display display resolution information, and the capture data subjected to resolution conversion is transmitted to the remote terminal.

  However, conventionally known external input devices are limited in resolution to support image display in the transmission terminal. Therefore, when the user desires to share the display data, the resolution of the external input device must be changed to a resolution supported by the transmission terminal, and there is a problem that it takes time to operate the external input device. .

  Such a problem can occur in the data conference system described in Patent Document 1 as well. That is, when an external input device is connected to the terminal on the local site side and the display data of the external input device is transmitted to the terminal side, the user must make a setting for changing the resolution of the external input device, which is a burden on the operation. The problem that it takes is left.

  The present invention was created in view of the above-described conventional problems, and the object of the present invention is to share display data with other transmission terminals without the user changing the resolution of the external input device. This is to reduce the burden of user operations.

  In order to solve the above-described problem, a transmission system according to an embodiment of the present invention is connected to a display device that is communicably connected to another transmission terminal and can be connected to another display device. A transmission system comprising: a transmission terminal that outputs image data for display; and an external input device that transmits display data to the transmission terminal, wherein the display device is the display device When the display device is switched to, the resolution acquisition means for acquiring the displayable resolution and the currently displayed resolution of the another display device, and when the display device is switched to the another display device, the resolution The selected resolution when the resolution selected from the displayable resolutions of the other display device acquired by the acquisition unit and the resolution currently displayed by the other display device are different. Resolution changing means for changing the resolution of the other display device, and resolution information for transmitting first resolution information representing the changed resolution to the external input device when the resolution changing means changes the resolution. A transmission means, and the external input device receives resolution information reception means for receiving the first resolution information transmitted from the transmission terminal, the received first resolution information, and the external input. Resolution conversion means for converting the resolution of the display data to a resolution suitable for the other display device of the transmission terminal, based on second resolution information representing the resolution of the display device of the apparatus, and the resolution conversion means Display data transmitting means for transmitting the display data whose resolution has been converted from the transmission terminal to the other transmission terminal to transmit the display data to the other transmission terminal.

  According to the present invention, it is possible to share display data with other transmission terminals without the user changing the resolution of the external input device, thereby reducing the burden on the user's operation.

It is the schematic which shows the whole structure of the transmission system which concerns on one Embodiment of this invention. It is an external view of the transmission terminal which comprises the transmission system of FIG. It is a hardware block diagram of the transmission terminal which comprises the transmission system of FIG. FIG. 2 is a hardware configuration diagram of a transmission management system (relay device, external input device, or program providing system) that constitutes the transmission system of FIG. 1. FIG. 2 is a functional block diagram of a transmission terminal, a relay device, and a transmission management system that constitute the transmission system of FIG. 1. It is a figure for demonstrating the image quality of image data. It is a figure which shows an example of a change quality management table. It is a figure which shows an example of a relay apparatus management table. It is a figure which shows an example of a terminal authentication management table. It is a figure which shows an example of a terminal management table. It is a figure which shows an example of a destination list management table. It is a figure which shows an example of a session management table. It is a figure which shows an example of an address priority management table. It is a figure which shows an example of a transmission rate priority management table. It is a figure which shows an example of a quality management table. It is a sequence diagram which shows an example of the process which manages the status information showing the operating state of each relay apparatus. It is a sequence diagram which shows an example of the process of the preparation stage which starts communication between each transmission terminal. It is a sequence diagram which shows an example of the process which a transmission management system narrows down a relay apparatus. It is a flowchart which shows an example of a relay apparatus narrowing-down process. It is a figure which shows the calculation state of the point of priority at the time of performing a relay apparatus narrowing-down process. It is a sequence diagram which shows an example of the process which a transmission terminal selects a relay apparatus. It is a flowchart which shows an example of a relay apparatus selection process. It is a sequence diagram which shows an example of the process which transmits / receives image data and audio | voice data between transmission terminals. It is a functional block diagram of an external input device. It is a figure which shows an example of the resolution management table. It is a sequence diagram which shows an example of the process which displays the display data which the external input device displayed on the transmission terminal which is the other party of a meeting. It is a figure which shows an example of the screen which an external input device displays. It is a figure which shows an example of the screen on which a transmission terminal displays image data and display data. FIG. 10 is a sequence diagram illustrating an example of processing (in the case of using an uncompressed format) for converting the resolution of display data transmitted from the external input device to the transmission terminal based on the display resolution of the transmission terminal and the display resolution of the external input device. It is a flowchart which shows an example of the process which converts resolution. It is a flowchart which shows an example of the process of the resolution control (a change of the resolution is included) which a transmission terminal performs. It is a figure for supplementary explanation of the judgment processing performed at Step S102 of Drawing 31, and is a figure showing an example of a screen of auxiliary information display when a segment area is added (a) and not added (b). . It is a sequence diagram which shows the other example (when using a compression format) of converting the resolution of the display data which an external input device transmits to a transmission terminal based on the display resolution of a transmission terminal, and the display resolution of an external input device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<< Overall Configuration of Embodiment >>
FIG. 1 is a schematic diagram showing an overall configuration of a transmission system 1 according to an embodiment of the present invention. Hereinafter, this embodiment will be described with reference to FIG.

  In general, a transmission system includes a data providing system that communicates content data in one direction from one transmission terminal to the other transmission terminal via a transmission management system, and information and emotion between a plurality of transmission terminals via a transmission management system. Communication system that communicates each other. This communication system is a system for mutually transmitting information, emotions, etc. between a plurality of communication terminals (corresponding to “transmission terminal”) via a communication management system (corresponding to “transmission management system”). Alternatively, a video conference system, a video phone system, a voice conference system, a voice phone system, a PC (Personal computer) screen sharing system, and the like can be given as examples.

  In the present embodiment, transmission is performed assuming a television or video conference system as an example of a communication system, a television or video conference management system as an example of a communication management system, and a television or video conference terminal as an example of a communication terminal. A system, a transmission management system, and a transmission terminal will be described. That is, the transmission terminal and the transmission management system of the present invention are applied not only to a television or video conference system, but also to a communication system or a transmission system.

  The transmission system 1 (see FIG. 1) of the present embodiment includes a plurality of transmission terminals 10aa, 10ab, 10ba, 10bb, 10ca, 10cb, 10da and 10db, and a plurality of displays 120aa, 120ab, 120ba and 120bb for each transmission terminal. , 120ca, 120cb, 120da and 120db, a plurality of external input devices 40aa, 40ab, 40ba, 40bb, 40ca, 40cb, 40da and 40db, such as a plurality of PCs connected to the respective transmission terminals, and a plurality of relay devices 30a, 30b , 30c and 30d, the transmission management system 50, the program providing system 90, and the maintenance system 100.

  Unless otherwise specified in the following description, when any transmission terminal among the plurality of transmission terminals 10aa,..., 10db is indicated, it is expressed as “transmission terminal 10”, and the plurality of displays 120aa,. ..., 120 db indicates an arbitrary display as “display 120”, and indicates a plurality of external input devices 40 aa,. It is expressed as “external input device 40”, and when it indicates an arbitrary relay device among the plurality of relay devices 30a,..., 30d, it is expressed as “relay device 30”. The display 120 is an example of a display device of the transmission terminal 10.

  The transmission terminal 10 transmits / receives image data, audio data, and the like to / from other transmission terminals 10. In the present embodiment, the case where the image of the image data is a moving image will be described, but it may be a still image as well as a moving image. Moreover, both the moving image and the still image may be included in the image of the image data. The relay device 30 is for relaying image data and audio data between the transmission terminals 10. The transmission management system 50 is for managing a plurality of transmission terminals 10 and a plurality of relay apparatuses 30 in an integrated manner.

  The external input device 40 is connected to the transmission terminal 10 and transmits display data representing an image displayed on a display device (a display 216 described later) of the external input device 40 to the transmission terminal 10.

A plurality of routers 70a, 70b, 70c, 70d, 70e, and 70f shown in FIG. 1 are for selecting an optimum route through which image data and audio data are transmitted and received. Unless otherwise specified in the following description, an arbitrary router among the plurality of routers 70a,..., 70f is expressed as “router 70”. The program providing system 90 includes an HD (Hard Disk) (not shown). The HD includes a transmission terminal program for realizing various functions or various means in the transmission terminal 10, a relay device program for realizing various functions or various means in the relay device 30, and the transmission management system 50. A transmission management program for realizing various functions or various means is stored. The program providing system 90 is constructed so that the transmission terminal program, the relay device program, and the transmission management program stored in the HD can be transmitted to the transmission terminal 10, the relay device 30, and the transmission management system 50, respectively.

  The transmission terminals 10aa and 10ab, the relay device 30a, and the router 70a are communicably connected via a LAN (Local Area Network) 2a. The transmission terminal 10ba, the transmission terminal 10bb, the relay device 30b, and the router 70b are The LAN 2b is communicably connected. The LAN 2a and the LAN 2b are communicably connected by a dedicated line 2ab including the router 70c, and are constructed in a predetermined area A. For example, the region A is Japan, the LAN 2a is constructed in a Tokyo office, and the LAN 2b is constructed in an Osaka office.

  On the other hand, the transmission terminals 10ca and 10cb, the relay device 30c, and the router 70d are communicably connected by the LAN 2c, and the transmission terminals 10da and 10db, the relay device 30d, and the router 70e are communicably connected by the LAN 2d. Yes. The LAN 2c and the LAN 2d are communicably connected by a dedicated line 2cd including the router 70f, and are constructed in a predetermined area B. For example, region B is the United States, LAN 2c is built in a New York office, and LAN 2d is Washington D.C. C. Is built in the office. Area A and area B are communicably connected via the Internet 2i from routers 70c and 70f, respectively.

  In addition, the transmission management system 50, the program providing system 90, and the maintenance system 100 are communicably connected to the transmission terminal 10 and the relay device 30 via the Internet 2i. The transmission management system 50, the program providing system 90, and the maintenance system 100 may be installed in the region A or the region B, or may be installed in other regions.

  In this embodiment, the communication network 2 is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d.

  In FIG. 1, four sets of numbers (with parentheses) written under each transmission terminal 10, each relay device 30, transmission management system 50, each router 70, program providing system 90, and maintenance system 100 are An IP address in general IPv4 (Internet Protocol version 4) is simply shown. For example, the IP address of the transmission terminal 10aa is “1.2.1.3”. In addition, IPv6 may be used instead of IPv4, but in order to simplify the description, description will be made using IPv4.

<< Hardware Configuration of Embodiment >>
Next, the hardware configuration of this embodiment will be described.

  FIG. 2 is an external view of the transmission terminal 10 constituting the transmission system 1 according to the present embodiment. In the following description, the longitudinal direction of the transmission terminal 10 is the X-axis direction, the direction orthogonal to the X-axis direction in the horizontal plane is the Y-axis direction, and the direction orthogonal to the X-axis direction and the Y-axis direction (vertical direction) is the Z-axis direction. Will be described.

  As shown in FIG. 2, the transmission terminal 10 includes a housing 1100, an arm 1200, and a camera housing 1300. Among these, the front side wall surface 1110 of the housing 1100 is provided with an air intake surface (not shown) formed by a plurality of air intake holes, and the rear side wall surface 1120 of the housing 1100 has a plurality of exhaust holes. A formed exhaust surface 1121 is provided. As a result, by driving a cooling fan built in the housing 1100, the outside air behind the transmission terminal 10 is taken in via the intake surface (not shown) and exhausted to the rear of the transmission terminal 10 via the exhaust surface 1121. be able to. A sound collecting hole 1131 is formed in the right wall surface 1130 of the housing 1100, and sounds such as voice, sound, noise can be collected by a built-in microphone 114 described later.

  An operation panel 1150 is formed on the right wall surface 1130 side of the housing 1100. The operation panel 1150 is provided with a plurality of operation buttons (108a to 108e) to be described later, a power switch 109 to be described later, and an alarm lamp 119 to be described later, and outputs sound from a built-in speaker 115 to be described later. A sound output surface 1151 formed by a plurality of sound output holes for passing through is formed. In addition, a housing portion 1160 as a recess for housing the arm 1200 and the camera housing 1300 is formed on the left wall surface 1140 side of the housing 1100. The right side wall surface 1130 of the housing 1100 is provided with a plurality of connection ports (1132a to 1132c) for electrically connecting cables to an external device I / F 118 described later. On the other hand, the left wall surface 1140 of the housing 1100 is provided with a connection port (not shown) for electrically connecting a cable 120c for the display 120 to a display I / F 117 described later.

  In the following description, “operation button 108” is used to indicate any operation button among the operation buttons (108a to 108e), and any connection port among the connection ports (1132a to 1132c) is indicated. Will be described using “connection port 1132”.

  The arm 1200 is attached to the housing 1100 via a torque hinge 1210. More specifically, the arm 1200 is provided to be rotatable in the vertical direction with respect to the housing 1100 within a tilt angle θ1 of 135 degrees. In the example of FIG. 2, the tilt angle θ1 is 90 degrees.

  The camera housing 1300 is provided with a built-in camera 1021 that can capture images of users, documents, rooms, and the like. A torque hinge 1310 is formed in the camera housing 1300. The camera housing 1300 is attached to the arm 1200 via a torque hinge 1310. More specifically, the camera housing 1300 has a pan angle θ2 of ± 180 degrees and a tilt angle θ3 of ± 45 degrees with respect to the arm 1200, with the state shown in FIG. In the range, it is provided to be rotatable in the vertical direction and the horizontal direction, respectively.

  Note that the relay device 30, the transmission management system 50, and the program providing system 90 have the same external appearance as a general server computer, and thus the description of the external appearance is omitted.

FIG. 3 is a hardware configuration diagram of the transmission terminal 10 constituting the transmission system 1 according to the present embodiment. As shown in FIG. 3, the transmission terminal 10 includes a CPU (Central Processing Unit) 101 that controls the operation of the entire transmission terminal 10, a ROM (Read Only Memory) 102 that stores a transmission terminal program, and a work of the CPU 101. A RAM (Random Access Memory) 103 used as an area, a flash memory 104 for storing various data such as image data and audio data, and an SSD (control for reading and writing various data to and from the flash memory 104 under the control of the CPU 101 Solid State Drive) 105, a media drive 107 that controls reading or writing (storage) of data to a recording medium 106 such as a flash memory, an operation button 108 that is operated when a destination of the transmission terminal 10 is selected, Switching the transmission terminal 10 ON / OFF A power switch 109 for transmitting data, a network I / F 111 for transmitting data using the communication network 2 to be described later, a camera 112 for capturing an image of a subject under the control of the CPU 101, and driving of the camera 112. An image sensor I / F 113 to be controlled, a microphone 114 for inputting sound, a speaker 115 for outputting sound, and a sound input / output I for processing input / output of sound signals between the microphone 114 and the speaker 115 according to the control of the CPU 101. / F116, a display I / F 117 that transmits image data to an external display 120 according to the control of the CPU 101, an external device I / F 118 that transmits / receives various data to / from an external device, and various functions of the transmission terminal 10 The alarm lamp 119 that informs you of abnormalities and the above components And it includes an address bus, a data bus, and the like bus line 110 for electrically connecting to.

  Note that the recording medium 106 is detachable from the transmission terminal 10. Further, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used. In addition, the camera 112 includes a solid-state imaging device that converts light into electric charges and digitizes a subject image (video), such as a CCD (Charge Coupled Device) device, a CMOS (Complementary Metal Oxide Semiconductor) device, and the like. . The display 120 is configured using a liquid crystal element, an organic EL element, or the like that displays a subject image, an operation icon, or the like.

  In addition, the transmission terminal program may be recorded in a computer-readable recording medium such as the recording medium 106 and distributed as a file in an installable or executable format.

  FIG. 4 is a hardware configuration diagram of the transmission management system 50 constituting the transmission system 1 according to the present embodiment. The transmission management system 50 includes a CPU 201 that controls the overall operation of the transmission management system 50, a ROM 202 that stores a transmission management program, a RAM 203 that is used as a work area for the CPU 201, and an HD (Hard Disk) that stores various data. 204, an HDD (Hard Disk Drive) 205 that controls reading and writing of various data with respect to the HD 204 according to the control of the CPU 201, and a media drive 207 that controls reading and writing (storage) of data with respect to a recording medium 206 such as a flash memory. A display I / F 208 for displaying various information such as a cursor, menu, window, characters, and images on the display 216, a network I / F 209 for transmitting data using the communication network 2 described later, A keyboard 211 having a plurality of keys for inputting characters, numerical values, various instructions, a mouse 212 for selecting and executing various instructions, selecting a processing target, moving a cursor, etc., and a removable recording medium As an example, a CD-ROM drive 214 that controls reading and writing of data with respect to a CD-ROM (Compact Disc Read Only Memory) 213, and an external device I / F 215 for transmitting and receiving information to and from an external device, A bus line 210 such as an address bus or a data bus is provided for electrically connecting the above components. The display 216 is also an example of a display device of the external input device 40.

  In addition, the transmission management program may be an installable or executable file that is recorded on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213 and distributed. .

  Further, since the external input device 40 has the same hardware configuration as the transmission management system 50, the description thereof is omitted. However, the ROM 202 stores an external input device program for controlling the external input device 40. Also in this case, the program for the external input device is a file in an installable or executable format, and is recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Also good.

  Further, since the relay device 30 has the same hardware configuration as the transmission management system 50, the description thereof is omitted. However, the ROM 202 stores a relay device program for controlling the relay device 30. Also in this case, the relay device program is a file in an installable or executable format, and is recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Good.

  Further, since the program providing system 90 has the same hardware configuration as the transmission management system 50, the description thereof is omitted. However, the ROM 202 stores a program providing system program for controlling the program providing system 90. Also in this case, the program providing system program is a file in an installable or executable format, recorded on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213, and distributed. Also good. The program providing system program may be stored in the HD 204 instead of the ROM 202.

  Further, since the maintenance system 100 has the same hardware configuration as the transmission management system 50, the description thereof is omitted. The maintenance system 100 is a computer for performing maintenance, management, or maintenance of at least one of the transmission terminal 10, the relay device 30, the transmission management system 50, and the program providing system 90. For example, when the maintenance system 100 is installed in the country and the transmission terminal 10, the relay device 30, the transmission management system 50, or the program providing system 90 is installed outside the country, the maintenance system 100 is remotely connected via the communication network 2. In addition, maintenance such as maintenance, management, and maintenance of at least one of the transmission terminal 10, the relay device 30, the transmission management system 50, and the program providing system 90 is performed.

  In addition, the maintenance system 100 does not go through the communication network 2 but includes a model number, a manufacturing number, a sales destination, a maintenance check in at least one of the transmission terminal 10, the relay device 30, the transmission management system 50, and the program providing system 90. Alternatively, maintenance such as failure history management may be performed.

  As another form of the detachable recording medium, it is provided by being stored in a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray Disc. You may comprise.

<< Functional Configuration of Embodiment >>
Next, the functional configuration of the present embodiment will be described with reference to FIGS. 5 to 15, 24, and 25. FIG. 5 is a functional block diagram of the transmission terminal 10, the relay device 30, and the transmission management system 50 that constitute the transmission system 1 of the present embodiment. In the example illustrated in FIG. 5, the transmission terminal 10, the relay device 30, and the transmission management system 50 are connected so that data communication can be performed via the communication network 2. The external input device 40 is connected so that data can be transmitted to and received from the transmission terminal 10. Since the program providing system 90 and the maintenance system 100 shown in FIG. 1 are not directly related to the video conference communication, they are not shown in FIG.

  FIG. 6 is a diagram for explaining the image quality of the image data. 7 is an example of a modified quality management table, FIG. 8 is an example of a relay device management table, FIG. 9 is an example of a terminal authentication management table, FIG. 10 is an example of a terminal management table, and FIG. 11 is an example of a destination list management table. 12 is an example of a session management table, FIG. 13 is an example of an address priority management table, FIG. 14 is an example of a transmission rate priority management table, and FIG. 15 is an example of a quality management table. FIG. 24 is a functional block diagram of the external input device 40, and FIG. 25 is a diagram showing an example of a resolution management table.

<Functional configuration of transmission terminal>
As shown in FIG. 5, the transmission terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14a, an image display control unit 14b, a display image acquisition unit 14c, Audio input unit 15a, audio output unit 15b, final narrowing unit 16, delay detection unit 17, external information transmission / reception unit 18, storage / read processing unit 19, resolution acquisition unit 21a, resolution determination unit 21b And a resolution selecting unit 21c and a resolution changing unit 21d. Each of these functional units is a function or means realized by any one of the constituent elements shown in FIG. 3 operating according to a command from the CPU 101 according to a program stored in the ROM 102. Further, the transmission terminal 10 has a storage unit 1000 constructed by the SSD 105 shown in FIG.

<Functional parts of transmission terminal>
Next, each functional unit of the transmission terminal 10 will be described in detail. The transmission / reception unit 11 of the transmission terminal 10 is realized by the network I / F 111 shown in FIG. 3, and transmits / receives various data to / from other transmission terminals 10, the relay device 30, and the transmission management system 50 via the communication network 2. The operation input receiving unit 12 is realized by the operation button 108 and the power switch 109 shown in FIG. 3, and receives various inputs by the user. For example, when the user turns on the power switch 109 (FIG. 3), the operation input reception unit 12 (FIG. 5) receives the power ON and turns on the power. Further, the operation input receiving unit 12 receives resolution information representing the resolution input by the user.

  The login request unit 13 is realized by a command from the CPU 101 shown in FIG. 3, and requests that the transmission management system 50 request a login from the transmission / reception unit 11 via the communication network 2 when the power ON is received. The login request information shown and the current IP address of the transmission terminal 10 are automatically transmitted.

  The imaging unit 14a is realized by the camera 112 and the imaging element I / F 113 shown in FIG. 3, and generates image data representing an image obtained by imaging a subject. The image display control unit 14b is realized by the display I / F 117 illustrated in FIG. 3, and performs control for rendering the image data and causing the display 120 to display an image represented by the image data. The display image acquisition unit 14 c acquires image data representing an image displayed on the display 120. In the present embodiment, data representing an image captured by the camera 112 is referred to as “image data”. Further, data representing an image displayed on the display 216 of the external input device 40, which will be described in detail later, is referred to as “display data”. The image data and display data are data in a format such as JPEG (Joint Photographic Experts Group), Bitmap, GDI (Graphics Device Interface).

  The voice input unit 15a is realized by the microphone 114 and the voice input / output I / F 116 shown in FIG. 3. The voice input unit 15a receives the voice of the user and converts the voice into a voice signal, thereby outputting voice data related to the voice signal. To do. The audio output unit 15b is realized by the speaker 115 and the audio input / output I / F 116 shown in FIG. 3, and converts an audio signal related to the audio data into audio and outputs the audio.

  The final narrowing unit 16 is a functional unit for performing final narrowing processing for finally narrowing down to one relay device 30 from among the plurality of relay devices 30, and according to the command from the CPU 101 shown in FIG. 16a, the calculation part 16b, and the selection part 16c are implement | achieved. Among these, the measurement unit 16 a measures the reception date and time when the pre-transmission information is received by the transmission / reception unit 11 for each pre-transmission information described later received by the transmission / reception unit 11. For each pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b determines the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The time required from transmission to reception is calculated. The selection unit 16c finally selects one relay device by selecting the relay device 30 to which the pre-transmission information requiring the shortest required time is relayed among the required times calculated by the calculation unit 16b.

  The delay detection unit 17 is realized in accordance with a command from the CPU 101 shown in FIG. 3 and detects a delay time (ms) of image data or audio data sent from another transmission terminal 10 via the relay device 30. The external information transmitting / receiving unit 18 transmits / receives data to / from the external input device 40 via the external device I / F 118 shown in FIG. The storage / reading processing unit 19 is executed by the SSD 105 shown in FIG. 3, and performs processing of storing various data in the storage unit 1000 and reading out various data stored in the storage unit 1000.

  The resolution acquisition unit 21 a acquires the displayable resolution of the display 120 connected to the transmission terminal 10. The resolution determination unit 21b determines whether the displayable resolution acquired by the resolution acquisition unit 21a includes a resolution other than the reference aspect ratio used for image data transmission. The resolution selection unit 21c selects a resolution other than the reference aspect ratio or a resolution of the reference aspect ratio from among the displayable resolutions based on the determination result performed by the resolution determination unit 21b. The resolution changing unit 21d changes the resolution of the display 120 to the resolution represented by the resolution information received by the operation input receiving unit 12. Here, the resolution represents the number of pixels existing per length on the display screen, and includes a vertical resolution and a horizontal resolution.

  The storage unit 1000 includes a terminal ID (Identification) and password for identifying the transmission terminal 10, image data and audio data, a relay device ID for identifying the relay device 30 that transmits various data, an IP address of the destination terminal, and the like. Remembered. The storage unit 1000 stores a display data acquisition program 1451, a display data transmission program 1452, and a resolution conversion program 1453. When these programs are transmitted to the external input device 40 and installed in the external input device 40, the external input device 40 includes a display data acquisition unit 451, a display data transmission unit 452, and a resolution conversion unit 453, which will be described later. become. Details of the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453 will be described in <Functional Units of External Input Device>. The storage unit 1000 includes a resolution storage unit 1005 that stores resolution information indicating the resolution of display data output from the transmission terminal 10.

  The external information transmission / reception unit 18 is an example of a resolution information transmission unit and a display data reception unit, and the image display control unit 14b is an example of a display control unit.

  Further, the terminal ID and the relay device ID described later in this embodiment indicate identification information such as a language, characters, symbols, or various signs used to uniquely identify the transmission terminal 10 and the relay device 30, respectively. Further, the terminal ID and the relay device ID may be identification information in which at least two of the languages, characters, symbols, and various indicia are combined. In the following description, the transmission terminal 10 as a request source requesting the start of a video conference is referred to as a “request source terminal”, and the transmission terminal 10 as a destination as a request destination is described as a “destination terminal”.

<Functional configuration of external input device>
As shown in FIG. 24, the external input device 40 includes a transmission / reception unit 41, a connection detection unit 42, an installation determination unit 43a, a program acquisition unit 43b, a resolution acquisition unit 44, a display data acquisition unit 451, and a display. The data transmission unit 452, the resolution conversion unit 453, the operation input reception unit 46, the display control unit 47, the mount unit 48, and the storage / read processing unit 49 are included. Each of these functional units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. The display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453 are a display data acquisition program 1451, a display data transmission program 1452, and a resolution that are stored in the storage unit 1000 (see FIG. 5) of the transmission terminal 10. This is realized by downloading the conversion program 1453 by the program acquisition unit 43b and installing these programs. Further, the external input device 40 has a storage unit 4000 constructed by the ROM 202, RAM 203 or HDD 205 shown in FIG. Although not specifically shown, the external input device 40 is equipped with an OS (Operating System) such as Windows (registered trademark), and thereby has a function of executing a program when connected to another device. Yes.

<Functional parts of external input device>
Next, each functional unit of the external input device 40 will be described in detail. First, the transmission / reception unit 41 (see FIG. 24) is realized by the network I / F 209 illustrated in FIG. 4, and transmits / receives various data (information) to / from the transmission terminal 10. The transmission / reception unit 41 receives, as information related to the present invention, resolution information (first resolution information) indicating the resolution after the change when the display resolution is changed on the transmission terminal 10 side. The connection detection unit 42 detects that various types of data can be transmitted to and received from the transmission terminal 10 by the external device I / F 215 illustrated in FIG.

  When the connection detection unit 42 detects that the external input device 40 can transmit / receive to / from the transmission terminal 10, the installation determination unit 43a displays the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453 externally. It is determined whether or not it is installed in the input device 40. When the installation determination unit 43a determines that the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453 are not installed in the external input device 40, the program acquisition unit 43b stores the storage unit 1000 of the transmission terminal 10. The display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453 are acquired from (see FIG. 5).

  The resolution acquisition unit 44 acquires resolution information (second resolution information) indicating the resolution of the display 216 of the external input device 40. The operation input receiving unit 46 receives information input by a user operation. The display control unit 47 causes the display 216 to display an image read by a storage / reading processing unit 49 described later. The mount unit 48 mounts the storage unit 1000 of the transmission terminal 10. Accordingly, the external input device 40 can install the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453.

  The display data acquisition unit 451 acquires display data representing an image displayed on the display 216 of the external input device 40. The display data transmission unit 452 transmits the display data acquired by the display data acquisition unit 451 to the transmission terminal 10. The resolution conversion unit 453 includes the resolution (second resolution information) of the display 216 of the external input device 40 and the display 120 of the transmission terminal 10 transmitted from the transmission terminal 10 and stored in the storage unit 4000 of the external input device 40. The resolution of the display data transmitted from the external input device 40 to the transmission terminal 10 is converted on the basis of the resolution (the first resolution state).

  The storage / reading processing unit 49 is executed by the HDD 205 shown in FIG. 4, and performs processing for storing various data in the storage unit 4000 and reading various data stored in the storage unit 4000.

  The transmission / reception unit 41 is an example of a resolution information reception unit, the display data transmission unit 452 is an example of a display data transmission unit, the resolution conversion unit 453 is an example of a resolution conversion unit, and the display control unit 47 is a display control unit. It is an example of a means.

<< Resolution management table >>
A resolution management DB 4001 is constructed in the storage unit 4000, and a resolution management table as shown in FIG. 25 as an example is stored in the resolution management DB 4001. In this resolution management table, information representing the resolution in the vertical direction (height direction) and the resolution in the horizontal direction (width direction) of the display 120 of the transmission terminal 10 received by the transmission / reception unit 41 is stored in association with each other. For example, in the resolution management table shown in FIG. 25, the resolution (width) of the display 120 of the transmission terminal 10 is 1024, and the resolution (height) is 768.

<Functional configuration of relay device>
Next, functions (or means) of the relay device 30 will be described. As shown in FIG. 5, the relay device 30 includes a transmission / reception unit 31, a state detection unit 32, a data quality confirmation unit 33, a change quality management unit 34, a data quality change unit 35, and a storage / read processing unit 39. And have. Each of these functional units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. Further, the relay device 30 has a storage unit 3000 constructed by any of the ROM 202, RAM 203, and HDD 205 shown in FIG.

《Change quality control table》
In the storage unit 3000, a change quality management DB 3001 configured by a change quality management table as shown in FIG. In the change quality management table, the IP address of the transmission terminal 10 as a relay destination of image data and the image quality of the image data relayed by the relay device 30 are associated with the relay destination and managed.

  Here, the image resolution of the image data handled in the transmission system 1 of the present embodiment will be described. Referring to FIG. 6, the horizontal direction consists of 160 pixels and the vertical direction consists of 120 pixels as shown in (a), and the low-resolution image as the base image, and the horizontal direction is 320 as shown in (b). There are a medium resolution image consisting of 240 pixels in the vertical direction and a high resolution image consisting of 640 pixels in the horizontal direction and 480 pixels in the vertical direction, as shown in FIG. Among these, when passing through a narrow band route, low-quality image data consisting only of low-resolution image data serving as a base image is relayed. When the band is relatively wide, medium-quality image data composed of low-resolution image data serving as a base image and medium-resolution image data is relayed. When the bandwidth is very wide, high-quality image data composed of low-resolution image data serving as a base image, medium-resolution image data, and high-resolution image data is relayed. In the modified quality management table shown in FIG. 7, for example, when the relay device 30 relays image data to the destination terminal 10db (see FIG. 1) having the IP address “1.3.2.4”, this The image quality (image quality) of the relayed image data is “high quality”.

<Each functional unit of the relay device>
Next, each functional unit of the relay device 30 will be described in detail. In the following description, in describing each functional unit of the relay device 30, the relationship with main components for realizing each functional unit of the relay device 30 among the components illustrated in FIG. 4 is also described. explain.

  The transmission / reception unit 31 (see FIG. 5) of the relay device 30 is realized by the network I / F 209 shown in FIG. 4, and communicates with other transmission terminals 10, the relay device 30, and the transmission management system 50 via the communication network 2. To send and receive various data. The state detection unit 32 is realized by a command from the CPU 201 illustrated in FIG. 4, and detects the operating state of the relay device 30 having the state detection unit 32. The operating state includes “ON line”, “OFF line”, and “failing” state.

  The data quality confirmation unit 33 is realized by an instruction from the CPU 201 shown in FIG. 4, and searches the change quality management table (see FIG. 7) using the IP address of the destination terminal 10 as a search key, and corresponds to the IP address. The image quality of the relayed image data is confirmed by extracting the image quality of the relayed image data. The change quality management unit 34 is realized by an instruction from the CPU 201 shown in FIG. 4 and changes the contents of the change quality management DB 3001 based on quality information described later transmitted from the transmission management system 50. For example, during a video conference by transmitting / receiving high-quality image data between the request source terminal 10aa having the terminal ID “01aa” and the destination terminal 10db having the terminal ID “01db”. When the request source terminal bb and the destination terminal ca that perform another video conference start the video conference via the communication network 2 and the destination terminal 10db receives a delay in receiving the image data, the relay device 30 Needs to reduce the image quality of the relayed image data from high to medium. In such a case, the content of the change quality management DB 3001 is changed so as to lower the image quality of the image data relayed by the relay device 30 from the high image quality to the medium image quality based on the quality information indicating the medium image quality.

  The data quality changing unit 35 is realized by a command from the CPU 201 shown in FIG. 4 and changes the image quality of the image data sent from the request source terminal 10 based on the contents of the changed change quality management DB 3001. . The storage / reading processing unit 39 is realized by the HDD 205 shown in FIG. 4, and performs processing of storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

<Functional configuration of transmission management system>
Next, functions (or means) of the transmission management system 50 will be described. As shown in FIG. 5, the transmission management system 50 includes a transmission / reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state acquisition unit 55, a narrowing unit 56, a session, A management unit 57, a quality determination unit 58, a storage / read processing unit 59, and a delay time management unit 60 are provided. Each of these functional units is a function or means realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to a program stored in the ROM 202. The transmission management system 50 also includes a storage unit 5000 constructed by the ROM 202, RAM 203, or HDD 205 shown in FIG.

<< Relay device management table >>
In the storage unit 5000, a relay device management DB 5001 configured by a relay device management table as shown in FIG. 8 is constructed. In this relay device management table, for each relay device ID of each relay device 30, the operating status of each relay device 30, the date and time when the transmission management system 50 received the status information indicating the operating status, the IP of the relay device 30 The address and the maximum data transmission rate (Mbps) in the relay device 30 are associated and managed. For example, in the relay device management table shown in FIG. 8, the relay device 30a (see FIG. 1) with the relay device ID “111a” has an “ON line” operating state, and the transmission management system 50 receives the status information. The date and time of the relay device 30a is “13:00 on November 10, 2009”, the IP address of this relay device 30a is “1.2.1.2”, and the maximum data transmission rate in this relay device 30a is It is shown to be 100 Mbps.

《Terminal authentication management table》
Further, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table as shown in FIG. 9 is constructed. In this terminal authentication management table, each password is associated with each terminal ID of all transmission terminals 10 managed by the transmission management system 50 and managed. For example, the terminal authentication management table shown in FIG. 9 indicates that the terminal ID of the transmission terminal 10aa (see FIG. 1) is “01aa” and the password is “aaaa”.

《Terminal management table》
Further, in the storage unit 5000, a terminal management DB 5003 configured by a terminal management table as shown in FIG. 10 is constructed. In this terminal management table, for each terminal ID of each transmission terminal 10, the operating state of each transmission terminal 10, the reception date and time when login request information described later is received by the transmission management system 50, and the IP address of the transmission terminal 10 are associated. Managed. For example, in the terminal management table shown in FIG. 10, the transmission terminal 10aa (see FIG. 1) whose terminal ID is “01aa” is in the “ON line” operating state, and the log-in request information is received by the transmission management system 50. The date and time is “13:40 on November 10, 2009”, and the IP address of the transmission terminal 10aa is “1.2.1.3”.

<< Destination list management table >>
Further, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table as shown in FIG. 11 is constructed. In the destination list management table, all terminal IDs of destination terminals 10 registered as candidates for the destination terminal 10 are managed in association with the terminal ID of the request source terminal 10 that requests the start of the video conference. For example, in the destination list management table shown in FIG. 11, the candidate of the destination terminal 10 that can request the start of the video conference from the request source terminal 10aa (see FIG. 1) whose terminal ID is “01aa” is the terminal ID. The transmission terminal 10ab is “01ab”, the transmission terminal 10ba has a terminal ID “01ba”, and the transmission terminal 10db has a terminal ID “01db”. The candidates for the destination terminal 10 are updated by being added or deleted in response to a request for addition or deletion from the request source terminal 10 to the transmission management system 50.

<Session management table>
In the storage unit 5000, a session management DB 5005 including a session management table as shown in FIG. In this session management table, for each selection session ID used to execute a session for selecting the relay device 30, the relay device ID of the relay device 30, the terminal ID of the request source terminal 10, the terminal ID of the destination terminal 10, The delay time (ms) and the reception date / time are managed in association with each other. Here, the delay time (ms) is a reception delay time when image data is received at the destination terminal 10, and the reception date and time is a delay information indicating the delay time transmitted from the destination terminal 10. The date and time received by the transmission management system 50.

  For example, in the session management table shown in FIG. 12, the relay device 30a (relay device ID “111a”) selected in the session executed using the selection session ID “se1” has the terminal ID “01aa”. It is shown that image data and audio data are relayed between the request source terminal 10aa (see FIG. 1) and the destination terminal 10db whose terminal ID is “01db”. In addition, the destination terminal 10db indicates that the delay time of the image data is “200 (ms)” as of “November 10, 2009, 14:00”.

  When a video conference is performed between the two transmission terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal 10 instead of the destination terminal 10. . However, when a video conference is performed between three or more transmission terminals 10, the reception date and time of the delay information is managed based on the delay information transmitted from the transmission terminal 10 on the reception side of the image data and the audio data. To do.

<< Address priority management table >>
Further, in the storage unit 5000, a priority management DB 5006 configured by an address priority management table as shown in FIG. 13 is constructed. In this address priority management table, in accordance with the difference in the four dot address (Dot Address) portions of the general IPv4 IP addresses, the address priority points are associated and managed so as to increase. The For example, in the address priority management table shown in FIG. 13, when the three values are the same IP address from the top to the bottom of the dot address, the address priority point is “5”. When the two values are the same IP address from the top to the bottom of the dot address, the address priority point is “3”. In this case, whether the value of the lowest dot address is the same is not related to the priority. In the case of IP addresses having the same highest value of dot addresses and different second values from the top, the address priority point is “1”. In this case, whether or not the values of the third and lowest dot addresses from the top are the same is not related to the priority. In the case of IP addresses having different uppermost values of dot addresses, the address priority point is “0”. In this case, whether or not the values of the second, third, and lowest dot addresses from the top are the same does not relate to the priority.

<< Transmission speed priority management table >>
The priority management DB 5006 built in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. In this transmission rate priority management table, the transmission rate priority points are associated and managed in accordance with the value of the maximum data transmission rate (Mbps) in the relay device 30 so as to increase. For example, in the transmission rate priority management table shown in FIG. 14, when the maximum data transmission rate in the relay apparatus 30 is 1000 Mbps or more, the transmission rate priority point is “5”. When the maximum data transmission rate in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission rate priority point is “3”. When the maximum data transmission rate in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission rate priority point is “1”. When the maximum data transmission rate in the relay device 30 is less than 10 Mbps, the transmission rate priority point is “0”.

《Quality control table》
Further, the storage unit 5000 has a quality management DB 5007 configured by a quality management table as shown in FIG. In this quality management table, the image data quality (image quality) to be relayed by the relay device 30 is managed in association with the delay time (ms) of the image data at the request source terminal 10 or the destination terminal 10.

<Each functional part of the transmission management system>
Next, each functional unit of the transmission management system 50 will be described in detail. In the following description, in describing each functional unit of the transmission management system 50, the main components for realizing the functional units of the transmission management system 50 among the components illustrated in FIG. The relationship is also explained.

  First, the transmission / reception unit 51 (see FIG. 5) is executed by the network I / F 209 shown in FIG. Send and receive data. The terminal authentication unit 52 searches the terminal authentication management DB 5002 in the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and is identical to the terminal authentication management DB 5002. Terminal authentication is performed by determining whether the terminal ID and password are managed. The state management unit 53 stores the terminal ID of the request source terminal 10, the operation status of the request source terminal 10, the transmission management in the terminal management table (FIG. 10) in order to manage the operation status of the request source terminal 10 that has requested login. The system 50 stores and manages the reception date and time when the login request information is received and the IP address of the request source terminal 10 in association with each other.

  The terminal extraction unit 54 searches the destination list management table (FIG. 11) using the terminal ID of the request source terminal 10 that requested the login as a key, and the terminal IDs of candidate destination terminals 10 that can communicate with the request source terminal 10. Is read out, the terminal ID is extracted. Further, the terminal extraction unit 54 searches the destination list management table (FIG. 11) using the terminal ID of the request source terminal 10 that has requested login as a key, and sets the terminal ID of the request source terminal 10 as a candidate for the destination terminal 10. The terminal IDs of other registered request source terminals 10 are also extracted.

  The terminal status acquisition unit 55 searches the terminal management table (FIG. 10) using the candidate terminal ID of the destination terminal 10 extracted by the terminal extraction unit 54 as a search key, and for each terminal ID extracted by the terminal extraction unit 54. Read the operating status. Thereby, the terminal state acquisition unit 55 can acquire the operation state of candidates of the destination terminal 10 that can communicate with the request source terminal 10 that has requested login. Further, the terminal state acquisition unit 55 searches the terminal management table (FIG. 10) using the terminal ID extracted by the terminal extraction unit 54 as a search key, and also acquires the operating state of the request source terminal 10 that has requested login.

  The narrowing-down unit 56 includes a selection session ID generation unit 56a, a terminal IP address extraction unit 56b, a primary selection unit 56c, and a priority determination unit 56d. Accordingly, the narrowing-down unit 56 performs a primary narrowing-down process before the final narrowing-down process in order to support a final narrowing-down process that finally narrows down from a plurality of relay apparatuses 30 to one relay apparatus 30. That is, the selection session ID generation unit 56 a generates a selection session ID used for executing a session for selecting the relay device 30. The terminal IP address extraction unit 56b, based on the terminal ID of the request source terminal 10 and the terminal ID of the destination terminal 10 included in the start request information sent from the request source terminal 10, is a terminal management table (FIG. 10), the IP address of each corresponding transmission terminal 10 is extracted. The primary selection unit 56c selects the relay device ID of the relay device 30 whose operation state is “ON line” from among the relay devices 30 managed in the relay device management table (FIG. 8), thereby relaying. The device 30 is selected.

  Further, the primary selection unit 56c searches the relay device management table (FIG. 8) based on the IP address of the request source terminal 10 and the IP address of the destination terminal 10 extracted by the terminal IP address extraction unit 56b. Then, for each dot address of the IP address of the selected relay device 30, it is investigated whether it is the same as or different from each dot address in each IP address of the request source terminal 10 and the destination terminal 10. Furthermore, the primary selection unit 56c, for each relay device, the upper two relays having higher points among the higher points with respect to the transmission terminal 10 at the address priority points and the integration points obtained by integrating the transmission speed priority points. By selecting the device 30, the relay device 30 is further selected.

  In the present embodiment, the top two relay devices 30 with high points are selected. However, the present invention is not limited to this. If one relay device 30 can be narrowed down as many as possible, the top points with high points are selected. Three or more relay devices 30 may be selected.

  The priority determination unit 56d refers to the priority management table (see FIG. 13) and determines an address priority point for each relay device 30 investigated by the primary selection unit 56c. Further, the priority determination unit 56d searches the priority management table (see FIG. 14) based on the maximum data transmission rate of each relay device 30 managed in the relay device management table (see FIG. 8). The transmission speed priority point is determined for each relay device 30 narrowed down by the primary selection unit 56c.

  The session management unit 57 stores the selection session ID generated by the selection session ID generation unit 56a, the terminal ID of the request source terminal, and the terminal of the destination terminal in the session management DB 5005 (session management table of FIG. 12) of the storage unit 5000. Store and manage IDs in association with each other. In addition, the session management unit 57 selects the relay device ID of the relay device 30 that is finally selected by the selection unit 16c of the transmission terminal 10 for each selection session ID with respect to the session management table (FIG. 12). Remember and manage.

  The quality determination unit 58 searches the quality management table (FIG. 15) using the delay time as a search key and extracts the image quality of the corresponding image data, thereby determining the image quality of the image data to be relayed to the relay device 30. . The storage / reading processing unit 59 is executed by the HDD 205 shown in FIG. 4, and performs processing for storing various data in the storage unit 5000 and reading out various data stored in the storage unit 5000. The delay time management unit 60 extracts the corresponding terminal ID by searching the terminal management table (FIG. 10) using the IP address of the destination terminal 10 as a search key, and in the session management table (FIG. 12), The delay time indicated by the delay information is stored and managed in the field portion of the delay time in the record including the extracted terminal ID.

<< Processing and Operation of Embodiment >>
Next, various processes and operations performed in the transmission system 1 of the present embodiment will be described with reference to FIGS. 16 to 23 and FIGS. 26 to 33. FIG. 16 is a sequence diagram illustrating an example of a process for managing state information representing the operating state of each relay device 30 transmitted from each relay device 30 to the transmission management system 50. FIG. 17 is a sequence diagram illustrating an example of a preparation stage process for starting communication between the transmission terminals 10. FIG. 18 is a sequence diagram illustrating an example of processing in which the transmission management system 50 narrows down the relay devices 30. FIG. 19 is a flowchart illustrating an example of processing for narrowing down the relay devices 30. FIG. 20 is a diagram illustrating a calculation state of priority points when the transmission management system 50 performs a process of narrowing down the relay devices 30. FIG. 21 is a sequence diagram illustrating an example of processing in which the transmission terminal 10 selects the relay device 30. FIG. 22 is a flowchart illustrating an example of processing for selecting the relay device 30. FIG. 23 is a sequence diagram illustrating an example of processing for transmitting and receiving image data and audio data between the transmission terminals 10.

  FIG. 26 is a sequence diagram illustrating an example of a process of displaying the display data displayed by the external input device 40 on the transmission terminal 10 that is a conference partner. FIG. 27 is a diagram illustrating an example of a screen displayed by the external input device 40. FIG. 28 is a diagram illustrating an example of a screen on which the transmission terminal 10 displays image data and display data. FIG. 29 is a sequence diagram illustrating an example of processing related to the resolution change performed by the transmission terminal 10 and the resolution conversion performed by the external input device 40 (when an uncompressed format is used). FIG. 30 is a flowchart illustrating an example of processing for converting the resolution. FIG. 31 is a flowchart illustrating an example of resolution control (including resolution change) performed by the transmission terminal 10. FIG. 32 is a diagram showing an example of an auxiliary information display screen for supplementarily explaining the determination process performed in step S102 of FIG. FIG. 33 is a sequence diagram illustrating another example of processing related to the resolution change performed by the transmission terminal 10 and the resolution conversion performed by the external input device 40 (when the compression format is used).

  First, with reference to FIG. 16, processing for managing state information representing the operating state of each relay device 30 transmitted from each relay device 30 to the transmission management system 50 will be described. First, in each relay device 30 (30a to 30d), the state detection unit 32 (FIG. 5) periodically detects the operating state of the relay device 30 as its own device (S1-1 to S1-4). . The transmission / reception unit 31 of each relay device 30 periodically transmits each status information to the transmission management system 50 via the communication network 2 so that the operation state of each relay device 30 is managed in real time on the transmission management system 50 side. Transmit (steps S2-1 to S2-4). Each state information includes the relay device ID for each relay device 30 and the operating state detected by the state detection unit 32 of the relay device 30 related to each relay device ID. In the present embodiment, the relay devices 30a, 30b, and 30d are operating normally and are “ON line”, while the relay device 30c is operating, but the relay operation of the relay device 30c is not performed. A case where a problem occurs in the program to be executed and the line is “OFF line” is shown.

  Next, in the transmission management system 50, the transmission / reception unit 51 receives each status information sent from each of the relay devices 30a to 30d, and via the storage / read processing unit 59, the relay device management DB 5001 (see FIG. The status information is stored and managed for each relay device ID in the relay device management table 8 (steps S3-1 to S3-4). As a result, the operating status of “ON line”, “OFF line”, or “failing” is stored and managed for each relay device ID in the relay device management table (FIG. 8). At this time, the reception date and time when the status information is received by the transmission management system 50 is also stored and managed for each relay device ID. If status information is not sent from the relay device 30, the field portion of the operating state and the field portion of the reception date and time in each record of the relay device management table (FIG. 8) are blank or the previous time The operating status and date / time of reception are maintained and managed.

  Next, processing in a preparatory stage before starting communication between the transmission terminal 10aa and the transmission terminal 10db will be described using FIG. First, when the user turns on the power switch 109 (FIG. 3), the operation input receiving unit 12 (FIG. 5) receives the power ON and turns on the power (step S21). Next, the log-in request unit 13 automatically transmits log-in request information indicating a log-in request from the transmission / reception unit 11 to the transmission management system 50 via the communication network 2 when the power-on is received (step S22). ). This log-in request information includes a terminal ID and a password for identifying the transmission terminal 10aa that is the own device as the request source. The terminal ID and password are data read from the storage unit 1000 via the storage / read processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the transmission terminal 10aa to the transmission management system 50, the transmission management system 50 that is the reception side can grasp the IP address of the transmission terminal 10aa that is the transmission side.

  Next, the terminal authentication unit 52 (FIG. 5) of the transmission management system 50 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and the terminal authentication management of the storage unit 5000. The DB 5002 (terminal authentication management table in FIG. 9) is searched, and terminal authentication is performed by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). When it is determined by the terminal authentication unit 52 that the login request is from the transmission terminal 10 having a valid use authority because the same terminal ID and password are managed, the state management unit 53 The DB 5003 (terminal management table of FIG. 10) stores the terminal ID of the transmission terminal 10aa, the operating state, the reception date and time when the login request information is received, and the IP address of the transmission terminal 10aa in association with each other (step S24). Accordingly, in the terminal management table (FIG. 10), the transmission terminal ID “01aa”, the operation state “ON line”, the reception date and time “2009.11.10.13:40”, and the terminal IP address “1.2. 1.3 "is managed in association with each other.

  Next, the transmission / reception unit 51 of the transmission management system 50 transmits the authentication result information indicating the authentication result obtained by the terminal authentication unit 52 to the request source terminal 10aa that made the login request via the communication network 2. (Step S25). In the present embodiment, the case where the terminal authentication unit 52 determines that the terminal has a valid use authority will be described below.

  The terminal extraction unit 54 of the transmission management system 50 may search the destination list management table (FIG. 11) using the terminal ID “01aa” of the request source terminal 10aa that requested the login as a search key, and communicate with the request source terminal 10aa. Extraction is performed by reading out candidate terminal IDs of possible destination terminals 10 (step S26). Here, the terminal IDs “01ab”, “01ba”, and “01db” of the destination terminals 10ab, 10ba, and 10db corresponding to the terminal ID “01aa” of the request source terminal 10aa are extracted.

  Next, the terminal state acquisition unit 55 uses the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminal 10 extracted by the terminal extraction unit 54 as a search key, as a terminal management table (FIG. 10). ) And the operation state (“OFF line”, “ON line”, “ON line”) for each terminal ID extracted by the terminal extraction unit 54 is read out, so that the transmission terminals (10ab, 10ba, 10db) Each operating state is acquired (step S27).

  Next, the transmission / reception unit 51 uses the terminal ID (“01ab”, “01ba”, “01db”) as the search key used in step S27 and the operating status (10ab, 10ba, 10db) of the corresponding destination terminal (10ab, 10ba, 10db). The destination state information including “OFF line”, “ON line”, “ON line”) is transmitted to the request source terminal 10aa via the communication network 2 (step S28). As a result, the request source terminal 10aa transmits the current operating states ("OFF line", "" of the transmission terminals (10ab, 10ba, 10db) that are candidates for the destination terminal 10 that can communicate with the request source terminal 10aa. "ON line", "ON line").

  Further, the terminal extraction unit 54 of the transmission management system 50 searches the destination list management table (FIG. 11) using the terminal ID “01aa” of the request source terminal 10aa that requested the login as a search key, and the terminal of the request source terminal 10aa. The terminal IDs of other request source terminals 10 in which the ID “01aa” is registered as a candidate for the destination terminal 10 are extracted (step S29). In the destination list management table shown in FIG. 11, the terminal IDs of the other request source terminals 10 to be extracted are “01ab”, “01ba”, and “01db”.

  Next, the terminal status acquisition unit 55 of the transmission management system 50 searches the terminal management table (FIG. 10) using the terminal ID “01aa” of the request source terminal 10aa that requested the login as a search key, and the request source terminal 10aa. Is acquired (step S30).

  Next, the transmission / reception unit 51 uses the terminal management table (FIG. 10) among the transmission terminals (10ab, 10ba, 10db) of the terminal IDs (“01ab”, “01ba”, and “01db”) extracted in step S29. Destination status information including the terminal ID “01aa” of the request source terminal 10aa acquired in step S30 and the operating status “ON line” is transmitted to the transmission terminals (10ba, 10db) whose operating status is “ON line”. Transmit (steps S31-1, S31-2). When the transmission / reception unit 51 transmits the destination status information to the transmission terminals (10ba, 10db), it is managed in the terminal management table (FIG. 10) based on each terminal ID (“01ba”, “01db”). Refers to the IP address of the terminal. Thereby, the terminal ID “01aa” and the operation state “ON line” of the request source terminal 10aa that requested the login are transmitted to each of the other destination terminals (10db and 10ba) that can communicate with the request source terminal 10aa that requested the login. Can be told.

  On the other hand, also in the other transmission terminals 10, when the user turns on the power switch 109 (FIG. 3) (step S21), the operation input receiving unit 12 (FIG. 5) receives the power ON, and the steps S22 to S31- Since the same processing as the processing of 1 and 31-2 is performed, the description thereof is omitted.

  Next, processing for narrowing down the relay devices 30 will be described with reference to FIG. In the present embodiment, the request source terminal 10aa determines at least one of the transmission terminals 10ba and 10db whose operation state is “ON line” based on the destination state information received in step S28 among the transmission terminals 10 as destination candidates. Can communicate with one side. Therefore, in the following description, a case where the user of the request source terminal 10aa has selected to start communication with the destination terminal 10db will be described.

  First, when the user presses the operation button 108 (FIG. 3) to select the transmission terminal 10db, the operation input reception unit 12 (FIG. 5) of the transmission terminal 10aa receives a selection for setting the destination as the transmission terminal 10db (step S41). ). Next, the transmission / reception unit 11 of the transmission terminal 10aa includes start request information indicating that communication is to be started, including the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db. It transmits to the system 50 (step S42). As a result, the transmission / reception unit 51 of the transmission management system 50 receives the start request information and grasps the IP address “1.2.1.3” of the request source terminal 10aa that is the transmission source. Then, based on the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db included in the start request information, the state management unit 53 stores information on the terminal management table (FIG. 10) of the terminal management DB 5007. Then, the field portion of the operating state of the record including the terminal ID “01aa” and the terminal ID “01db” is changed to “busy” (step S43). In this state, the request source terminal 10aa and the destination terminal 10db do not start communication (call) but are in a call state, and when another transmission terminal 10 tries to communicate with the request source terminal 10aa or the destination terminal 10db, so-called A sound or display indicating the busy state is output.

  Next, a process of executing a session for selecting the relay device 30 will be described by steps S44 to S48 (FIG. 18) and steps S61-1 to 66 (see FIG. 21). First, the selection session ID generating unit 56a (FIG. 5) of the transmission management system 50 generates a selection session ID used for executing a session for selecting the relay device 30 (step S44). Then, the session management unit 57 stores the selection session ID “se1” generated in step S44, the terminal ID “01aa” of the request source terminal 10aa, and the destination terminal 10db in the session management table (FIG. 12) of the storage unit 5000. The terminal ID “01db” is stored and managed in association with each other (step S45).

  Next, the narrowing down unit 56 performs primary narrowing down of the relay device 30 for relaying communication between the request source terminal 10aa and the destination terminal 10db based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006. (Step S46).

  Here, with reference to FIG. 19, the “relay device narrowing down” process in step S46 will be described in more detail. First, the terminal IP address extracting unit 56b is based on the terminal ID “01aa” of the request source terminal 10aa and the terminal ID “01db” of the destination terminal 10db included in the start communication information transmitted from the request source terminal 10aa. The terminal management table (FIG. 10) is searched to extract the IP address (“1.2.1.3”, “1.3.2.4”) of the corresponding transmission terminal (10aa, 10db) (step S46-1). Next, the primary selection unit 56c selects the relay device (30a, 30b, 30d) that is “ON line” among the operating states of each relay device 30 managed in the relay device management table (FIG. 8). Each relay device ID (111a, 111b, 111d) is selected (step S46-2). Further, the primary selection unit 56c is based on the IP address “1.2.1.3” of the request source terminal 10aa and the IP address “1.3.2.4” of the destination terminal 10db extracted in step S46-1. By searching the relay device management table (FIG. 8), each IP address (“1.2.1.2.”, “ For each dot address of “1.2.2.2” and “1.3.2.2”), the IP addresses (“1.2.1.3”, “1” of the request source terminal 10aa and the destination terminal 10db). .3.2.4 "), it is checked whether it is the same as or different from each dot address (step S46-3).

  Next, the priority determining unit 56d refers to the priority management table (FIG. 13), and determines an address priority point for each relay device (30a, 30b, 30d) investigated in step 46-3. (Step S46-4). An example of the result of this determination process is as shown in FIG. FIG. 20 is a diagram illustrating a calculation state of priority points when the relay device 30 performs the narrowing-down process. In the example of FIG. 20, an address priority point, a transmission speed priority point, and an integration point are shown for each relay device ID. The address priority points include a point for each requesting terminal 10aa and a point for the destination terminal 10db of each relay device 30. The integration point is the sum of the higher one of the two address priority points and the transmission speed priority point.

  In the present embodiment, the IP address “1.2.1.2” of the relay device 30a is “same. Same. Same. Different” with respect to the IP address “1.2.1.3” of the request source terminal 10aa. Therefore, as shown in FIG. 20, the address priority point is “5”. The IP address “1.2.1.2” of the relay device 30a is “same.different.different.different” with respect to the IP address “1.3.2.4” of the destination terminal 10db. The priority point is “1”. Further, since the IP address “1.2.2.2” of the relay device 30b is “same. Same. Different. Different” with respect to the IP address “1.2.1.3” of the request source terminal 10aa. The address priority point is “3”. Further, since the IP address “1.2.2.2” of the relay device 30b is “same. Different. Same. Different” with respect to the IP address “1.3.2.4” of the destination terminal 10db, the address The priority point is “1”. Further, the IP address “1.3.2.2” of the relay device 30d is “same.different.different.different” with respect to the IP address “1.2.1.3” of the request source terminal 10aa. The address priority point is “1”. The IP address “1.3.2.2” of the relay device 30d is “same. Same. Same. Different” with respect to the IP address “1.3.2.4” of the destination terminal 10db. The priority point is “5”.

  Next, returning to FIG. 19, the priority determination unit 56d determines the priority management table (FIG. 14) based on the maximum data transmission rate of each relay device 30 managed by the relay device management table (FIG. 8). Is searched for, for each relay device (30a, 30b, 30d) narrowed down by the first narrowing-down process in step S46-2, a point of transmission speed priority is determined (step S46-5). In this embodiment, since the maximum data transmission rate of the relay device 30a is 100 (Mbps) (see FIG. 8), the transmission rate priority is 3 points with reference to the transmission rate priority (FIG. 14). Similarly, since the maximum data transmission rate of the relay device 30b is calculated to be 1000 (Mbps), the transmission rate priority is 5 points. Moreover, since the maximum data transmission rate of the relay device 30d is calculated to be 10 (Mbps), the transmission rate priority is 1 point.

  Next, for each relay device (30a, 30b, 30d), the primary selection unit 56c integrates the higher point of the transmission terminals (10aa, 10db) and the transmission speed priority point at the address priority point. Among the integrated points, the top two relay devices 30 with the highest points are selected (step 46-6). In the present embodiment, as shown in FIG. 20, since the relay device IDs (111a, 111b, 111d) have the integration points “8”, “8”, and “6”, respectively, the relay with the relay device ID “111a” is performed. The device 30a and the relay device 30b with the relay device ID “111b” are selected.

  When the “relay device narrowing down” process in step S46 is completed, the transmission / reception unit 51 (FIG. 5) of the transmission management system 50 sets the number of narrowed relay devices 30 to the destination terminal 10db via the communication network 2. The relay device narrowing information for transmission is transmitted (step S47). The relay device narrowing information includes the number “2” of relay devices 30 narrowed down in step S46, the terminal ID “01aa” of the request source terminal 10aa, and the selection session ID “se1”. As a result, the transmission terminal 10db grasps the number of relay devices 30 and the transmission terminal 10 from which a request to start a video conference is made in the execution of the session with the selection session ID “se1”. And the IP address “1.1.1.2” of the transmission management system 50 that is the transmission source of the relay device narrowing information can be grasped.

  Next, the transmission terminal 10db transmits reception completion information indicating that the reception of the relay device narrowing information is completed from the transmission / reception unit 11 to the transmission management system 50 via the communication network 2 (step S48). This reception completion information includes the session ID “se1”. As a result, the transmission management system 50 indicates that the transmission of the number of relay devices executed with the session ID “se1” has been completed, and the IP address “1.3.2.4” of the destination terminal 10db that is the transmission source. I can grasp it.

  Next, a process in which the destination terminal 10db selects the relay device 30 will be described with reference to FIG. First, before starting a video conference, the transmission management system 50 transmits pre-relay request information indicating that a relay is requested in advance to each of the relay devices (30a, 30b) narrowed down in step S46. (Steps S61-1, 61-2). This pre-relay request information includes the session ID “se1”, the IP address “01aa” of the request source terminal 10aa, and the IP address “01db” of the destination terminal 10db. As a result, the relay device (30a, 30b) can know which selection session it belongs to, what the request source terminal 10 is, and what the destination terminal 10 is, and a prior relay request. It is possible to grasp the IP address “1.1.1.2” of the transmission management system 50 that is the information transmission source.

  Next, each of the relay devices (30a, 30b) receives a video conference from the transmission / reception unit 31 (FIG. 5) via the communication network 2 to the request source terminal 10aa grasped in steps S61-1, 61-2. Prior to the start of the transmission, the transmission request information indicating that the transmission information including ping (Packet Internet Groper) described later is transmitted to each relay apparatus (30a, 30b) as the own apparatus is transmitted (step S62-). 1, 62-2). This pre-transmission information includes the session ID “se1”. Accordingly, the request source terminal 10aa grasps that the transmission information is transmitted to each relay device (30a, 30b) in the selection process of the relay device 30 that is executed with the session ID “se1”, and the prior transmission. The IP address (“1.2.1.2”, “1.2.2.2”) of the relay device (30a, 30b) that is the transmission source of the request information can be grasped.

  Note that the transmission management system 50 does not directly notify the request source terminal 10aa of the IP address of the destination terminal 10db, but instead sends the IP address of the destination terminal 10db to the relay device 30a as in step S61-1. It is notified that each relay terminal 30b requests the requester terminal 10aa to transmit the pre-transmission request information to the own apparatus (relay apparatus 30b) as in step 61-2. In order to ensure security, the IP address of the other transmission terminal 10 is not notified.

  Next, the request source terminal 10aa transmits the pre-transmission information from the transmission / reception unit 11 to the relay device (30a, 30b) via the communication network 2 (steps S63-1, 63-2). Prior to transmission of image data and audio data, this pre-transmission information is transmitted to the destination terminal 10db via each relay device (30a, 30b) instead of these image data and audio data, so that the request source terminal This is information used to measure the time required from transmission of 10aa to reception of the destination terminal 10db. In addition, the advance transmission information includes the ping for confirming that the request source terminal 10aa, the relay devices (30a, 30b) and the destination terminal 10db are connected so as to be communicable, and the advance transmission information from the request source terminal 10aa. The transmitted transmission date and time and the session ID “se1” are included. As a result, each relay device (30a, 30b) can grasp that the pre-transmission information has been sent in the execution of the session with the selection session ID “se1”, and can also request that the pre-transmission information is transmitted. The IP address “1.2.1.3” of the original terminal 10aa can be grasped.

  Next, each relay device (30a, 30b) sets the IP address “1.3.2.4” of the destination terminal 10db included in the prior relay request information received in steps S61-1, 61-2. On the other hand, the prior transmission information is relayed (steps S64-1 and 64-2). As a result, the destination terminal 10db can recognize that the pre-transmission information has been sent in the execution of the session with the session ID “se1”, and the relay device (30a) that is the transmission source (relay source) of the pre-transmission information , 30b) ("1.2.1.2", "1.2.2.2").

  Next, the final narrowing unit 16 (FIG. 5) of the destination terminal 10db finally narrows down to one relay device 30 that relays image data and audio data in the video conference based on the pre-transmission information (step S65). .

  Here, with reference to FIG. 5 and FIG. 22, the “relay device selection” process in step S65 will be described in more detail. First, the measurement unit 16a (FIG. 5) of the final narrowing unit 16 sets the reception date and time when the transmission / reception unit 11 of the transmission terminal 10db receives each pre-transmission information relayed by each relay device (30a, 30b). Measure (Step S65-1). Next, the calculation unit 16b determines, for each pre-transmission information whose reception time is measured, from the transmission of each pre-transmission information based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information. The time required until reception is calculated (step S65-2). Next, the selection unit 16c determines whether or not the number of pieces of pre-transmission information corresponding to the number “2” of relay devices 30 scheduled to be relayed has been received in the execution of the session with the session ID “se1” (Step S1). S65-3). And when not having received all (in the case of NO at step S65-3), the selection unit 16c has passed a predetermined time (here 1 minute) after receiving the pre-transmission information at the transmission terminal 10db. Is determined (step S65-4). If the predetermined time has not elapsed (NO in step S65-4), the process returns to step S65-1. On the other hand, if YES in step 65-3 (when all are received) or YES in step S65-4 (when a predetermined time has elapsed), the selection unit 16c has been calculated by the calculation unit 16b so far. One relay device 30 that relays the pre-transmission information that required the shortest required time is selected (step S65-5). In the embodiment shown in FIG. 21, the relay device 30a selects that the pre-transmission information relayed by the relay device 30a is shorter than the pre-transmission information relayed by the relay device 30b. An example is shown.

  In the above embodiment, the relay device 30a is narrowed down on the destination terminal 10db side, but it is needless to say that the processing mode is not limited thereto. For example, when the destination terminal 10db transmits all the required time information indicating the required time from transmission to reception of the pre-transmission information to the request source terminal 10aa or the transmission management system 50, the request source terminal 10aa side or the transmission management system On the 50th side, one relay device 30a may be finally narrowed down.

  Next, returning to FIG. 21, the destination terminal 10db transmits selection information indicating that the relay device 30a has been selected from the transmission / reception unit 11 (FIG. 5) to the transmission management system 50 via the communication network 2 (step). S66). This selection information includes the session ID “se1” and the relay device ID “111a” of the selected relay device 30a. As a result, the transmission management system 50 can grasp that the relay device 30a has been selected in the execution of the session with the session ID “se1”, and the IP address “1.3” of the transmission terminal 10db that is the transmission source of the selection information. .2.4 ".

  Next, the session management unit 57 of the transmission management system 50 adds the above-mentioned final field to the relay device ID field portion of the record including the selection session ID “se1” in the session management table (FIG. 12) of the session management DB 5005. The relay device ID “111a” of the relay device 30a selected as one is stored and managed (step S67-1), and the transmission / reception unit 51 has the relay device ID “111a” and the IP address “1” of the destination terminal 10db. ..3.2.4 "is transmitted to the request source terminal 10aa (step S67-21). Then, the transmission / reception unit 51 of the transmission management system 50 transmits relay start request information indicating a request to start relaying to the relay device 30a via the communication network 2 (step S68). This relay start request information includes the IP addresses (“1.2.1.3” and “1.3.2.4”) of the request source terminal 10aa and the destination terminal 10db to be relayed. As a result, the relay device 30a establishes a session for communicating the three image data of low resolution, medium resolution and high resolution and the audio data between the transmission terminal 10aa and the transmission terminal 10db (step S69). Thereby, each transmission terminal (10aa, 10db) can start a video conference.

  In the above-described embodiment, the transmission management system 50 transmits the relay device narrowing information to the destination terminal 10db in step S47 (FIG. 18), and through steps S48 to S64-1, 64-2, Although the relay device selection processing (step S65) is performed on the destination terminal 10db side, it is needless to say that the processing mode is not limited thereto. For example, in step S47, the transmission management system 50 transmits the relay device narrowing information to the request source terminal 10aa. Thereafter, in steps S64-1 and 64-2, the transmission source and the reception source of each information are requested. The original terminal 10aa and the destination terminal 10db may be interchanged. As a result, the request source terminal 10aa can perform the relay device selection process instead of step S65, and can also transmit the selection information instead of step S66.

  Next, with reference to FIG. 5 and FIG. 23, processing for transmitting and receiving image data and audio data in order to conduct a video conference between the request source terminal 10aa and the destination terminal 10db will be described. First, the request source terminal 10aa transmits the image data of the subject imaged by the imaging unit 14a and the audio data of the audio input by the audio input unit 15a from the transmission / reception unit 11 to the relay device 30a via the communication network 2. (Step S81). In the present embodiment, high-quality image data consisting of three low resolution, medium resolution, and high resolution shown in FIG. 6 and audio data are transmitted. Thereby, in the relay device 30a, the transmission / reception unit 31 receives the image data and the audio data of the three resolutions. Then, the data quality confirmation unit 33 searches the change quality management table (FIG. 7) using the IP address “1.3.2.4” of the destination terminal 10db as a search key, and relays an image corresponding to the IP address. By extracting the image quality of the data, the image quality of the image data to be relayed is confirmed (step S82). In the present embodiment, the image quality of the confirmed image data is “high image quality” and is the same as the image quality of the image data received by the transmission / reception unit 31. The voice data is transferred to the destination terminal 10db (step S83). As a result, the destination terminal 10db receives the image data and the audio data at the transmission / reception unit 11, the image display control unit 14b displays an image based on the image data on the display 120, and the audio output unit 15b is based on the audio data. Audio can be output.

  Next, the delay detection unit 17 of the transmission terminal 10db detects the reception delay time of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every second) (step S84). In this embodiment, the case where the delay time is 200 (ms) will be described below.

  The transmission / reception unit 11 of the destination terminal 10db transmits delay information indicating the delay time “200 (ms)” to the transmission management system 50 via the communication network 2 (step S85). Thereby, the transmission management system 50 can grasp the delay time and the IP address “1.3.2.4” of the transmission terminal 10db that is the transmission source of the delay information.

  Next, the delay time management unit 60 of the transmission management system 50 searches the terminal management table (FIG. 10) using the IP address “1.3.2.4” of the destination terminal 10db as a search key, and corresponds to the IP address. Terminal ID “01db” to be extracted, and the delay indicated by the delay information in the field portion of the delay time in the record of the terminal ID “01db” on the session management table (FIG. 12) of the session management DB 5005 The time “200 (ms)” is stored and managed (step S86).

  Next, the quality determination unit 58 of the transmission management system 50 searches the quality management table (FIG. 15) using the delay time “200 (ms)” as a search key, and the image quality “of image data corresponding to the delay time”. The “medium image quality” is extracted, and the image quality is determined as “medium image quality” (step S87).

  Next, the transmitting / receiving unit 51 uses the relay device ID “111a” associated with the terminal ID “01db” in the session management table of the session management table (FIG. 12) as a search key, and the relay device management table (FIG. 8). ) And the IP address “1.2.1.2” of the relay device 30a corresponding to the relay device ID is extracted (step S88). Then, the transmission / reception unit 51 transmits quality information indicating the image quality “medium image quality” of the image data determined in step S87 to the relay device 30a via the communication network 2 (step S89). This quality information includes the IP address “1.3.2.4” of the destination terminal 10db used as the search key in step S86. As a result, in the relay device 30a, the change quality management unit 34 stores the IP address “1.3.2.4” of the transmission terminal 10 (destination terminal 10db in this case) of the transmission destination in the change quality management table (FIG. 7). ”And the image quality“ medium image quality ”of the relayed image data are stored and managed in association with each other (step S90).

  Next, the transmission terminal 10aa transmits the high-quality image data including the low image quality, the medium image quality, and the high image quality, and the audio data to the relay device 30a in the same manner as the processing performed in step S81. (Step S91). As a result, in the relay device 30a, the data quality confirmation unit 33 uses the IP address “1.3.2.4” of the destination terminal 10db as a search key, as in the process performed in step S82. (FIG. 7) is searched, and the image quality “medium image quality” of the image data to be relayed corresponding to the IP address is extracted to confirm the image quality of the image data to be relayed (step S92). In this embodiment, since the image quality of the confirmed image data is “medium image quality” and lower than the image quality “high image quality” of the image data received by the transmission / reception unit 31, the data quality changing unit 35 The image quality of the image data is changed by suppressing the image quality from “high image quality” to “medium image quality” (step S93). Then, the transmitting / receiving unit 31 transmits the image data in which the image quality of the image data is changed to “medium image quality” and the audio data in which the sound quality of the audio is not changed to the transmission terminal 10 db via the communication network 2. (Step S94). As described above, when there is a reception delay on the destination terminal 10db side that receives the image data, the relay device 30a changes the quality of the image so that the person participating in the video conference does not feel uncomfortable. can do.

  Next, with reference to FIG. 26, processing for transmitting display data representing an image displayed on the display 216 of the external input device 40 to the other transmission terminal 10 and displaying it on the display 120 after the relay device 30 is determined will be described. While explaining. Here, a case where information displayed by the external input device 40aa connected to the transmission terminal 10aa is displayed on the transmission terminal 10db that is the destination terminal will be described as an example.

  When the relay device 30 is determined as described above, the relay device ID “111a” and the IP address “1.3.2.4” of the destination terminal 10db transmitted by the transmission management system 50 in step S67-21 are transmitted. The transmission / reception part 11 of terminal 10aa receives. Then, the storage / read processing unit 19 stores the received relay device ID “111a” and IP address “1.3.2.4” in the storage unit 1000 (step S67-22).

  When the external input device 40aa is connected to the transmission terminal 10aa, the connection detection unit 42 (FIG. 24) of the external input device 40aa detects (detects) the connection (step S70). When the connection detection unit 42 detects the connection between the external input device 40aa and the transmission terminal 10aa, the installation determination unit 43a includes the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453. (Step S71). Specifically, the installation determination unit 43a determines whether a display data acquisition program 1451 (see FIG. 5), a display data transmission program 1452, and a resolution conversion program 1453 are installed in the external input device 40aa. When each program is installed, it is determined that the external input device 40aa includes a functional unit corresponding to each program.

  On the other hand, when the installation determination unit 43a determines that the external input device 40aa does not include any of the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453, the mount unit 48 transmits the transmission. The storage unit 1000 of the terminal 10aa is mounted (step S72). Next, the program acquisition unit 43b is a functional unit that is not included in the external input device 40aa among the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453 stored in the storage unit 1000 of the transmission terminal 10. A corresponding program is acquired and installed (step S73). As a result, the external input device 40aa includes a display data acquisition unit 451, a display data transmission unit 452, and a resolution conversion unit 453.

  When the processing in step S73 is completed, the transmission / reception unit 41 requests permission to execute each of the above programs, that is, permission to operate the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453. Is transmitted to the transmission terminal 10aa (step S74). And when the transmission / reception part 41 receives the information showing the permission with respect to a request | requirement from transmission terminal 10aa, the display data acquisition part 451 will acquire the display data showing the image currently displayed on the display 216 (step S75). Next, the resolution conversion unit 453 converts the resolution of the display data acquired by the display data acquisition unit 451, and the display data transmission unit 452 converts the resolution to the external information transmission / reception unit 18 of the transmission terminal 10aa. The displayed display data is transmitted (step S76). Details of the processing for converting the resolution of the display data by the resolution conversion unit 453 will be described later. Here, the process in which the resolution conversion unit 453 converts the resolution is different from the process performed based on the operation status of the relay device 30 and the communication speed between the transmission terminal 10 and the relay device 30. The resolution is not converted depending on the operation status of the relay device 30 and the communication speed between the transmission terminal 10 and the relay device 30.

  When the external information transmitting / receiving unit 18 of the transmission terminal 10aa as the transmission source receives display data from the external input device 40aa, the storage / reading processing unit 19 uses the relay device ID “111a” stored in the storage unit 1000, the destination, The IP address “1.3.2.4” of the transmission terminal 10db is read (step S77). Then, for the relay device 30 indicated by the relay device ID “111a” read in step S74, the transmission / reception unit 11 displays the display data whose resolution has been converted and the IP address “1” of the transmission terminal 10db as the destination. ..3.2.4 "is transmitted (step S78). When the relay device 30 receives the display data transmitted from the transmission terminal 10aa in step S78, the image quality of the display data is changed based on the IP address “1.3.2.4” of the destination transmission terminal 10db ( In step S79, display data is transmitted to the transmission terminal 10db (step S80). The details of the process in step S77 are the same as the process for changing the quality of the audio data and the image data (steps S81 to S94 in FIG. 23), and thus the description thereof is omitted. When the transmission / reception unit 11 of the transmission terminal 10db receives display data transmitted from the relay device 30, the image display control unit 14b causes the display 120 to display an image represented by the received display data. For example, in the example of the screen display shown in FIG. 28, the image displayed by the external input device 10aa is displayed on the left side of the screen, and the image is captured by the imaging unit 14a of the transmission terminal 10aa on the upper right side of the screen for transmission / reception. The image data transmitted by the unit 11 is displayed. In the lower right portion of the screen, image data captured by the imaging unit 14a of the transmission terminal 10db is displayed.

  Next, FIG. 29 shows processing for converting the resolution of display data transmitted from the external input device 40 to the transmission terminal 10 based on the resolution of the display 120 of the transmission terminal 10 and the resolution of the display 216 of the external input device 40. This will be described with reference to a sequence diagram. In the example of FIG. 29, display data is transmitted in an uncompressed format.

  29, first, when the transmission / reception unit 41 of the external input device 40 transmits a request for resolution information of the display 120 of the transmission terminal 10 (step S901), the resolution acquisition unit 21a of the transmission terminal 10 The resolution information representing the resolution of the display 120 of the transmission terminal 10 stored in 1005 is acquired (step S902). When the resolution acquisition unit 21a acquires the resolution information in step S902, the external information transmitting / receiving unit 18 transmits the acquired resolution information to the external input device 40 (step S903). In the external input device 40, when the transmission / reception unit 41 receives the resolution information of the display 120 of the transmission terminal 10, the storage / reading processing unit 49 stores the received resolution information in the resolution management table of the resolution management DB 4001 ( Step S904).

  Next, the display data acquisition unit 451 of the external input device 40 acquires display data representing an image displayed on the display 216 by the display control unit 47 (step S905). Further, the resolution acquisition unit 44 acquires the resolution information of the display 216 (step S906). The resolution conversion unit 453 then displays the resolution represented by the resolution information of the display 120 of the transmission terminal 10 stored in the resolution management DB 4001 and the resolution represented by the resolution information of the display 216 of the external input device 40 acquired in step S906. Based on the above, the resolution of the display data to be transmitted to the transmission terminal 10 is converted (step S907).

  Details of the “resolution conversion” processing performed by the resolution conversion unit 453 in step S907 will be described with reference to FIG. First, for convenience of explanation and description, among the resolution information of the display 216 of the external input device 40 acquired in step S906, the resolution in the vertical direction (height (H) direction) is “H40”, and the horizontal direction (width ( The resolution of (W) direction) is “W40”. Similarly, in the resolution information of the display 120 of the transmission terminal 10 stored in the resolution management DB 4001, the vertical resolution is “H10” and the horizontal resolution is “W10”. Based on these resolutions H40, W40, H10, and W10, the resolution conversion unit 453 sets the vertical resolution of the display data to “H ′” and the horizontal resolution of the display data to “W” as described below. Convert to ''.

  Referring to FIG. 30, first, the resolution conversion unit 453 determines whether H40 ≦ H10 and W40 ≦ W10 are satisfied (step S9071). If it is determined that H40 ≦ H10 and W40 ≦ W10 (YES in step S9071), the resolution conversion unit 453 converts the vertical resolution H ′ and the horizontal resolution W ′ of the display data, H ′ = H40 and W ′ = W40 are set, respectively, and the resolution is not converted (step S9072).

  On the other hand, if it is determined that H40 ≦ H10 and W40 ≦ W10 are not satisfied (NO in step S9071), the resolution conversion unit 453 determines whether (W40> W10 and H40 ≦ H10) or (W40> W10, and It is determined whether or not H40> H10 and H10 ≧ W10) (step S9073).

  When it is determined that (W40> W10 and H40 ≦ H10) or (W40> W10 and H40> H10 and H10 ≧ W10) (YES in step S9073), the resolution conversion unit 453 The horizontal resolution W ′ and the vertical resolution H ′ of the display data are converted into W ′ = W10 and H ′ = H40 * W10 / W40, respectively (step S9074). Here, W10 / W40 represents the compression rate in the width (W) direction.

  On the other hand, when it is determined that (W40> W10 and H40 ≦ H10) or (W40> W10 and H40> H10 and H10 ≧ W10) is not satisfied (NO in step S9073), the resolution conversion unit 453 The horizontal resolution W ′ and the vertical resolution H ′ of the display data are converted into W ′ = W40 * H10 / H40 and H ′ = H10, respectively. (Step S9075). Here, H10 / H40 represents the compression rate in the height (H) direction.

  Note that the case where it is determined in step S9073 that (W40> W10 and H40> H10 and H10 ≧ W10) is not satisfied is, for example, that the vertical resolution of the display 216 of the external input device 40 is the display of the transmission terminal 10. When the horizontal resolution of the display 216 of the external input device 40 is higher than the vertical resolution of 120 and equal to or lower than the horizontal resolution of the display 120 of the transmission terminal 10 (H40> H10 and W40 ≦ W10). ). Alternatively, the horizontal resolution and the vertical resolution of the display 216 of the external input device 40 are higher than the horizontal resolution and the vertical resolution of the display 120 of the transmission terminal 10, respectively, and the display 120 of the transmission terminal 10. In the case where the vertical resolution is lower than the horizontal resolution (W40> W10, H40> H10, and H10 <W10).

  That is, in step S907, when the vertical resolution of the display 216 of the external input device 40 is lower than the vertical resolution of the display 120 of the transmission terminal 10, the resolution conversion unit 453 transmits the vertical resolution of the display data. Conversion to the vertical resolution of the display 120 of the terminal 10 (H ′ = H10). Then, the resolution conversion unit 453 converts the horizontal resolution of the display data so as to have the same aspect ratio as the resolution of the display 120 of the transmission terminal 10 (W ′ = W40 * H10 / H40).

  Also, when the horizontal resolution of the display 120 of the transmission terminal 10 is lower than the horizontal resolution of the display 216 of the external input device 40, the resolution conversion unit 453 converts the horizontal resolution of the display data to the display of the transmission terminal 10. The horizontal resolution is converted to 120 (W ′ = W10). Then, the resolution conversion unit 453 converts the vertical resolution of the display data so that the aspect ratio of the resolution is the same as that of the display 120 of the transmission terminal 10 (H ′ = H40 * W10 / W40).

  Returning to FIG. 29, when the resolution of the display data is converted by the resolution conversion unit 453, the display data transmission unit 452 transmits the converted display data to the transmission terminal 10 (step S908).

  When the external information transmission / reception unit 18 of the transmission terminal 10 receives the display data transmitted from the external input device 40 (step S908), the image display control unit 14b renders the received display data and displays it on the display 120 (step S908). S909). Then, the display image acquisition unit 14c acquires data representing the image displayed on the display 120 by the image display control unit 14b in step S909 (step S910). Then, the transmission / reception unit 11 transmits the data acquired by the image acquisition unit 14c to the relay device 30 (step S911).

  Further, when the operation input receiving unit 12 receives the resolution information, the resolution changing unit 21d changes the resolution of the display 120 of the transmission terminal 10 (step S912). This “resolution change” is performed by the “resolution change” process in steps S105, S109, S117, and S122 in the process flow of FIG.

  When the resolution is changed, the resolution acquisition unit 21a acquires resolution information indicating the changed resolution (step S902), and the external information transmitting / receiving unit 18 transmits the resolution information to the external input device 40 (step S903). ). Subsequent processes are the same as those performed in step S904 and subsequent steps, and a description thereof will be omitted.

  Next, resolution control (including resolution change) performed by the transmission terminal 10 will be described with reference to the flowchart illustrated in FIG.

  In FIG. 31, when the process is started by turning on the power of the transmission terminal 10, the resolution acquisition unit 21a (FIG. 5) accesses the display 120 and acquires a displayable resolution (step S101).

  Next, the resolution determination unit 21b uses a resolution other than the reference aspect ratio (for example, 16: 9, resolution is 1280 × 720) used for image data communication between the transmission terminal 10 and the relay device 30 as the displayable resolution. Is determined (step S102). This determination is performed because it is desired to select a resolution other than the reference aspect ratio and add a segment area for displaying auxiliary information if possible. FIG. 32 (a) shows an example of the screen in that case. A segment area SR is added to the lower end of the screen, and the auxiliary information AX (error message and operation button) is displayed in the segment area SR. Show. On the other hand, in the example shown in FIG. 32B, the segment area is not added, and the auxiliary information AX is displayed at the lower end of the screen.

  When a resolution other than the reference aspect ratio is included (YES in step S102), the resolution selection unit 21c selects a resolution based on a predetermined reference from resolutions other than the reference aspect ratio (step S103). As a predetermined standard, the highest resolution (width x height having the maximum value) is selected with priority, or the resolution of a specific aspect ratio (for example, 8: 5) is selected with priority. Or selecting a resolution suitable for the character size, the number of characters, etc. of the auxiliary information, or allowing the user to define a resolution to be prioritized (for example, WXGA (1280 × 800) or XGA (1024 × 768)). A criterion can be used.

  Next, an image processing unit (not shown), which is a function of the image display control unit 14b, expands / contracts the image data generated internally with the reference aspect ratio to the selected resolution and adds a segment area. (Step S104). Specifically, this image processing unit performs the following processing. First, the image before conversion is expanded and contracted symmetrically so as to match the width after conversion. When the width before conversion and after conversion are the same, expansion / contraction is not performed (the enlargement ratio or reduction ratio is 1 time). Next, the height of the image after expansion / contraction is checked, and a segment area is added to the image after expansion / contraction so as to be the height after conversion. In the example shown in FIG. 32A, a segment area SR is added to the lower end of the screen. The position where the segment area is added is not limited to the lower end, but may be added to the upper end, or a segment area having a half height may be added to the upper and lower ends of the screen.

  Next, the resolution changing unit 21d changes the resolution of the display 120 to be the selected resolution (step S105). By this processing, the display 120 is displayed with the changed resolution.

Next, a flag setting unit (not shown), which is a function of the image display control unit 14b, sets the flag “overlap” to “false” (step S106). This flag is referenced when the image display control unit 14b detects an error or the like and performs display control of auxiliary information different from the currently displayed image data. The value of “overlap” is “true” when displaying the auxiliary information so as to overlap the image data (in the case of FIG. 32B), and when controlling so as not to overlap (in the case of FIG. 32A) ) Has a value of "false".

  On the other hand, when the resolution other than the reference aspect ratio is not included (NO in step S102), the resolution selection unit 21c selects a resolution based on a predetermined reference from the reference aspect ratio (step S107). The predetermined standard is the same as that described in step S103.

  Next, an image processing unit (not shown), which is a function of the image display control unit 14b, expands / contracts the image data generated internally with the reference aspect ratio to the selected resolution (step S108). In this case, since the expansion / contraction is the same aspect ratio, no blank area is generated in any direction of the width and height of the image data, and no segment area is added (see the screen display example in FIG. 32B). .

  Next, the resolution changing unit 21d changes the resolution of the display 120 to be the selected resolution (step S109). By this processing, the display 120 is displayed with the changed resolution. Next, a flag setting unit (not shown), which is a function of the image display control unit 14b, sets the flag “overlap” to “true” (step S110).

  Up to this point, the initial process is performed when the transmission terminal 10 is turned on. After this, a loop process is performed to cope with the case where the display 120 is replaced after the power is turned on.

  The resolution acquisition unit 21a accesses the display 120 and acquires the currently displayed resolution (step S111). Next, the resolution acquisition unit 21a accesses the display 120 to acquire a displayable resolution (step S112).

  Next, the resolution determination unit 21b determines whether the displayable resolution includes a resolution other than a reference aspect ratio (for example, 16: 9) used for image data communication between the transmission terminal 10 and the relay device 30. Is determined (step S113). When a resolution other than the reference aspect ratio is included (YES in step S113), the resolution selection unit 21c selects a resolution based on a predetermined reference from resolutions other than the reference aspect ratio (step S114). The predetermined standard is the same as that described in step S103.

  Next, the resolution determination unit 21b determines whether the currently displayed resolution is different from the selected resolution (step S115). If the currently displayed resolution is different from the selected resolution (YES in step S115), an image processing unit (not shown), which is a function of the image display control unit 14b, is internally generated with a reference aspect ratio. The image data is expanded or contracted to the selected resolution, and a segment area is added (step S116).

  Next, the resolution changing unit 21d changes the resolution of the display 120 to be the selected resolution (step S117). By this processing, the display 120 is displayed with the changed resolution. Next, a flag setting unit (not shown), which is a function of the image display control unit 14b, sets the flag “overlap” to “false” (step S118). On the other hand, when the currently displayed resolution and the selected resolution are the same (NO in step S115), the processes in steps S116 to S118 are not performed. This is because it is not necessary to change the resolution.

  On the other hand, when the resolution other than the reference aspect ratio is not included (NO in step S113), the resolution selection unit 21c selects the resolution based on a predetermined reference from the reference aspect ratio (step S119). The predetermined standard is the same as that described in step S103.

  Next, the resolution determination unit 21b determines whether the currently displayed resolution is different from the selected resolution (step S120). If the currently displayed resolution is different from the selected resolution (YES in step S120), an image processing unit (not shown), which is a function of the image display control unit 14b, is internally generated with a reference aspect ratio. The image data is expanded or contracted to the selected resolution (step S121).

  Next, the resolution changing unit 21d changes the resolution of the display 120 to be the selected resolution (step S122). By this processing, the display 120 is displayed with the changed resolution. Next, a flag setting unit (not shown) which is a function of the image display control unit 14b sets the flag “overlap” to “true” (step S123). On the other hand, when the currently displayed resolution is the same as the selected resolution (NO in step S120), the processes in steps S121 to S123 are not performed. This is because it is not necessary to change the resolution.

  The above processing is repeated while the program is running. Therefore, even when the display is reconnected to another display 120 having a different resolution, the processing in FIG. 31 is continued. This repetitive processing corresponds to processing for periodically monitoring the change in the resolution of the display 120, but it is also possible to execute the processing by introducing an event mechanism well known in the Windows (registered trademark) program. . In this case, when the resolution is changed by being connected to another display 120, it is registered when the program is started so that an event notification is made. Thereafter, every time this event occurs, the image display control unit 14b may execute the processing from step S111 in FIG.

<< Main effects of embodiment >>
As described above, according to the transmission system 1 according to the present embodiment, when the resolution of the display 120 of the transmission terminal 10 is changed, the external input device 40 receives the changed resolution information and receives the received resolution information. The resolution of the display data to be transmitted to the transmission terminal 10 is converted based on the resolution information after the change and the resolution information of the display 216 of the own device. That is, the external input device 40 performs resolution conversion of the external input device 40 triggered by the change in the resolution of the transmission terminal 10 and transmits the display data after the resolution conversion to the transmission terminal 10. Thereby, the problem as seen in the above-mentioned prior art (patent document 1) can be solved. That is, when the external input device 40 is connected to the transmission terminal 10 and the display data of the external input device 40 is transmitted to the transmission terminal 10, the user does not need to set the resolution conversion of the external input device 40. Can be reduced or substantially eliminated.

  In addition, since the display data whose resolution has been converted on the external input device 40 side is transmitted to the transmission terminal 10 in response to the resolution change on the transmission terminal 10 side as described above, on the transmission terminal 10 side, There is no need to perform processing for converting the resolution of the display data received from the external input device 40 to a resolution suitable for the display 120 of the transmission terminal 10. Thereby, the processing load of the transmission terminal 10 can be reduced.

  Further, since the external input device 40 is configured to transmit display data to the transmission terminal 10 that communicates with the other transmission terminals 10, the external input device 40 is connected to the transmission terminal 10 on the other side on the own device 10 side. There is no need to manage the destination of the device 40. This makes it easier to manage destinations.

  Even when the external input device 40 does not include the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453, the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program are transmitted from the transmission terminal 10. 1453 is received and installed so that each of the functional units 451, 452 and 453 is provided, so that display data representing an image displayed on the display 216 of the external input device 40 is transmitted to the transmission terminal. can do.

  Further, even if the LAN 2 environment such as the IP address of the relay device 30 in the communication network 2 can be obtained, it is difficult to obtain the Internet 2i environment. Therefore, in the present embodiment, first, the available environment information is narrowed down to two or more of the plurality of relay devices 30 that relay the image data and the voice data, and then the image data and the voice data are transmitted between the plurality of transmission terminals 10. Prior to actual transmission / reception, prior transmission information is transmitted / received instead of image data and audio data. As a result, it is possible to narrow down to one relay device 30 that can actually relay the pre-transmission information earliest.

  That is, by selecting the relay device 30 to which the upper two or more IP addresses close to any of the IP addresses of the transmission terminals 10 are respectively assigned, two or more candidates for the relay device 30 to be finally used are selected. Can leave. Thereby, by transmitting and receiving the pre-transmission information between the transmission terminal 10 as the request source and the transmission terminal 10 as the destination via each relay device 30 as a candidate, Among the relay devices 30, it is possible to narrow down the relay devices 30 that relay the pre-transmission information with the shortest time required for transmission / reception. This makes it possible to transmit and receive high-quality image data or audio data to the maximum extent in the current communication network 2 environment.

  In the present embodiment, when narrowing down the relay devices 30, not only the relay device 30 having an IP address close to the IP address of the transmission terminal 10 that conducts the video conference is preferentially selected, but the maximum data in each relay device 30 is also selected. Two or more relay apparatuses 30 are selected in consideration of the transmission speed. As a result, the candidates for the relay device 30 in accordance with the actual environment of the communication network 2 can be narrowed down. Further, when narrowing down the relay devices 30, the candidates for the relay devices 30 that match the actual network 2 environment can be narrowed down because the operation states are narrowed down from the relay devices 30 that are ON lines.

  In this embodiment, display data is transmitted using the Bitmap format (uncompressed format). However, the cause of image degradation is that the display data after resolution conversion is reduced (FIG. 30). Step S9074 or S9075). For this reason, it is possible to suppress display deterioration (image quality deterioration) when sharing display data, such as making fine characters easy to read.

<< Other Embodiments >>
In the embodiment shown in FIG. 29, the case where display data transmitted from the external input device 40 to the transmission terminal 10 and further transmitted from the transmission terminal 10 to the relay device 30 is transmitted in the Bitmap format (uncompressed format) has been described as an example. However, it goes without saying that the present invention is not limited to such a transmission format. For example, display data may be transmitted using a compression format such as JPEG or GIF.

  Hereinafter, a processing example in the case of using the compression format will be described with reference to FIG. The processing flow shown in FIG. 33 has processing (steps S907-A, S908-A, and S910-A) related to compression and decompression of display data added to the processing flow shown in FIG. It is different in point. Since processing performed in steps other than these added steps is the same as in the case of FIG. 29, description thereof is omitted.

  In FIG. 33, after the resolution conversion unit 453 of the external input device 40 converts the resolution of the display data (step S907), a predetermined compression (encoding) process is performed on the display data after the resolution conversion (step S907). S907-A). In the present embodiment, the display data acquisition unit 451 performs this compression processing.

  On the other hand, in the transmission terminal 10, after the external information transmitting / receiving unit 18 receives the compressed display data after resolution conversion transmitted from the external input device 40 (step S908), the received compressed display data is decompressed (decoded). This is performed (step S908-A). In this embodiment, the decompression process is performed by the display image acquisition unit 14c.

  Next, after the image display control unit 14b renders the decompressed (decoded) display data and displays it on the display 120 (step S909), the display image acquisition unit 14c displays the display displayed on the display 120. Data is acquired (step S910). The display data (uncompressed data) acquired here is divided into two systems. The display image acquisition unit 14c performs predetermined compression (encoding) on the one display data (step S910-A). Then, the transmission / reception unit 11 transmits the compressed display data acquired by the display image acquisition unit 14c to the relay device 30 via the communication network 2 (step S911).

  The compressed display data is further received by another transmission terminal 10 (FIG. 1) via the relay device 30. The transmission terminal 10 that has received the compressed display data renders the display data after the image display control unit 14b performs a predetermined decompression (decoding) process as described above, and displays the display data on the display 120. The screen display data can be shared with the transmission terminal 10 as the transmission source. That is, by transmitting the display data of the external input device 40 to the own terminal (transmission terminal 10) and the other transmission terminals 10, the display data can be shared between the transmission terminals 10 at each base.

  On the other hand, the other display data of the display data (uncompressed data) acquired by the display image acquisition unit 14c in step S910 and divided into two systems is subjected to resolution change processing in step S912. Subsequent processing is the same as the processing described in relation to FIG. 29 described above, and thus description thereof is omitted.

  As described above, when the display data converted in resolution by the external input device 40 is converted into a compressed format and transmitted (transmission terminal 10 → relay device 30 → other transmission terminal 10), it is uncompressed as in the Bitmap format. Compared with the case where the format is used (FIG. 29), the transmission capacity can be reduced, so the communication load can be reduced.

<< Supplement of Embodiment >>
In the above embodiment, the case where the program corresponding to the functional unit not provided in the external input device 40 is acquired from the transmission terminal 10 and the program is installed (steps S72 and S73 in FIG. 26) has been described as an example. Of course, the means for realizing the functional unit that the external input device 40 does not have is not limited to this. For example, it may be recorded in a storage medium as an installer-type program for separate distribution, and the user of the external input device 40 may be installed using this storage medium.

  Further, the external input device 40 may include any one of the display data acquisition unit 451, the display data transmission unit 452, and the resolution conversion unit 453 in advance. Accordingly, the external input device 40 is a program corresponding to a function that the external input device 40 has in advance among the display data acquisition program 1451, the display data transmission program 1452, and the resolution conversion program 1453 stored on the transmission terminal 10 side. Can be omitted. As a result, the load on the transmission terminal 10 and the external input device 40 can be reduced.

  In the above embodiment, the transmission terminal 10, the relay device 30, and the transmission management system 50 are connected to each other via the communication network 2. However, the transmission terminal 10, the relay device 30, and the transmission management system 50 are connected to the communication network. May be included. In addition, the transmission terminal 10 transmits and receives various data to and from other transmission terminals 10 via the relay device 30, but various types of data are transmitted between the plurality of transmission terminals 10 without using the relay device 30. Data may be directly transmitted and received.

  Further, the transmission management system 50 and the program providing system 90 in the above embodiment may be constructed by a single computer, or constructed by a plurality of computers arbitrarily assigned by dividing each unit (function or means). May be. In addition, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may be transmitted by dividing it into a plurality of modules. It may be transmitted. Furthermore, when the program providing system 90 is constructed by a plurality of computers, a plurality of modules may be divided and transmitted from each computer.

  In addition, the transmission terminal program, the relay device program, and the transmission management program in the above embodiment, the HD 204 storing the program, and the program providing system 90 including the HD 204 are The program product is used when the transmission terminal program, the relay device program, and the transmission management program are provided to users or the like domestically or abroad.

  In the above embodiment, as an example of the image quality of the image data relayed by the relay device 30, the change quality management table (FIG. 7) and the quality management table (FIG. 15) are used. However, it is needless to say that the present invention is not limited to this embodiment. For example, management may be performed by paying attention to the depth of image quality of image data, the sampling frequency of audio data, the bit length of audio data, and the like. In the above relay device management table (FIG. 8), terminal management table (FIG. 10), and session management table (FIG. 12), the reception date and time are managed, but the present invention is not limited to this. It is sufficient to manage at least the reception time.

  In the above embodiment, the relay device management table (FIG. 8) manages the IP address of the relay device, and the terminal management table (FIG. 10) manages the IP address of the transmission terminal. However, the present invention is not limited to this. Instead, if the relay device specifying information for specifying the relay device 30 on the communication network 2 or the terminal specifying information for specifying the transmission terminal 10 on the communication network 2, each FQDN (Fully Qualified Domain Name) is used. ) May be managed. In this case, an IP address corresponding to the FQDN is acquired by a known DNS (Domain Name System) server. Not only “relay device specifying information for specifying the relay device 30 in the communication network 2” but also “relay device connection destination information indicating the connection destination to the relay device 30 on the communication network 2” or “communication network” 2 may be expressed as “relay device destination information indicating a destination to the relay device 30 on the network 2”. Similarly, not only “terminal specifying information for specifying the transmission terminal 10 in the communication network 2” but also “terminal connection destination information indicating a connection destination to the transmission terminal 10 on the communication network 2” or “communication network 2”. It may be expressed as “terminal destination information indicating the destination to the transmission terminal 10”.

In the above-described embodiment, the case of the video conference system has been described as an example of the transmission system 1. However, the present invention is not limited to this, and may be a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone. Good. Further, the transmission system 1 may be a car navigation system. In this case, for example, one of the transmission terminals 10 corresponds to a car navigation device mounted on a car, and the other of the transmission terminals 10 is mounted on a management terminal or management server of a management center that manages car navigation, or another car. It corresponds to the car navigation device that is used.

  Also, in the process of acquiring display data by the external input device 40 (step S905 in FIG. 29), not the entire screen but only the portion displayed in the area (window) in the screen as described below is acquired. You may make it do. FIG. 27 shows an example of a screen displayed by the external input device 40. In this example, an area (window) (a) and an area (b) are displayed on the screen. Of these, the area to be shared with the transmission terminal 10 on the other side is indicated by operating the pointer indicated by (c) in FIG. 27 with the mouse 212 (FIG. 4). The display data for the area (a) pointed to at this time is acquired by the display data acquisition unit 451.

  In the above embodiment, the case where the display 216 is provided in the external input device 40 has been described. However, the present invention is not limited to this, and the display 216 may be a separate device from the external input device 40.

1: Transmission system 2: Communication network 10 (10aa,..., 10db): Transmission terminal 11: Transmission / reception unit 14b: Image display control unit (display control means)
18: External information transmitter / receiver (resolution information transmitting means, display data receiving means)
21a: Resolution acquisition unit (resolution acquisition means)
21b: Resolution determination unit (resolution determination means)
21c: Resolution selection unit (resolution selection means)
21d: Resolution changing unit (resolution changing means)
30 (30a, ..., 30d): Relay device 40 (40aa, ..., 40db): External input device 41: Transmitter / receiver (resolution information receiving means)
43a: Installation determination unit 43b: Program acquisition unit 44: Resolution acquisition unit 47: Display control unit (display control means)
50: Transmission management system 120, 216: Display (display device)
451: Display data acquisition unit 452: Display data transmission unit (display data transmission means)
453: Resolution converter (resolution converter)
1000, 3000, 4000, 5000: storage unit

Japanese Patent No. 4738628

Claims (13)

A transmission terminal that is communicably connected to another transmission terminal and that is connected to a display device that can be switched to another display device, and that outputs image data for display to the display device, and display data to the transmission terminal A transmission system comprising an external input device for transmitting
The transmission terminal is
Resolution acquisition means for acquiring the displayable resolution and the currently displayed resolution of the another display device when the display device is connected to the other display device;
When the display device is switched to the other display device, the resolution selected from the displayable resolutions of the other display device acquired by the resolution acquisition means, and the other display device is currently A resolution changing means for changing the resolution of the other display device to the selected resolution when the displayed resolution is different;
Resolution information transmitting means for transmitting first resolution information representing the resolution after the change to the external input device when the resolution changing means changes the resolution;
Have
The external input device is:
Resolution information receiving means for receiving the first resolution information transmitted from the transmission terminal;
Based on the received first resolution information and second resolution information indicating the resolution of the display device of the external input device, the resolution of the display data is adapted to the other display device of the transmission terminal. Resolution conversion means for converting to resolution;
Display data transmitting means for transmitting the display data, the resolution of which has been converted by the resolution converting means, to the transmission terminal so as to be transmitted from the transmission terminal to the other transmission terminal;
A transmission system comprising:   The transmission system according to claim 1, wherein the display data transmission unit of the external input device transmits the display data whose resolution has been converted to the transmission terminal in an uncompressed format.   The external input device further includes display control means for converting the display data whose resolution has been converted into a compressed format, and the display data transmitting means transmits the display data converted into the compressed format to the transmission terminal. The transmission system according to claim 1, wherein: A relay device interposed between the transmission terminal and the other transmission terminal;
The transmission terminal transmits the display data converted from the resolution transmitted from the external input device to the other transmission terminal via the relay device,
The said other transmission terminal renders the display data in which the said resolution was transmitted transmitted from the said transmission terminal, and displays it on the display apparatus of the said other transmission terminal. The transmission system according to claim 1. The transmission terminal is
Resolution determination means for determining whether or not the displayable resolution acquired by the resolution acquisition means includes a resolution other than a predetermined reference aspect ratio of image data;
When a resolution other than the reference aspect ratio is included based on a result of determination performed by the resolution determination unit, a resolution other than the reference aspect ratio is selected from the displayable resolutions, and the resolution other than the reference aspect ratio is selected. Resolution selection means for selecting the resolution of the reference aspect ratio,
The transmission system according to any one of claims 1 to 4, characterized by comprising: The transmission terminal is
Display data receiving means for receiving display data converted from the resolution transmitted from the external input device;
Display control means for displaying an image on the display device of the transmission terminal based on the display data received by the display data receiving means;
The transmission system according to any one of claims 1 to 4, characterized by comprising: An external input device that transmits display data to a transmission terminal connected to a display device that is communicably connected to another transmission terminal and can be switched to another display device,
When the display device is switched to the other display device, the resolution that the other display device can display and the resolution that the other display device currently displays are acquired and acquired in the transmission terminal. When the resolution selected from the displayable resolutions is different from the acquired resolution currently displayed on the other display device, the resolution of the other display device is changed to the selected resolution. Resolution information receiving means for receiving first resolution information representing the changed resolution from the transmission terminal,
Based on the received first resolution information and second resolution information indicating the resolution of the display device of the external input device, the resolution of the display data is adapted to the other display device of the transmission terminal. Resolution conversion means for converting to resolution;
Display data transmitting means for transmitting the display data, the resolution of which has been converted by the resolution converting means, to the transmission terminal so as to be transmitted from the transmission terminal to the other transmission terminal;
An external input device comprising:   The external input device according to claim 7, wherein the display data transmission unit transmits the display data whose resolution has been converted to the transmission terminal in an uncompressed format.   It further has a display control means for converting the display data whose resolution is converted into a compressed format, and the display data transmitting means transmits the display data converted into the compressed format to the transmission terminal. The external input device according to claim 7. The resolution conversion means includes
The vertical resolution represented by the second resolution information is equal to or lower than the vertical resolution represented by the first resolution information, and the horizontal resolution represented by the second resolution information is the first resolution. 10. The method according to claim 7, wherein when the resolution is equal to or less than a horizontal resolution represented by the information, the resolution of the display data is converted into a resolution represented by the second resolution information. The external input device described. The resolution conversion means includes
The horizontal resolution represented by the second resolution information is higher than the horizontal resolution represented by the first resolution information, and the vertical resolution represented by the second resolution information is the first resolution. If the vertical resolution represented by the information is below, or
The horizontal resolution represented by the second resolution information is higher than the horizontal resolution represented by the first resolution information, and the vertical resolution represented by the second resolution information is the first resolution. When the resolution in the vertical direction represented by the information is higher than the resolution in the vertical direction represented by the first resolution information, the horizontal direction of the display data is displayed. 10. The external input device according to claim 7, wherein the resolution is converted into a horizontal resolution represented by the first resolution information. 11. The resolution conversion means includes
The vertical resolution represented by the second resolution information is higher than the vertical resolution represented by the first resolution information, and the horizontal resolution represented by the second resolution information is the first resolution. If the horizontal resolution represented by the information is below, or
The horizontal resolution represented by the second resolution information is higher than the horizontal resolution represented by the first resolution information, and the vertical resolution represented by the second resolution information is the first resolution. When the vertical resolution represented by the first resolution information is lower than the horizontal resolution represented by the first resolution information, the vertical direction of the display data is higher than the vertical resolution represented by the information. The external input device according to claim 7, wherein the resolution is converted into a vertical resolution represented by the first resolution information. A display resolution conversion program for execution by an external input device that transmits display data to a transmission terminal connected to a display device that is communicably connected to another transmission terminal and can be switched to another display device. There,
When the display device is switched to the other display device, the resolution that the other display device can display and the resolution that the other display device currently displays are acquired and acquired in the transmission terminal. When the resolution selected from the displayable resolutions is different from the acquired resolution currently displayed on the other display device, the resolution of the other display device is changed to the selected resolution. A resolution information receiving step for receiving first resolution information representing the changed resolution from the transmission terminal,
Based on the received first resolution information and second resolution information indicating the resolution of the display device of the external input device, the resolution of the display data is adapted to the other display device of the transmission terminal. A resolution conversion step for converting to a resolution;
A display data transmission step of transmitting display data whose resolution has been converted in the resolution conversion step to the transmission terminal in order to transmit the display data from the transmission terminal to the other transmission terminal;
A display resolution conversion program for causing the external input device to execute.

Images