JP2018037806A - Transmission management system, transmission management method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically determine a language of a text to be displayed in a transmission terminal as a language to be used by a user of the transmission terminal.SOLUTION: A transmission management system according to an embodiment is the transmission management system to determine a translation language for translating voice data to be transmitted via a communication network between multiple transmission terminals includes: terminal management means for managing areas where the transmission terminals are installed for each piece of terminal identification information for identifying the transmission terminals; language management means for managing languages set for each area; and language determination means for determining a language set in an area where the transmission terminal to be a destination of the voice data is installed as the translation language of the voice data.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a transmission management system, a transmission management method, and a program.

  An example of a transmission system that transmits and receives content data between a plurality of transmission terminals via a relay device is a video conference system that performs a video conference or the like via a communication network such as the Internet. In a conventional video conference system, when a participant who has a video conference (a user of a conference terminal configuring the video conference system) speaks, audio data corresponding to the speech is input to the conference terminal of the participant. The language of the input voice data is automatically determined, and the voice data is converted into text data. The converted text data is translated into a predetermined language (hereinafter referred to as “translation language”), and the translated text data (hereinafter referred to as “translation text data”) is transmitted to the conference terminal of each participant. A text corresponding to the received translated text data (hereinafter referred to as “translated text”) is displayed on the conference terminal of each participant. Thereby, each participant can share a speaker's statement automatically.

  However, the conventional video conference system cannot automatically determine the language used by participants other than the speaker. For this reason, each participant had to set in advance the language used by himself / herself as the translation language of the text data displayed on his / her conference terminal. If this setting is not made, the translated text translated into the second language is displayed on the conference terminal of the participant who uses the first language, and the participant may not be able to share the speech of the speaker .

  The present invention has been made in view of the above problems, and an object of the present invention is to automatically determine the language of a text displayed on a transmission terminal as a language used by a user of the transmission terminal.

  A transmission management system according to an embodiment is a transmission management system that determines a translation language for translating audio data communicated between a plurality of transmission terminals via a communication network, and a terminal for identifying the transmission terminal For each piece of identification information, there are installed terminal management means for managing the area where the transmission terminal is installed, language management means for managing the language set for each area, and the transmission terminal serving as the destination of the audio data. Language determination means for determining the language set in the region as the translation language of the audio data.

  According to each embodiment of the present invention, a language that is likely to be used by a user can be automatically determined as a translation language of text data displayed on a transmission terminal used by the user.

1 is a schematic diagram of a transmission system according to a first embodiment of the present invention. It is the conceptual diagram which showed the transmission / reception state of the image data in the transmission system, audio | voice data, and various management information. It is a conceptual diagram explaining the image quality of image data. It is an external view of the terminal which concerns on this embodiment. It is a hardware block diagram of the terminal which concerns on this embodiment. It is a hardware block diagram of the management system, relay apparatus, program provision system, maintenance system, or service provision system which concerns on this embodiment. It is a functional block diagram of each terminal, apparatus, and system which comprise the transmission system which concerns on this embodiment. It is a functional block diagram of the last narrowing-down part. It is a functional block diagram of a primary narrowing-down part. It is a conceptual diagram which shows a change quality management table. It is a conceptual diagram which shows a relay apparatus management table. It is a conceptual diagram which shows a terminal authentication management table. It is a conceptual diagram which shows a terminal management table. It is a conceptual diagram which shows a destination list management table. It is a conceptual diagram which shows a session management table. It is a conceptual diagram which shows an address priority management table. It is a conceptual diagram which shows a transmission rate priority management table. It is a conceptual diagram which shows a quality management table. It is a conceptual diagram which shows a language management table. It is the sequence figure which showed the process which manages the status information which shows the operating state of each relay apparatus. It is the sequence diagram which showed the process of the preparation stage which starts remote communication between terminals. It is the sequence diagram which showed the process which narrows down a relay apparatus. FIG. 10 is a process flow diagram illustrating a process of narrowing down relay devices. It is the figure which showed the calculation state of the point of priority at the time of performing the narrowing-down process of a relay apparatus. FIG. 6 is a sequence diagram illustrating processing for a transmission terminal to select a relay device according to the first embodiment of the present invention. It is the processing flowchart which showed the process which selects a relay apparatus in a transmission terminal. It is the sequence diagram which showed the process which transmits / receives image data and audio | voice data between transmission terminals. FIG. 5 is a sequence diagram illustrating processing for displaying a translated text on a transmission destination terminal according to the first embodiment of the present invention. FIG. 10 is a sequence diagram illustrating processing for a transmission terminal to select a relay device according to a second embodiment of the present invention. FIG. 10 is a sequence diagram illustrating a process for displaying a translated text on a transmission destination terminal according to the third embodiment of the present invention.

[First Embodiment]
Hereinafter, the first embodiment of the present invention will be described with reference to FIGS.

<< Overall Configuration of Embodiment >>
FIG. 1 is a schematic diagram of a transmission system 1 according to this embodiment of the present invention. FIG. 2 is a conceptual diagram showing a state of transmission / reception of image data, audio data, and various management information in the transmission system. FIG. 3 is a conceptual diagram illustrating the image quality of image data.

  This transmission system includes a data providing system that transmits content data in one direction from one transmission terminal to the other transmission terminal via a transmission management system. The transmission system includes a communication system that transmits information, emotions, and the like between a plurality of transmission terminals via a transmission management system. This communication system is a system for mutually transmitting information, emotions, and the like between a plurality of communication terminals (corresponding to “transmission terminals”) via a communication management system (corresponding to “transmission management system”). Examples of the communication system include a video conference system and a video phone system.

  In the present embodiment, a video conference system as an example of a communication system, a video conference management system as an example of a communication management system, and a video conference terminal as an example of a communication terminal are assumed. Then, a transmission system, a transmission management system, and a transmission terminal will be described. That is, the transmission terminal and the transmission management system of the present invention are not only applied to a video conference system, but also applied to a communication system or a transmission system.

  First, the transmission system 1 shown in FIG. 1 includes a plurality of transmission terminals (10aa, 10ab,...), A display (120aa, 120ab,...) For each transmission terminal (10aa, 10ab,. A relay device (30a, 30b, 30c, 30d) is provided. The transmission system 1 includes a transmission management system 50, a program providing system 90, a maintenance system 100, and a service providing system 130.

  The plurality of transmission terminals 10 perform transmission by transmitting and receiving image data and audio data as an example of content data.

  In the following, “transmission terminal” is simply represented as “terminal”, and “transmission management system” is simply represented as “management system”. Further, an arbitrary terminal among the plurality of terminals (10aa, 10ab,...) Is represented as “terminal 10”, and an arbitrary display among the plurality of displays (120aa, 120ab,...) Is represented as “display 120”. ing. An arbitrary relay device among the plurality of relay devices (30a, 30b, 30c, 30d) is represented as “relay device 30”. Further, a terminal as a request source that requests the start of a video conference is represented as a “request source terminal”, and a terminal as a request destination (relay destination) is represented as a “destination terminal”. Furthermore, a terminal as a transmission source that transmits audio data is represented as a “transmission source terminal”, and a terminal as a transmission destination that receives audio data is represented as a “transmission destination terminal”.

  As shown in FIG. 2, in the transmission system 1, a management information session sei for transmitting and receiving various types of management information is transmitted between the request source terminal and the destination terminal via the management system 50. Established. Further, between the request source terminal and the destination terminal, four pieces of data of high resolution image data, medium resolution image data, low resolution image data, and audio data are transmitted and received via the relay device 30. Four sessions are established. Here, these four sessions are collectively shown as an image / audio data session sed.

  Further, in the present embodiment, between the voice data transmission source terminal and the service providing system 130, voice data, text data obtained by converting the voice data into text, translated text data obtained by translating the text data into a translation language, Are sent and received. The translation text data is transmitted from the transmission source terminal to the transmission destination terminal via the management system 50, and the translation text corresponding to the translation text data is displayed on the transmission destination terminal.

  Here, the resolution of the image data handled in the present embodiment will be described. As shown in FIG. 3 (a), the horizontal image is composed of 160 pixels and the vertical image is 120 pixels, and a low-resolution image serving as a base image and a horizontal image as illustrated in FIG. 3 (b). There are medium resolution images consisting of 320 pixels and 240 pixels vertically. Further, as shown in FIG. 3C, there is a high-resolution image having 640 pixels in the horizontal direction and 480 pixels in the vertical direction. Among these, when passing through a narrow band route, low-quality image data consisting only of low-resolution image data serving as a base image is relayed. When the band is relatively wide, low-resolution image data serving as a base image and medium-quality image data composed of medium-resolution image data are relayed. When the bandwidth is very wide, high-resolution image data composed of low-resolution image data serving as base image quality, intermediate-resolution image data, and high-resolution image data is relayed.

  The relay device 30 shown in FIG. 1 relays content data among a plurality of terminals 10. The management system 50 centrally manages the login authentication from the terminal 10, the management of the call status of the terminal 10, the management of the destination list, and the communication status of the relay device 30. The image of the image data may be a moving image or a still image, or both a moving image and a still image.

  A plurality of routers (70a, 70b, 70c, 70d, 70ab, 70cd) select an optimum route for image data and audio data. In the following, an arbitrary router among the routers (70a, 70b, 70c, 70d, 70ab, 70cd) is represented as “router 70”.

  The program providing system 90 includes an HD (Hard Disk) 204 described below, and stores a terminal program for causing the terminal 10 to realize various functions (or to cause the terminal 10 to function as various means). The terminal program can be transmitted to The HD 204 of the program providing system 90 also stores a relay device program for causing the relay device 30 to realize various functions (or to cause the relay device 30 to function as various means). The relay device program can be transmitted. Further, the HD 204 of the program providing system 90 also stores a transmission management program for causing the management system 50 to realize various functions (or to cause the management system 50 to function as various means). The program providing system 90 can transmit a transmission management program to the management system 50. Furthermore, the HD 204 of the program providing system 90 also stores a service providing program for causing the service providing system 130 to realize various functions (or to cause the service providing system 130 to function as various means). The program providing system 90 can transmit a service providing program to the service providing system 130.

  The maintenance system 100 is a computer for maintaining, managing, or maintaining at least one of the terminal 10, the relay device 30, the management system 50, the program providing system 90, and the service providing system 130. For example, consider a case where the maintenance system 100 is installed in the country, and the terminal 10, the relay device 30, the management system 50, the program providing system 90, or the service providing system 130 is installed outside the country. In this case, the maintenance system 100 remotely maintains, manages, and maintains at least one of the terminal 10, the relay device 30, the management system 50, the program providing system 90, and the service providing system 130 via the communication network 2. Perform maintenance such as. In addition, the maintenance system 100 manages the model number, serial number, sales destination, maintenance inspection, or failure history in at least one of the terminal 10, the relay device 30, and the management system 50 without using the communication network 2. Perform maintenance such as. In addition, the maintenance system 100 does not use the communication network 2 to manage the model number, the manufacturing number, the sales destination, the maintenance inspection, or the failure history in at least one of the program providing system 90 and the service providing system 130. Maintenance may be performed.

  The service providing system 130 is a computer for generating translated text data from speech data corresponding to a speaker's speech (speech). The service providing system 130 provides a text conversion service that converts voice data into text data, and a translation service that translates the text data into a translation language and generates translated text data. The service providing system 130 communicates with the terminal 10 via the communication network 2.

  In the present embodiment, when voice data is input, the transmission source terminal transmits the input voice data to the service providing system 130. The service providing system 130 converts the received voice data into text data, and returns the converted text data to the transmission source terminal. The source terminal transmits the text data received from the service providing system 130 to the service providing system 130 again in order to translate the text data. The service providing system 130 translates the received text data into a translation language, and returns the obtained translated text data to the transmission source terminal. The transmission source terminal transmits the received translated text data to the transmission destination terminal.

  By the way, the terminals (10aa, 10ab, 10ac,...), The relay device 30a, and the router 70a are communicably connected via the LAN 2a. The terminals (10ba, 10bb, 10bc,...), The relay device 30b, and the router 70b are communicably connected via a LAN 2b. The LAN 2a and the LAN 2b are communicably connected by a dedicated line 2ab including a router 70ab, and are constructed in a predetermined area A. For example, the region A is Japan, the LAN 2a is constructed in a Tokyo office, and the LAN 2b is constructed in an Osaka office.

  On the other hand, the terminals (10ca, 10cb, 10cc,...), The relay device 30c, and the router 70c are communicably connected via a LAN 2c. The terminal 10d (a, 10db, 10dc,...), The relay device 30d, and the router 70d are communicably connected via a LAN 2d. The LAN 2c and the LAN 2d are communicably connected via a dedicated line 2cd including the router 70cd, and are constructed in a predetermined area B. For example, region B is the United States of America, LAN 2c is built in a New York office, and LAN 2d is Washington D.C. C. Is built in the office. Area A and area B are connected to each other via routers (70ab, 70cd) via the Internet 2i.

  In addition, the management system 50, the program providing system 90, and the service providing system 130 are connected to the terminal 10 and the relay device 30 through the Internet 2i so as to be communicable. The management system 50, the program providing system 90, and the service providing system 130 may be installed in the region A or the region B, or may be installed in other regions.

  In the present embodiment, the communication network 2 of the present embodiment is constructed by the LAN 2a, the LAN 2b, the dedicated line 2ab, the Internet 2i, the dedicated line 2cd, the LAN 2c, and the LAN 2d. The communication network 2 may have a place where wireless communication such as WiFi (Wireless Fidelity) or Bluetooth (registered trademark) is performed in addition to wired communication.

  Further, in FIG. 1, four sets of numbers shown below each terminal 10 and the like simply indicate an IP address in general IPv4. For example, the IP address of the terminal 10aa is “1.2.1.3”. In addition, IPv6 may be used instead of IPv4, but IPv4 will be used for the sake of simplicity.

  In addition, each terminal 10 is not only a call between a plurality of business establishments or a call between different rooms in the same business establishment, but a call within the same room or a call between the outdoors and indoors or outdoors and outdoors. May be used. When each terminal 10 is used outdoors, wireless communication such as a cellular phone communication network is performed.

<< Hardware Configuration of Embodiment >>
Next, the hardware configuration of this embodiment will be described. In the present embodiment, when a delay occurs in the reception of image data at the terminal 10 as the destination (relay destination), the image resolution of the image data is changed by the relay device 30 before the terminal 10 as the relay destination. A case where image data is transmitted will be described.

  FIG. 4 is an external view of the terminal 10 according to the present embodiment. As shown in FIG. 4, the terminal 10 includes a housing 1021, an arm 1074, and a camera housing 1075. Among these, a plurality of intake holes 1021e are formed on substantially the entire front wall 1021a of the housing 1021, and a plurality of exhaust holes (not shown) are formed on substantially the entire rear wall 1021b of the housing 1021. ing. Thus, by driving a cooling fan built in the housing 1021, the outside air in front of the terminal 10 can be taken in through the intake hole 1021e and exhausted to the rear of the terminal 10 through the exhaust hole. A sound collecting hole 1021f is formed at the center of the front side wall 1021a, and sounds such as voice, sound, and noise can be collected by a built-in microphone 114 described later.

  An operation panel 1022 is formed on the left side of the housing 1021 when viewed from the front. The operation panel 1022 is provided with a later-described operation button 108 and a later-described power switch 109, and a plurality of sound output holes 1022 f for allowing an output sound from a later-described built-in speaker 115 to pass therethrough. ing. Further, an accommodation portion 1021p as a recess for accommodating the arm 1074 and the camera housing 1075 is formed on the right side of the housing 1021 when viewed from the front.

  The arm 1074 is attached to the housing 1021 via the torque hinge 1073. The arm 1074 has a panning angle θ1 of ± 180 degrees with respect to the housing 1021, and a range of a tilt angle θ2 of 90 degrees (approximately 45 degrees tilting so that a click feeling is generated). It is configured to be rotatable in the vertical and horizontal directions.

  The camera housing 1075 is provided with a built-in camera 112 to be described later, and can capture images of a user, a room, and the like. The camera housing 1075 is formed with a torque hinge 1075a. The camera housing 1075 is attached to the arm 1074 via a torque hinge 1075a. The state in which the camera housing 1075 is linear with respect to the arm 1074 is defined as 0 degree, and the tilt angle θ3 is approximately 100 degrees on the front side of the terminal 10 and approximately 90 degrees on the rear side of the terminal 10. It is configured to be rotatable.

  The relay device 30, the management system 50, the program providing system 90, the maintenance system 100, and the service providing system 130 have the same external appearance as a general server computer, and thus the description of the external appearance is omitted.

  FIG. 5 is a hardware configuration diagram of the terminal 10 according to this embodiment of the present invention. The terminal 10 of this embodiment includes a CPU (Central Processing Unit) 101 that controls the operation of the entire terminal 10 and a ROM (Read Only Memory) 102 that stores a program used to drive the CPU 101 such as an IPL (Initial Program Loader). It has. The terminal 10 also includes a RAM (Random Access Memory) 103 used as a work area of the CPU 101, a flash memory 104 that stores various data such as a terminal program, image data, and audio data. Further, the terminal 10 includes an SSD (Solid State Drive) 105 that controls reading or writing of various data with respect to the flash memory 104 according to the control of the CPU 101. Furthermore, the terminal 10 includes a media drive 107 that controls reading or writing (storage) of data with respect to the recording medium 106 such as a flash memory, and an operation button 108 that is operated when selecting a destination of the terminal 10. Yes. The terminal 10 includes a power switch 109 for switching on / off the power of the terminal 10 and a network I / F (Interface) 111 for data transmission using the communication network 2.

  The terminal 10 also includes a built-in camera 112 that captures an image of a subject under the control of the CPU 101 to obtain image data, an image sensor I / F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs sound, and sound. Is provided. The built-in speaker 115 is provided. Further, the terminal 10 includes an audio input / output I / F 116 that processes input / output of an audio signal between the microphone 114 and the speaker 115 according to the control of the CPU 101. Furthermore, the terminal 10 includes a display I / F 117 that transmits image data to an external display 120 according to the control of the CPU 101, and an external device connection I / F 118 for connecting various external devices. The terminal 10 includes a bus line 110 such as an address bus and a data bus for electrically connecting the above-described components as shown in FIG.

  The display 120 is a display unit configured by a liquid crystal or an organic EL that displays an image of a subject, an operation icon, and the like. The display 120 is connected to the display I / F 117 by a cable 120c. The cable 120c may be an analog RGB (VGA) signal cable or a component video cable. Further, the cable 120c may be a cable for HDMI (High-Definition Multimedia Interface) or DVI (Digital Video Interactive) signal.

  The camera 112 includes a lens and a solid-state image sensor that converts an image (video) of an object by converting light into electric charges. As the solid-state image sensor, a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) Etc. are used.

  External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 118 by a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven in preference to the built-in camera 112 under the control of the CPU 101. Similarly, when an external microphone is connected or when an external speaker is connected, each of the external microphones and the built-in speaker 115 is given priority over the internal microphone 114 and the internal speaker 115 according to the control of the CPU 101. An external speaker is driven.

  The recording medium 106 is detachable from the terminal 10. Further, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 101, not only the flash memory 104 but also an EEPROM (Electrically Erasable and Programmable ROM) or the like may be used.

  Further, the terminal program may be recorded in a computer-readable recording medium such as the recording medium 106 and distributed as a file in an installable or executable format. The terminal program may be stored in the ROM 102 instead of the flash memory 104.

  Furthermore, the flash memory 104 stores voice data, text data, translated text data, speaker language information, and translated language information. The speaker language information is information indicating the speaker language. The speaker language is a language used by the user of the transmission source terminal, and is a language of voice data and text data. The translation language information is information indicating the translation language. The translation language is a language used by the user of the transmission destination terminal, and is a language of translated text data. The translation language corresponds to a language for translating speech data and text data. Text data, translated text data, speaker language information, and translated language information are received via the network I / F 111.

  FIG. 6 is a hardware configuration diagram of the management system 50 according to this embodiment of the present invention. The management system 50 includes a CPU 201 that controls the overall operation of the management system 50, a ROM 202 that stores a program used to drive the CPU 201 such as an IPL, and a RAM 203 that is used as a work area for the CPU 201. The management system 50 also includes an HD 204 that stores various data such as a transmission management program, and an HDD (Hard Disk Drive) 205 that controls reading or writing of various data with respect to the HD 204 according to the control of the CPU 201. The management system 50 also includes a media drive 207 that controls reading or writing of data with respect to a recording medium 206 such as a flash memory, and a display 208 that displays various types of information such as a cursor, menu, window, character, or image. In addition, the management system 50 includes a network I / F 209 for transmitting data using the communication network 2 and a keyboard 211 having a plurality of keys for inputting characters, numerical values, various instructions, and the like. The management system 50 also includes a mouse 212 for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The management system 50 also includes a CD-ROM drive 214 that controls reading or writing of various data with respect to a CD-ROM (Compact Disc Read Only Memory) 213 as an example of a removable recording medium. Further, the management system 50 includes a bus line 210 such as an address bus or a data bus for electrically connecting the above-described components as shown in FIG.

  The transmission management program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213. . The transmission management program may be stored in the ROM 202 instead of the HD 204.

  Further, since the relay device 30 has the same hardware configuration as that of the management system 50, the description thereof is omitted. However, a relay device program for controlling the relay device 30 is recorded in the HD 204. Also in this case, the relay device program is a file in an installable or executable format, and is recorded on a computer-readable recording medium such as the recording medium 206 or CD-ROM 213 for distribution. Good. The relay device program may be stored in the ROM 202 instead of the HD 204.

  Further, since the program providing system 90 has the same hardware configuration as that of the management system 50, the description thereof is omitted. However, a program providing program for controlling the program providing system 90 is recorded in the HD 204. Also in this case, the program providing program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Good. The program providing system program may be stored in the ROM 202 instead of the HD 204.

  Furthermore, since the maintenance system 100 has the same hardware configuration as the management system 50, the description thereof is omitted. However, a maintenance program for controlling the maintenance system 100 is recorded in the HD 204. Also in this case, the maintenance program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. . The maintenance program may be stored in the ROM 202 instead of the HD 204.

  Furthermore, since the service providing system 130 has the same hardware configuration as that of the management system 50, the description thereof is omitted. However, a service providing program for controlling the service providing system 130 is recorded in the HD 204. Also in this case, the service providing program is a file in an installable or executable format, and may be recorded and distributed on a computer-readable recording medium such as the recording medium 206 or the CD-ROM 213. Good. The service providing system program may be stored in the ROM 202 instead of the HD 204.

  As another example of the detachable recording medium, the recording medium is provided by being recorded on a computer-readable recording medium such as a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), or a Blu-ray Disc. May be.

<< Functional Configuration of Embodiment >>
Next, the functional configuration of this embodiment will be described. FIG. 7 is a functional block diagram of each terminal, device, and system constituting the transmission system 1 of the present embodiment. In FIG. 7, the terminal 10, the relay device 30, the management system 50, and the service providing system 130 are connected so that data communication can be performed via the communication network 2. Further, the program providing system 90 and the maintenance system 100 shown in FIG. 1 are omitted in FIG. 7 because they are not directly related to the video conference communication.

<Functional configuration of terminal>
The terminal 10 includes a transmission / reception unit 11, an operation input reception unit 12, a login request unit 13, an imaging unit 14, an audio input unit 15a, an audio output unit 15b, a final narrowing unit 16, a display control unit 17, a delay detection unit 18, and a storage A read processing unit 19 and a service request unit 20 are included. Each of these units is a function realized by any one of the constituent elements shown in FIG. 5 being operated by a command from the CPU 101 according to the terminal program developed from the flash memory 104 onto the RAM 103, or It is a means to function. Further, the terminal 10 has a storage unit 1000 constructed by the RAM 103 shown in FIG. 5 and the flash memory 104 shown in FIG.

(Functional configuration of terminal)
Next, each functional configuration of the terminal 10 will be described in detail with reference to FIGS. 5 and 7. In the following, in describing each functional configuration of the terminal 10, among the components illustrated in FIG. 5, a relationship with main components for realizing each functional configuration of the terminal 10 will also be described. .

  The transmission / reception unit 11 of the terminal 10 illustrated in FIG. 7 is realized by the instruction from the CPU 101 illustrated in FIG. 5 and the network I / F 111 illustrated in FIG. Various data (or information) is transmitted to and received from the device or system. The transmission / reception unit 11 starts receiving state information indicating the state of each terminal as a destination candidate from the management system 50 before starting a call with a desired destination terminal. This status information includes not only the operating status of each terminal 10 (online or offline status) but also whether it is possible to make a call even when online, whether it is in a call, whether it is away from the office, etc. Shows detailed status. In addition, the status information is not limited to the operating status of each terminal 10, but the cable 120 c is disconnected from the terminal 10 at the terminal 10, audio is output but no image is output, or audio is not output. Various states such as (MUTE) are shown. Below, the case where status information shows an operation state is demonstrated as an example.

  The operation input receiving unit 12 is realized by a command from the CPU 101 shown in FIG. 5 and the operation buttons 108 and the power switch 109 shown in FIG. 5 and receives various inputs by the user. For example, when the user turns on the power switch 109 shown in FIG. 5, the operation input receiving unit 12 shown in FIG. 7 receives the power ON and turns on the power.

  The login request unit 13 is realized by a command from the CPU 101 shown in FIG. The login request unit 13 receives the above-mentioned power-on reception, and login request information indicating that the login is requested from the transmission / reception unit 11 to the management system 50 via the communication network 2 and the current IP address of the requesting terminal. Will be sent automatically. When the user turns the power switch 109 from the ON state to the OFF state, the operation input receiving unit 12 completely turns off the power after the transmission / reception unit 11 transmits the state information indicating that the power supply is turned off to the management system 50. To do. As a result, the management system 50 side can grasp that the terminal 10 has been turned off from being turned on.

  The imaging unit 14 is realized by a command from the CPU 101 illustrated in FIG. 5 and the camera 112 and the image sensor I / F 113 illustrated in FIG. 5. Output data.

  The voice input unit 15a is realized by the instruction from the CPU 101 shown in FIG. 5 and the voice input / output I / F 116 shown in FIG. 5, and the user's voice is converted into a voice signal by the microphone 114. Thereafter, audio data relating to the audio signal is input. The audio output unit 15b is realized by a command from the CPU 101 shown in FIG. 5 and the audio input / output I / F 116 shown in FIG. 5, and outputs an audio signal related to the audio data to the speaker 115. The voice is output from 115.

  As shown in FIG. 8, the final narrowing unit 16 performs a final narrowing process for finally narrowing down from a plurality of relay devices 30 to one relay device 30, according to a command from the CPU 101 shown in FIG. A measurement unit 16a, a calculation unit 16b, and a final selection unit 16c are realized.

  Among these, the measurement unit 16 a measures the reception date and time when the pre-transmission information is received by the transmission / reception unit 11 for each pre-transmission information described later received by the transmission / reception unit 11. For each pre-transmission information whose reception date and time is measured by the measurement unit 16a, the calculation unit 16b determines the pre-transmission information based on the difference between the measured reception time and the transmission date and time included in the pre-transmission information. The required time (T) from transmission to reception is calculated. The final selection unit 16c finally selects one relay device by selecting the relay device 30 to which the pre-transmission information that required the shortest required time is relayed among the required times calculated by the calculation unit 16b. .

  The display control unit 17 is realized by a command from the CPU 101 illustrated in FIG. 5 and the display I / F 117 illustrated in FIG. As will be described later, the display control unit 17 combines the received image data with different resolutions, and performs control for transmitting the combined image data to the display 120. In addition, the display control unit 17 can transmit the destination list information received from the management system 50 to the display 120 and cause the display 120 to display the destination list.

  The delay detection unit 18 is realized by a command from the CPU 101 shown in FIG. 5 and detects a delay time (ms) of image data or audio data sent from another terminal 10 via the relay device 30. .

  The storage / read processing unit 19 is executed by the instruction from the CPU 101 shown in FIG. 5 and the SSD 105 shown in FIG. The storage / reading processing unit 19 stores various data in the storage unit 1000 and performs processing for reading out various data stored in the storage unit 1000. The storage unit 1000 stores a terminal ID (Identification) for identifying the terminal 10, a destination name, a password, and the like. Furthermore, the storage unit 1000 overwrites and stores image data and audio data received when a call is made with the destination terminal. Among these, an image is displayed on the display 120 by the image data before being overwritten, and sound is output from the speaker 115 by the audio data before being overwritten. Further, the storage unit 1000 stores the speaker language information and translation language information received from the management system 50, and the text data and translation text data received from the service providing system 130.

  The service request unit 20 is realized by a command from the CPU 101 shown in FIG. 5, and provides service to the service providing system 130 using the speaker language information and the translation language information stored in the storage unit 1000. Request and accept service provision.

  Note that the terminal ID of the present embodiment and a relay device ID described later indicate identification information such as a language, characters, symbols, or various signs used to uniquely identify the terminal 10 and the relay device 30, respectively. Further, the terminal ID and the relay device ID may be identification information in which at least two of the language, characters, symbols, and various indicia are combined.

<Functional configuration of relay device>
The relay device 30 includes a transmission / reception unit 31, a state detection unit 32, a data quality confirmation unit 33, a change quality management unit 34, a data quality change unit 35, and a storage / read processing unit 39. Each of these units functions or functions realized by any one of the constituent elements shown in FIG. 6 operating according to a command from the CPU 201 according to the relay device program expanded from the HD 204 onto the RAM 203. Means. The relay device 30 includes a storage unit 3000 configured by the RAM 203 illustrated in FIG. 6 and / or the HD 204 illustrated in FIG.

(Change quality control table)
In the storage unit 3000, a change quality management DB (Data Base) 3001 configured by a change quality management table as shown in FIG. In the change quality management table, the IP address of the terminal 10 as the relay destination (destination) of the image data and the image quality of the image data relayed by the relay device 30 are associated with the relay destination and managed.

(Each functional configuration of the relay device)
Next, each functional configuration of the relay device 30 will be described in detail. In the following, in describing each functional configuration of the relay device 30, among the components illustrated in FIG. 6, the relationship with the main components for realizing each functional configuration of the relay device 30 is also described. explain.

  The transmission / reception unit 31 of the relay device 30 is realized by a command from the CPU 201 shown in FIG. 6 and the network I / F 209 shown in FIG. Sends and receives various data (or information) with the system.

  The state detection unit 32 is realized by a command from the CPU 201 illustrated in FIG. 6, and detects an operating state of the relay device 30 having the state detection unit 32. The operating state includes “online”, “offline”, “busy”, or “temporarily interrupted”.

  The data quality confirmation unit 33 is realized by a command from the CPU 201 shown in FIG. The data quality confirmation unit 33 searches the change quality management DB 3001 (see FIG. 10) using the IP address of the destination terminal as a search key, and extracts the image quality of the corresponding relayed image data, thereby relaying the image data to be relayed. Check the image quality.

  The change quality management unit 34 is realized by a command from the CPU 201 shown in FIG. 6 and changes the contents of the change quality management DB 3001 based on quality information to be described later sent from the management system 50. For example, a video conference is performed by transmitting and receiving high-quality image data between a request source terminal (terminal 10aa) whose terminal ID is “01aa” and a destination terminal (terminal 10db) whose terminal ID is “01db”. Think about what you are doing. Then, during the video conference, when the request source terminal (terminal 10bb) and the destination terminal (terminal 10ca) that conduct another video conference start the video conference via the communication network 2, the destination terminal (terminal 10db) ), A delay in receiving the image data occurs. In this case, the relay device 30 lowers the image quality of the image data relayed so far from high image quality to medium image quality. In such a case, the content of the change quality management DB 3001 is changed so as to lower the image quality of the image data relayed by the relay device 30 from the high image quality to the medium image quality based on the quality information indicating the medium image quality.

  The data quality changing unit 35 is realized by a command from the CPU 201 shown in FIG. 6, and the image quality of the image data sent from the transmission source terminal is changed based on the contents of the changed change quality management DB 3001. change.

  The storage / reading processing unit 39 is realized by an instruction from the CPU 201 illustrated in FIG. 6 and the HDD 205 illustrated in FIG. 6. The storage / reading processing unit 39 stores various data in the storage unit 3000 or is stored in the storage unit 3000. To read various data.

<Functional configuration of management system>
The management system 50 includes a transmission / reception unit 51, a terminal authentication unit 52, a state management unit 53, a terminal extraction unit 54, a terminal state acquisition unit 55, a primary narrowing unit 56, a session management unit 57, a quality determination unit 58, and a storage / read process. A unit 59, a delay time management unit 60, and a language determination unit 61. Each of these units functions or functions realized by any one of the constituent elements shown in FIG. 6 operating according to a command from the CPU 201 according to the management system program expanded from the HD 204 onto the RAM 203. Means. Further, the management system 50 has a storage unit 5000 constructed by the HD 204 shown in FIG.

(Relay device management table)
In the storage unit 5000, a relay device management DB 5001 configured by a relay device management table as shown in FIG. 11 is constructed. In this relay device management table, for each relay device ID of each relay device 30, the operation status of each relay device 30, the reception date and time when the status information indicating the operation status is received by the management system 50, and the IP of the relay device 30 Addresses are associated and managed. In the relay device management table, the maximum data transmission rate (Mbps) in the relay device 30 is managed in association with each relay device ID of each relay device 30. For example, in the relay device management table, the relay device 30a with the relay device ID “111a” has an “online” operating state, and the date and time when the status information was received by the management system 50 is “13 Nov. 10, 2009”. "00 hour" is shown. Further, the relay device 30a with the relay device ID “111a” has an IP address “1.2.1.2” of the relay device 30a, and the maximum data transmission speed in the relay device 30a is 100 Mbps. Has been.

(Terminal authentication management table)
Furthermore, in the storage unit 5000, a terminal authentication management DB 5002 configured by a terminal authentication management table as shown in FIG. In this terminal authentication management table, each password is associated with each terminal ID of all the terminals 10 managed by the management system 50 and managed. For example, the terminal authentication management table shown in FIG. 12 indicates that the terminal ID of the terminal 10aa is “01aa” and the password is “aaaa”.

(Terminal management table)
Further, in the storage unit 5000, a terminal management DB 5003 configured by a terminal management table as shown in FIG. 13 is constructed. In this terminal management table, for each terminal ID of each terminal 10, a destination name when each terminal 10 is a destination, an operating state of each terminal 10, a reception date and time when login request information described later is received by the management system 50, And the IP address of the terminal 10 is managed in association with each other. The destination name is set in advance, and includes the area where the terminal 10 is installed (such as a country name and business name) and the name of the terminal 10. For example, in the terminal management table shown in FIG. 13, the terminal 10aa with the terminal ID “01aa” has the destination name “Japan Tokyo Office AA terminal” and the operating state “online (can call)”. It has been shown. The terminal 10aa with the terminal ID “01aa” has the date and time when the login request information is received by the management system 50 “13:40 on November 10, 2009”, and the IP address of this terminal 10aa is “1.2. 1.3 ".

(Destination list management table)
Furthermore, in the storage unit 5000, a destination list management DB 5004 configured by a destination list management table as shown in FIG. 14 is constructed. In this destination list management table, all terminal IDs of destination terminals registered as destination terminal candidates are managed in association with terminal IDs of request source terminals that request the start of a call in a video conference. For example, in the destination list management table shown in FIG. 14, the requesting terminal (terminal 10aa) whose terminal ID is “01aa” is the destination terminal (terminal 10db) that can request the start of a video conference call. Candidates are shown. Specifically, as destination terminal candidates, a terminal 10ab having a terminal ID “01ab”, a terminal 10ba having a terminal ID “01ba”, a terminal 10bb having a terminal ID “01bb”, and the like are illustrated. This destination terminal candidate is updated by being added or deleted by a request for addition or deletion from an arbitrary request source terminal to the management system 50.

(Session management table)
In the storage unit 5000, a session management DB 5005 configured by a session management table as shown in FIG. 15 is constructed. In this session management table, for each selection session ID used to execute a session for selecting the relay device 30, the relay device ID of the relay device 30 used for relaying image data and audio data, and the request source terminal Are managed in association with each other. In the session temporary table, for each selection session ID, the terminal ID of the destination terminal and the reception delay time (ms) when the image data is received at the destination terminal are associated and managed. Also, in the session temporary table, for each selection session ID, the delay information indicating the delay time is sent from the destination terminal, and the reception date and time received by the management system 50 is associated and managed.

  For example, in the session management table shown in FIG. 15, the following is shown for the relay device 30a (relay device ID “111a”) selected in the session executed using the selection session ID “se1”. Has been. That is, the relay device 30a relays image data and audio data between a request source terminal (terminal 10aa) whose terminal ID is “01aa” and a destination terminal (terminal 10db) whose terminal ID is “01db”. It has been shown that Further, the relay device 30a is shown to have a delay time of 200 (ms) for the image data at the time of “14:00 on November 10, 2009” at the destination terminal (terminal 10db). When a video conference is performed between the two terminals 10, the reception date and time of the delay information may be managed based on the delay information transmitted from the request source terminal instead of the destination terminal. However, when a video conference is performed between three or more terminals 10, the reception date and time of delay information is managed based on the delay information transmitted from the terminal 10 on the image data and audio data receiving side.

(Address priority management table)
Furthermore, in the storage unit 5000, a priority management DB 5006 configured by an address priority management table as shown in FIG. 16 is constructed. In this address priority management table, the difference between dot addresses and the address priority are managed in association with each other. Specifically, in any terminal 10 and any relay device 30, the more the “same” of the four sets of dot addresses (Dot Address) of the IP addresses in general IPv4, the more points of address priority. Is managed to be higher. This “same” means that the dot address portions are the same, and “different” means that the dot address portions are different. For example, in the address priority management table shown in FIG. 16, when three values are the same IP address from the top to the bottom of the dot address, the address priority point is “5”. When the two values are the same IP address from the top to the bottom of the dot address, the address priority point is “3”. In this case, whether the value of the lowest dot address is the same or not is not related to the priority. In the case of IP addresses having the same highest value of dot addresses and different second values from the top, the address priority point is “1”. In this case, whether or not the values of the third and lowest dot addresses from the top are the same has no relation to the priority. In the case of IP addresses having different uppermost values of dot addresses, the address priority point is “0”. In this case, whether or not the values of the second, third, and lowest dot addresses from the top are the same has no relation to the priority.

(Transmission speed priority management table)
The priority management DB 5006 built in the storage unit 5000 also includes a transmission speed priority management table as shown in FIG. In this transmission rate priority management table, the maximum data transmission rate and the transmission rate priority are set such that the higher the maximum data transmission rate (Mbps) value in the relay device 30, the higher the transmission rate priority point. Associated and managed. For example, in the transmission rate priority management table shown in FIG. 17, when the maximum data transmission rate in the relay apparatus 30 is 1000 Mbps or more, the transmission rate priority point is “5”. When the maximum data transmission rate in the relay device 30 is 100 Mbps or more and less than 1000 Mbps, the transmission rate priority point is “3”. When the maximum data transmission rate in the relay device 30 is 10 Mbps or more and less than 100 Mbps, the transmission rate priority point is “1”. When the maximum data transmission rate in the relay device 30 is less than 10 Mbps, the transmission rate priority point is “0”.

(Quality control table)
Furthermore, in the storage unit 5000, a quality management DB 5007 configured by a quality management table as shown in FIG. In this quality management table, the longer the delay time (ms) of the image data at the requesting terminal or the destination terminal, the lower the image data delay time and the image data quality so that the image quality of the image data relayed by the relay device 30 is lowered. (Image quality) and are managed in association with each other.

(Language management table)
In the storage unit 5000, a language management DB 5008 configured by a language management table as shown in FIG. 19 is constructed. In this language management table, a region name and a language most used in the region are managed in association with each other. The language corresponding to the region is set in advance. For example, the language management table shown in FIG. 19 indicates that the language most used in Japan is Japanese. Such a language management table is prepared in the storage unit 5000 in advance. As will be described later, the speaker language and the translation language are determined based on the language management table. Note that the unit of the area where the language management table manages the language is not limited to the country, but may be a state or the like. That is, the language management table may manage languages for each region such as a state.

(Functional configuration of the management system)
Next, each functional configuration of the management system 50 will be described in detail. In the following, in describing each functional configuration of the management system 50, among the components shown in FIG. 6, the relationship with the main components for realizing each functional configuration of the management system 50 is also described. explain.

  The transmission / reception unit 51 is executed by the command from the CPU 201 shown in FIG. 6 and the network I / F 209 shown in FIG. Or information).

  The terminal authentication unit 52 is realized by a command from the CPU 102 shown in FIG. The terminal authentication unit 52 searches the terminal authentication management DB 5002 of the storage unit 5000 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys. Then, the terminal authentication unit 52 performs terminal authentication by determining whether or not the same terminal ID and password are managed in the terminal authentication management DB 5002.

  The state management unit 53 is realized by a command from the CPU 102 shown in FIG. In order to manage the operation state of the request source terminal, the state management unit 53 stores the terminal ID of the request source terminal, the operation state of the request source terminal, and the reception date and time when the login request information is received by the management system 50 in the terminal management DB 5003. , And the IP address of the requesting terminal is stored and managed in association with each other. The state management unit 53 also manages the terminal based on the state information indicating that the power is turned off, which is sent from the terminal 10 when the user turns the power switch 109 of the terminal 10 from ON to OFF. The operation state indicating online of the DB 5003 is changed to offline.

  The terminal extraction unit 54 is realized by a command from the CPU 102 shown in FIG. The terminal extraction unit 54 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the request source terminal that requested the login as a key, and reads out terminal IDs of candidate destination terminals that can talk to the request source terminal. Thus, the terminal ID is extracted. In addition, the terminal extraction unit 54 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID of the request source terminal that has requested the login as a key, and registers the terminal ID of the request source terminal as a destination terminal candidate. The terminal IDs of other request source terminals are also extracted.

  The terminal state acquisition unit 55 is realized by a command from the CPU 102 shown in FIG. The terminal status acquisition unit 55 searches the terminal management DB 5003 (see FIG. 13) using the terminal ID of the destination terminal candidate extracted by the terminal extraction unit 54 as a search key, and the terminal extracted by the terminal extraction unit 54 Read the operating status for each ID. Thereby, the terminal state acquisition unit 55 can acquire the operating state of the candidate destination terminal that can make a call with the request source terminal that has requested the login. Further, the terminal state acquisition unit 55 searches the terminal management DB 5003 using the terminal ID extracted by the terminal extraction unit 54 as a search key, and also acquires the operating state of the request source terminal that has requested login.

  The primary narrowing unit 56 is realized by a command from the CPU 102 shown in FIG. 6, and this final narrowing process is performed in order to support a final narrowing process that finally narrows down from a plurality of relay apparatuses 30 to one relay apparatus 30. Perform the previous primary narrowing process. For this purpose, the primary narrowing-down unit 56, as shown in FIG. 9, in response to an instruction from the CPU 201 shown in FIG. 5, the selection session ID generation unit 56a, the terminal IP address extraction unit 56b, the primary selection The unit 56c and the priority determination unit 56d are realized.

  Among these, the selection session ID generation unit 56 a generates a selection session ID used for executing a session for selecting the relay device 30. The terminal IP address extraction unit 56b searches the terminal management DB 5003 based on the terminal ID of the request source terminal and the terminal ID of the destination terminal included in the start request information sent from the request source terminal. The IP address of each corresponding terminal 10 is extracted. The primary selection unit 56c selects the relay device ID of the relay device 30 whose operation state is “online” from among the relay devices 30 managed by the relay device management DB 5001 (see FIG. 11). The device 30 is selected.

  Further, the primary selection unit 56c searches the relay device management DB 5001 (see FIG. 11) based on the IP address of the request source terminal and the IP address of the destination terminal extracted by the terminal IP address extraction unit 56b. As a result, the primary selection unit 56c investigates whether each dot address of the IP address of the selected relay device 30 is the same as or different from each dot address in each IP address of the request source terminal and the destination terminal. . Further, for each relay device, the primary selection unit 56c includes the higher two relay devices having the higher points of the integrated points obtained by integrating the higher points with respect to the terminal 10 at the address priority points and the transmission speed priority points. Select 30. Accordingly, the primary selection unit 56c performs further selection of the relay device 30. In the present embodiment, the top two relay devices 30 with high points are selected. However, the present invention is not limited to this. If one relay device 30 can be narrowed down as many as possible, the top points with high points are selected. Three or more relay devices 30 may be selected.

  The priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16), and determines an address priority point for each relay device 30 investigated by the primary selection unit 56c. The priority determination unit 56d searches the priority management DB 5006 (see FIG. 17) based on the maximum data transmission rate of each relay device 30 managed in the relay device management DB 5001 (see FIG. 11). Accordingly, the priority determination unit 56d determines a transmission speed priority point for each relay device 30 narrowed down by the first narrowing-down process by the primary selection unit 56c.

  Subsequently, the session management unit 57 is realized by a command from the CPU 102 shown in FIG. The session management unit 57 stores the selection session ID generated by the selection session ID generation unit 56a, the terminal ID of the request source terminal, and the terminal ID of the destination terminal in the session management DB 5005 (see FIG. 15) of the storage unit 5000. Store and manage in association. In addition, the session management unit 57, for the session management DB 5005 (see FIG. 15), for each selection session ID, the relay device of the relay device 30 that is finally selected by the final selection unit 16c of the terminal 10 Store and manage IDs.

  The quality determination unit 58 searches the quality management DB 5007 (see FIG. 18) using the delay time as a search key and extracts the image quality of the corresponding image data, thereby determining the image quality of the image data to be relayed to the relay device 30. To do.

  The storage / read processing unit 59 is executed by the instruction from the CPU 102 shown in FIG. 6 and the HDD 205 shown in FIG. 6, and stores various data in the storage unit 5000 or stored in the storage unit 5000. To read various data.

  The delay time management unit 60 is realized by a command from the CPU 102 shown in FIG. The delay time management unit 60 searches the terminal management DB 5003 (see FIG. 13) using the IP address of the destination terminal as a search key, thereby extracting the corresponding terminal ID. In addition, the delay time management unit 60, in the session management table of the session management DB 5005 (see FIG. 15), shows the delay time indicated by the delay information in the field of the delay time in the record including the extracted terminal ID. Remember and manage.

  The language determination unit 61 is realized by a command from the CPU 102 shown in FIG. The language determination unit 61 searches the language management DB 5008 (see FIG. 19) using the country name included in the destination name of the transmission source terminal as a search key, and determines the language set for the country name as the speaker language. Further, the language determination unit 61 searches the language management DB 5008 (see FIG. 19) using the country name included in the destination name of the transmission destination terminal as a search key, and determines the language set for the country name as the translation language.

<Functional configuration of service provision system>
The service providing system 130 includes a transmission / reception unit 131, a text unit 132, a translation unit 133, and an authentication unit 134. Each of these units functions or functions realized by any one of the constituent elements shown in FIG. 6 operating according to a command from the CPU 201 according to the relay device program expanded from the HD 204 onto the RAM 203. Means.

(Functional configuration of service providing system)
Next, each functional configuration of the service providing system 130 will be described in detail. In the following, in describing each functional configuration of the service providing system 130, among the components illustrated in FIG. 6, the main components for realizing each functional configuration of the service providing system 130 are described. Explain the relationship. Further, since an external cloud service provided for a fee or free of charge is assumed as the service providing system 130, a detailed functional configuration is omitted.

  The transmission / reception unit 131 of the service providing system 130 illustrated in FIG. 7 is realized by a command from the CPU 201 illustrated in FIG. 6 and the network I / F 209 illustrated in FIG. The transmission / reception unit 131 transmits / receives various data (or information) to / from other terminals, devices, or systems via the communication network 2.

  The text unit 132 is realized by a command from the CPU 201 shown in FIG. The text unit 132 provides the text service to the terminal 10 based on the speaker language information received from the terminal 10. That is, the text conversion unit 132 converts the voice data received from the terminal 10 into text and generates text data. The generated text data is returned to the terminal 10 by the transmission / reception unit 131.

  The translation unit 133 is realized by an instruction from the CPU 201 shown in FIG. The translation unit 133 provides a translation service to the terminal 10 based on the translated language information received from the terminal 10. That is, the translation unit 133 translates the text data received from the terminal 10 into a translation language, and generates translated text data. The generated translated text data is returned to the terminal 10 by the transmission / reception unit 131.

  The authentication unit 134 is realized by a command from the CPU 201 shown in FIG. The authentication unit 134 performs authentication processing of the terminal 10 based on the session request received from the terminal 10 and returns an authentication result to the terminal 10.

<< Processing or Operation of Embodiment >>
Next, a processing method in the transmission system 1 according to the present embodiment will be described with reference to FIGS. FIG. 20 is a sequence diagram illustrating processing for managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50. FIG. 21 is a sequence diagram showing processing in a preparation stage for starting a call between a plurality of terminals 10. FIG. 22 is a sequence diagram illustrating a process of narrowing down the relay devices 30. FIG. 23 is a process flow diagram illustrating a process of narrowing down the relay devices 30. FIG. 24 is a diagram illustrating a point calculation state when the relay device 30 performs the narrowing-down process. FIG. 25 is a sequence diagram illustrating processing in which the terminal 10 selects the relay device 30. FIG. 26 is a process flow diagram illustrating a process of selecting the relay device 30 at the terminal. FIG. 27 is a sequence diagram illustrating processing for transmitting and receiving image data and audio data between terminals. FIG. 28 is a sequence diagram showing a process for displaying the translated text on the destination terminal.

  First, a process for managing state information indicating the state of each relay device 30 transmitted from each relay device 30 to the management system 50 will be described with reference to FIG. First, in each relay device 30, the state detection unit 32 shown in FIG. 7 periodically detects the operating state of the relay device 30 that is its own device (steps S1-1 to S4). Then, the transmission / reception unit 31 of each relay device 30 periodically transmits each state information to the management system 50 via the communication network 2 so that the operation state of each relay device 30 is managed in real time on the management system 50 side. (Steps S2-1 to 4). Each state information includes the relay device ID for each relay device 30 and the operating state detected by the state detection unit 32 of the relay device 30 related to each relay device ID. In the present embodiment, the relay devices (30a, 30b, 30d) operate normally and are “online”. On the other hand, although the relay device 30c is in operation, a problem has occurred in the program for executing the relay operation of the relay device 30c, and it is “offline”.

  Next, in the management system 50, the transmission / reception unit 51 receives each state information sent from each relay device 30, and the relay device management DB 5001 of the storage unit 5000 receives the relay device via the storage / read processing unit 59. State information is stored and managed for each ID (steps S3-1 to S4). As a result, the operating status of “online”, “offline”, or “failing” is stored and managed for each relay device ID in the relay device management table as shown in FIG. At this time, the reception date and time when the status information is received by the management system 50 is also stored and managed for each relay device ID. If the status information is not sent from the relay device 30, the field portion of the operation state and the field portion of the reception date and time in each record of the relay device management table shown in FIG. The operation status and the reception date and time at the previous reception are shown.

  Next, with reference to FIG. 21, a description will be given of the transmission / reception processing of each management information in the preparation stage before starting a call between the terminal 10aa and the terminal 10db. In FIG. 21, various types of management information are transmitted and received by the management information session sei shown in FIG.

  First, when the user turns on the power switch 109 shown in FIG. 5, the operation input accepting unit 12 shown in FIG. 7 accepts the power on and turns on the power (step S21). The login request unit 13 automatically transmits login request information indicating a login request from the transmission / reception unit 11 to the management system 50 via the communication network 2 in response to the reception of the power ON (step S22). This login request information includes a terminal ID, a destination name, and a password for identifying the terminal 10aa that is the own terminal as the request source. These terminal ID, destination name, and password are data read from the storage unit 1000 via the storage / read processing unit 19 and sent to the transmission / reception unit 11. When the login request information is transmitted from the terminal 10aa to the management system 50, the management system 50 that is the receiving side can grasp the IP address of the terminal 10ab that is the transmitting side.

  Next, the terminal authentication unit 52 of the management system 50 uses the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys, and the terminal authentication management DB 5002 of the storage unit 5000 (see FIG. 12). ) The terminal authentication unit 52 performs terminal authentication by determining whether the same terminal ID and password are managed in the terminal authentication management DB 5002 (step S23). When it is determined by the terminal authentication unit 52 that the request is a login request from the terminal 10 having a valid use authority, the state management unit 53 stores the terminal ID of the terminal 10aa in the terminal management DB 5003 (see FIG. 13). The destination name and the operating state are stored in association with each other (step S24). Further, the state management unit 53 stores the reception date and time when the login request information is received and the IP address of the terminal 10aa in the terminal management DB 5003 (see FIG. 13) in association with each other (step S24). Accordingly, the terminal management table includes the terminal ID “01aa”, the destination name “Japan Tokyo Office AA terminal”, the operation state “online”, the reception date and time “2009.11.10.13:40”, and the terminal 10aa. An IP address “1.2.1.3” is associated and managed.

  Further, the language determination unit 61 searches the language management DB 5008 of the storage unit 5000 using the area (“Japan”) included in the destination name included in the login request information received via the transmission / reception unit 51 as a search key. The language corresponding to the region (“Japanese”) is determined as the speaker language.

  Then, the transmission / reception unit 51 of the management system 50 sends the authentication result information indicating the authentication result obtained by the terminal authentication unit 52 via the communication network 2 to the request source terminal (terminal 10aa) that made the login request. (Step S25).

  Further, the transmission / reception unit 51 of the management system 50 sends the speaker language information indicating the speaker language determined by the language determination unit 61 to the request source terminal (terminal 10aa) that has made the login request via the communication network 2. Send. When the transmitter / receiver 11 receives the speaker language information, the storage / read processing unit 19 of the request source terminal (terminal 10aa) stores the received speaker language information in the storage unit 1000. In the present embodiment, the case where the terminal authentication unit 52 determines that the terminal has a valid usage right will be described below.

  The terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID “01aa” of the request source terminal (terminal 10aa) that requested the login as a search key. The terminal extraction unit 54 reads and extracts the terminal IDs of candidate destination terminals that can communicate with the request source terminal (terminal 10aa) (step S26). Here, the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminals (terminals 10ab, 10ba, 10db) corresponding to the terminal ID “01aa” of the request source terminal (terminal 10aa) are extracted. Will be.

  Next, the terminal state acquisition unit 55 uses the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminal candidates extracted by the terminal extraction unit 54 as search keys, and the terminal management DB 5003 (FIG. 13). Search). The terminal status acquisition unit 55 reads out the operation status (“offline”, “online”, “online”) for each terminal ID extracted by the terminal extraction unit 54, thereby allowing each terminal (10ab, 10ba, 10db) to be read. The operating state is acquired (step S27).

  Further, the language determination unit 61 uses the terminal IDs (“01ab”, “01ba”, “01db”) of the destination terminal candidates extracted by the terminal extraction unit 54 as a search key, and the terminal management DB 5003 (see FIG. 13). Search for. The language determination unit 61 extracts the regions (“Japan”, “Japan”, “America”) included in the destination name for each terminal ID extracted by the terminal extraction unit 54 obtained as a search result. Then, the language determination unit 61 searches the language management DB 5008 (see FIG. 19) using the extracted country names of the terminal IDs (“Japan”, “Japan”, “USA”) as search keys. The language determination unit 61 converts the language (“Japanese”, “Japanese”, “English”) corresponding to the region obtained as a search result to the translation language of each candidate destination terminal (terminal 10ab, 10ba, 10db). Determine each.

  Next, the transmission / reception unit 51 transmits the destination state information to the request source terminal (terminal 10aa) via the communication network 2 (step S28). The destination status information includes the terminal IDs (“01ab”, “01ba”, “01db”) as search keys and the respective operating statuses (“offline”, “offline”, “10db”) corresponding to the terminal IDs. "Online", "Online"). As a result, the request source terminal (terminal 10aa) has the current operating states (“offline”, “online” of the terminals (10ab, 10ba, 10db) that are candidates for destination terminals that can communicate with the request source terminal (terminal 10aa). ”,“ Online ”).

  In addition, the transmission / reception unit 51 sends the translation language information indicating the translation languages (“Japanese”, “Japanese”, “English”) of each candidate destination terminal (terminals 10ab, 10ba, 10db) via the communication network 2. To the request source terminal (terminal 10aa) (step S28). That is, the translation language of each candidate of the destination terminal (transmission destination terminal) is determined in the management system 50 in a preparation stage before starting a call between the request source terminal (transmission source terminal) and the destination terminal (transmission destination terminal). To the request source terminal (transmission source terminal).

  When the transmission / reception unit 11 receives the translation language information, the storage / reading processing unit 19 of the request source terminal (terminal 10aa) stores the received translation language information in the storage unit 1000. Thereby, the request source terminal (terminal 10aa) can translate the translated languages (“Japanese”, “10”) of the terminals (10ab, 10ba, 10db) that are candidates for the destination terminal that can communicate with the request source terminal (terminal 10aa). Japanese "," English ").

  Further, the terminal extraction unit 54 of the management system 50 searches the destination list management DB 5004 (see FIG. 14) using the terminal ID “01aa” of the request source terminal (terminal 10aa) that requested the login as a search key. As a result, the terminal extraction unit 54 extracts the terminal IDs of other requesting terminals that have registered the terminal ID “01aa” of the requesting terminal (terminal 10aa) as candidates for the destination terminal (step S29). In the destination list management table shown in FIG. 14, the terminal IDs of the other request source terminals to be extracted are “01ab”, “01ba”, and “01db”.

  Next, the terminal state acquisition unit 55 of the management system 50 searches the terminal state management DB 5003 (see FIG. 13) using the terminal ID “01aa” of the request source terminal (terminal 10aa) that has made the login request as a search key. To do. Thereby, the terminal state acquisition unit 55 acquires the operating state of the request source terminal (terminal 10aa) that has requested the login (step S30).

  Then, the transmission / reception unit 51 sends the address to the terminals (10ba, 10db) whose operation status is “online” in the terminal management DB 5003 (see FIG. 13) among the terminals related to the terminal ID extracted in step S29. Status information is transmitted (steps S31-1, 2). The destination status information includes the terminal ID “01aa” of the request source terminal (terminal 10aa) acquired in step S30 and the operating status “online”. When the transmission / reception unit 51 transmits the destination status information to the terminals (10ba, 10db), it is managed by the terminal management table shown in FIG. 13 based on the terminal IDs (“01ba”, “01db”). Refers to the IP address of the terminal being used. As a result, the terminal ID “01aa” of the request source terminal (terminal 10aa) and the operation status “are sent to each of the other destination terminals (terminals 10db and 10ba) that can communicate with the request source terminal (terminal 10aa) that requested the login. “Online”.

  On the other hand, in the other terminals 10 as well, when the user turns on the power switch 109 shown in FIG. 6, the operation input receiving unit 12 shown in FIG. . Thereafter, the other terminals 10 perform the same processing as the processing of steps S22 to S31-1, 2 described above, and thus the description thereof is omitted.

  Next, processing for narrowing down the relay devices 30 will be described with reference to FIG. In FIG. 22, various types of management information are transmitted and received through the management information session sei shown in FIG. In the present embodiment, the request source terminal (terminal 10aa) is the terminal (10ba, 10db) whose operation status is online based on the status information of the terminal received in step S32 among the terminals 10 as destination candidates. You can make a call with at least one of these. Therefore, hereinafter, a case where the user of the request source terminal (terminal 10aa) has selected to start a call with the destination terminal (terminal 10db) will be described.

  First, when the user presses the operation button 108 shown in FIG. 5 and selects the terminal 10db, the operation input reception unit 12 shown in FIG. 7 starts a call with the destination terminal (terminal 10db). The request to be received is accepted (step S41). Then, the transmission / reception unit 11 of the request source terminal (terminal 10aa) includes the terminal ID “01aa” of the terminal 10aa and the terminal ID “01db” of the destination terminal (terminal 10db), and indicates a start request indicating that a call is to be started. Information is transmitted to the management system 50 (step S42). Accordingly, the transmission / reception unit 51 of the management system 50 can receive the start request information and grasp the IP address “1.2.1.3” of the request source terminal (terminal 10aa) that is the transmission source. .

  Then, based on the terminal ID “01aa” of the request source terminal (terminal 10aa) and the terminal ID “01db” of the destination terminal (terminal 10db) included in the start request information, the state management unit 53 determines the terminal management DB 5003 (see FIG. 13). ) Terminal management table. Specifically, the state management unit 53 changes both the field portions of the operation state of the record including the terminal ID “01aa” and the terminal ID “01db” to “busy” (step S43). In this state, the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) have not started a call, but are in a calling state. For this reason, when another terminal 10 tries to make a call with the request source terminal (terminal 10aa) or the destination terminal (terminal 10db), a notification sound or display indicating a so-called busy state is output.

  Next, a process for executing a session for selecting the relay device 30 will be described. First, the selection session ID generation unit 56a shown in FIG. 9 generates a selection session ID used for executing a session for selecting the relay device 30 (step S44). Then, the session management unit 57 associates the selection session ID “se1”, the terminal ID “01aa” of the request source terminal (terminal 10aa), and the terminal ID “01db” of the destination terminal (terminal 10db) with the session management DB 5005. Are stored and managed (step S45).

  Next, the primary narrowing-down unit 56 of the management system 50 relays the relay device 30 for relaying a call between the request source terminal and the destination terminal based on the relay device management DB 5001, the terminal management DB 5003, and the priority management DB 5006. Is first narrowed down (step S46).

  Here, the process in step S46 will be described in more detail with reference to FIGS. First, the terminal IP address extraction unit 56b determines the terminal based on the terminal ID “01aa” of the request source terminal (terminal 10aa) and the terminal ID “01db” of the destination terminal (terminal 10db) included in the start communication information. The management DB 5003 (see FIG. 13) is searched. As a result, the terminal IP address extracting unit 56b extracts the IP addresses (“1.2.1.3”, “1.3.2.4”) of the corresponding terminals (10aa, 10db) (steps). S46-1).

  Next, the primary selection unit 56c selects each relay device ID (111a) of the relay device (30a, 30b, 30d) that is “online” among the operating states of the relay device 30 managed by the relay device management DB 5001. , 111b, 111d) is selected (step S46-2). In addition, the primary selection unit 56c extracts the IP address “1.2.1.3” of the request source terminal (terminal 10aa) and the IP address “1.1” of the destination terminal (terminal 10db) extracted in step S46-1. Based on “3.2.4”, the relay device management DB 5001 (see FIG. 11) is searched. Thereby, the primary selection unit 56c is the same as each dot address in each IP address of the request source terminal and the destination terminal for each dot address of each IP address of the selected relay device (30a, 30b, 30d). It is investigated whether they are different (step S46-3). The IP addresses of the selected relay devices (30a, 30b, 30d) are “1.2.1.2.”, “1.2.2.2”, “1.3.2.2”. It is. Further, the IP addresses of the request source terminal (terminal 10aa) and the destination terminal (terminal 10db) are “1.2.1.3” and “1.3.2.4”.

  Next, the priority determination unit 56d refers to the priority management DB 5006 (see FIG. 16), and determines the point of the address priority for each relay device (30a, 30b, 30d) investigated in the above step 46-3. Is determined (step S46-4). When the result of this determination process is shown in a table, the state shown in FIG. 24 is obtained. FIG. 24 is a diagram illustrating a calculation state of priority points when the relay device 30 performs the narrowing-down process. In FIG. 24, an address priority point, a transmission speed priority point, and an integration point are shown for each relay device ID. The address priority points further indicate points for the request source terminal (terminal 10aa) and points for the destination terminal (terminal 10db) of each relay device 30. The integration point is the sum of the higher one of the two address priority points and the transmission speed priority point.

  In the present embodiment, the IP address “1.2.1.2” of the relay device 30a is different from the IP address “1.2.1.3” of the request source terminal (terminal 10aa). .. ”, the point of the address priority is“ 5 ”as shown in FIG. The IP address “1.2.1.2” of the relay device 30a is “same.different.different.different” with respect to the IP address “1.3.2.4” of the destination terminal (terminal 10db). Therefore, as shown in FIG. 24, the address priority point is “1”. Further, the IP address “1.2.2.2” of the relay device 30b is “same. Same. Different. Different” with respect to the IP address “1.2.1.3” of the request source terminal (terminal 10aa). Therefore, the address priority point is “3”. Further, the IP address “1.2.2.2” of the relay device 30b is “same. Different. Same. Different” with respect to the IP address “1.3.2.4” of the destination terminal (terminal 10db). Therefore, the point of address priority is “1”. Further, the IP address “1.3.2.2” of the relay device 30d is “same. Different. Different. Different” with respect to the IP address “1.2.1.3” of the request source terminal (terminal 10aa). Therefore, the address priority point is “1”. Further, the IP address “1.3.2.2” of the relay device 30d is “same. Same. Same. Different” with respect to the IP address “1.3.2.4” of the destination terminal (terminal 10db). Therefore, the address priority point is “5”.

  Next, returning to FIG. 23, the priority determination unit 56d determines the priority management DB 5006 (FIG. 17) based on the maximum data transmission rate of each relay device 30 managed by the relay device management DB 5001 (see FIG. 11). Search). Thereby, the priority determination part 56d determines the point of transmission speed priority for every relay apparatus (30a, 30b, 30d) narrowed down by the primary narrowing-down process by said step S46-2 (step S46-5). ). In the present embodiment, as shown in FIG. 11, the maximum data transmission rate of the relay device 30a is 100 (Mbps). Therefore, referring to the transmission rate priority shown in FIG. The degree is 3 points. Similarly, if the maximum data transmission rate of the relay device 30b is calculated to be 1000 (Mbps), the transmission rate priority is 5 points. Similarly, if the maximum data transmission rate of the relay device 30d is calculated to be 10 (Mbps), the transmission rate priority is 1 point.

  Next, for each relay device, the primary selection unit 56c selects the higher two points having the highest point among the integrated points obtained by integrating the higher points of the terminals and the transmission speed priority points at the address priority points. The relay device 30 is selected (step 46-6). In the present embodiment, since the relay device IDs (111a, 111b, 111d) have the integration points “8”, “8”, and “6”, respectively, the relay device 30a related to the relay device ID “111a”, and the relay The relay device 30b related to the device ID “111b” is selected.

  When the narrowing-down process in step S46 is completed, the transmission / reception unit 51 illustrated in FIG. 7 transmits the number of narrowed relay apparatuses 30 to the destination terminal (terminal 10db) via the communication network 2. Relay device narrowing information is transmitted (step S47). The relay device narrowing information includes the number “2” of the relay devices 30 narrowed down in the step 46, the terminal ID “01aa” of the request source terminal (terminal 10aa), and the selection session ID “se1”. It is. Thereby, the terminal 10db can grasp how many relay devices 30 are in the execution of the session with the selection session ID “se1” and which terminal 10 has requested to start a call. . Also, the terminal 10db can grasp the IP address “1.1.1.2” of the management system 50 that is the transmission source of the relay device narrowing information.

  Then, the terminal 10db transmits reception completion information indicating that reception of the relay device narrowing information is completed from the transmission / reception unit 11 to the management system 50 via the communication network 2 (step S48). This reception completion information includes the session ID “se1”. As a result, the management system 50 completes the transmission of the number of relay apparatuses being executed with the session ID “se1”, and the IP address “1.3.2.4” of the destination terminal (terminal 10db) that is the transmission source. Can be grasped.

  Next, a process in which the destination terminal (terminal 10db) selects the relay device 30 will be described with reference to FIGS. In FIG. 25, various types of management information are transmitted and received through the management information session sei shown in FIG.

  First, before starting a video conference call, the management system 50 provides pre-relay request information for requesting relay in advance to each of the relay devices (30a, 30b) narrowed down in step S46. Transmit (steps S61-1, 2). The pre-relay request information includes the session ID “se1”, the IP address “1.2.1.3” of the request source terminal (terminal 10aa), and the IP address “1.3.2” of the destination terminal (terminal 10db). .4 "is included. Thereby, each relay apparatus (30a, 30b) can grasp which selection session it belongs to, what the request source terminal is, and what the destination terminal is. Further, each relay device (30a, 30b) can grasp the IP address “1.1.1.2” of the management system 50 that is the transmission source of the prior relay request information.

  Next, each relay device transmits each of the relay devices as its own device before the start of the call to the request source terminal (terminal 10aa) grasped in steps S61-1, 2 from the transmission / reception unit 31 via the communication network 2. The prior transmission request information is transmitted to (step S62-1, 2). The pre-transmission request information indicates that pre-transmission information including ping (Packet Internet Groper) described later is transmitted. This pre-transmission information includes the session ID “se1”. As a result, the request source terminal (terminal 10aa) can grasp that the pre-transmission information is transmitted to each relay device (30a, 30b) in the selection process of the relay device 30 executed with the session ID “se1”. Further, the request source terminal (terminal 10aa) receives the IP addresses (“1.2.1.2”, “1.2.2.”) Of the relay apparatuses (30a, 30b) that are the transmission sources of the pre-transmission request information. 2)).

  Note that the IP address of the destination terminal is not notified directly from the management system 50 to the requesting terminal, so that each terminal 10 is not notified of the IP address of the other terminal 10 to ensure security. Because.

  Next, the request source terminal (terminal 10aa) transmits the pre-transmission information from the transmission / reception unit 11 to each relay device (30a, 30b) via the communication network 2 (steps S63-1, 2). Prior to the transmission of image data and audio data, this pre-transmission information is transmitted to the destination terminal via each relay device instead of these image data and audio data, so that the transmission from the request source terminal to the destination terminal is transmitted. Information used to measure the time required until reception. In addition, the advance transmission information includes a ping for confirming that the request source terminal, the relay device, and the destination terminal are connected to be communicable, a transmission date and time when the prior transmission information is transmitted from the request source terminal, and The session ID “se1” is included. As a result, each relay apparatus can grasp that the pre-transmission information has been sent in the execution of the session with the selection session ID “se1”, and the IP of the request source terminal that is the transmission source of this pre-transmission information. The address “1.2.1.3” can be grasped.

  Next, each relay device (30a, 30b) receives the IP address “1.3.2.4 of the destination terminal (terminal 10db) included in the pre-relay request information received in steps S61-1, 2 above. ", The above-mentioned pre-transmission information is relayed (steps S64-1, 2). Thereby, the destination terminal (terminal 10db) can grasp that the pre-transmission information has been sent in the execution of the session with the session ID “se1”. Further, the destination terminal (terminal 10db) receives the IP address (“1.2.1.2”, “1.2.2” of the relay device (30a, 30b) that is the transmission source (relay source) of this pre-transmission information. .2 ").

  Next, the final narrowing unit 16 of the destination terminal (terminal 10db) finally narrows down to one relay device 30 that relays image data and audio data in a video conference call based on the pre-transmission information (step S65). ).

  Here, the process in step S65 will be described in more detail with reference to FIGS. First, the measurement unit 16a of the final narrowing unit 16 illustrated in FIG. 8 receives the reception when the transmission / reception unit 11 of the terminal 10db receives, for each pre-transmission information relayed by each relay device (30a, 30b). The date and time are measured (step S65-1). Next, the calculation unit 16b determines, for each pre-transmission information whose reception time is measured, from the transmission of each pre-transmission information based on the difference between the reception date and time and the transmission date and time included in the pre-transmission information. The time required until reception is calculated (step S65-2). Next, the final selection unit 16c determines whether or not all pieces of pre-transmission information corresponding to the number “2” of relay devices 30 scheduled to be relayed have been received in the execution of the session with the session ID “se1” (Step S65). -3). If not all have been received (NO), the final selection unit 16c determines whether or not a predetermined time (here, 1 minute) has elapsed since the terminal 10db received the pre-transmission information (step S65-4). ). Further, if the predetermined time has not elapsed (NO), the process returns to step S65-1. On the other hand, when all of the above-mentioned steps 65-3 have been received (YES), the final selection unit 16c transmits the prior transmission information that has required the shortest required time among the required times calculated by the calculation unit 16b so far. One relay device 30 relayed is selected (step S65-5). Similarly, in step S65-4, if the predetermined time has elapsed (YES), the final selection unit 16c transmits in advance the shortest required time among the required times calculated by the calculation unit 16b so far. One relay device 30 that relayed the information is selected (step S65-5). In the present embodiment, the relay device 30a is selected on the assumption that the pre-transmission information relayed by the relay device 30a has a shorter time from transmission to reception than the pre-transmission information relayed by the relay device 30b. Is shown.

  Next, the destination terminal (terminal 10db) transmits selection information indicating that the relay device 30a has been selected from the transmission / reception unit 11 to the management system 50 via the communication network 2 (step S66). This selection information includes the session ID “se1” and the relay device ID “111a” of the selected relay device 30a. Thereby, the management system 50 can grasp that the relay device 30a is selected in the execution of the session with the session ID “se1”. Further, the management system 50 can grasp the IP address “1.3.2.4” of the destination terminal (terminal 10db) that is the transmission source of the selection information.

  Next, in the session management table, the session management unit 57 finally selects the relay device ID “111a” of the relay device 30a selected as one in the field portion of the relay device ID of the record including the session ID “se1”. Is stored and managed (step S67). Then, the transmission / reception unit 51 of the management system 50 transmits relay start request information indicating a request to start relaying to the relay device 30a via the communication network 2 (step S68). In this relay start request information, each IP address (“1.2.1.3”, “1.3.2.4”) of the request source terminal (terminal 10aa) and destination terminal (terminal 10db) to be relayed is included. It is included. Thereby, since the relay device 30a can grasp that the relay device 30a which is its own device has been selected, three image data of low resolution, medium resolution, and high resolution, and audio data are transmitted between the terminals. A session for making a call is established (step S69). Therefore, the terminals (10aa, 10db) can start a video conference call.

  In step S47, the management system 50 transmits the relay device narrowing information to the request source terminal. Thereafter, in steps S64-1 and S64-1, the information transmission source and the reception source are the request source terminal and the request source terminal. You may make it replace with a destination terminal. Thereby, the request source terminal (terminal 10aa) can perform the relay device selection process instead of step S65, and can also transmit the selection information instead of step S66.

  Next, a process of transmitting and receiving image data and audio data to perform a video conference call between the request source terminal and the destination terminal will be described with reference to FIGS. Note that image data and audio data are transmitted and received in one-way processing for transmitting image data and audio data from the terminal 10aa to the terminal 10db and in reverse processing for transmitting image data and audio data from the terminal 10db to the terminal 10aa. The detection of delay time and the like are the same processing. For this reason, the one-way communication will be described, and the reverse communication will be omitted.

  First, the request source terminal transmits image data of the subject imaged by the imaging unit 14a and audio data of the audio input by the audio input unit 15a from the transmission / reception unit 11 to the relay device 30a by the image / audio data session sed. (Step S81). In the present embodiment, the high-quality image data including the low resolution, the medium resolution, and the high resolution shown in FIG. 3 and the audio data are transmitted. Thereby, in the relay device 30a, the transmission / reception unit 31 receives the image data and the audio data of the three resolutions. Then, the data quality confirmation unit 33 searches the change quality management DB 3001 using the IP address “1.3.2.4” of the destination terminal as a search key, and extracts the image quality of the corresponding image data to be relayed. The image quality of the image data to be relayed is confirmed (step S82). In this embodiment, the image quality of the confirmed image data is “high image quality”, which is the same as the image quality of the image data received by the transmission / reception unit 31. Therefore, the relay device 30a transfers the image data with the same image quality and the sound data with the same sound quality to the destination terminal (terminal 10db) by the image / audio data session sed (step S83). As a result, the destination terminal (terminal 10db) receives the high-quality image data and the audio data composed of the low resolution, the medium resolution, and the high resolution at the transmission / reception unit 11. Then, the display control unit 17 can combine the three image quality image data to display an image on the display 120, and the sound output unit 15b can output sound based on the sound data.

  Next, the delay detection unit 18 of the terminal 10db detects the reception delay time of the image data received by the transmission / reception unit 11 at regular time intervals (for example, every second) (step S84). In the present embodiment, the following description is continued for a case where the delay time is 200 (ms).

  The transmission / reception unit 11 of the destination terminal (terminal 10db) sends delay information indicating the delay time “200 (ms)” to the management system 50 via the communication network 2 by the management information session sei shown in FIG. Transmit (step S85). As a result, the management system 50 can grasp the delay time and the IP address “1.3.2.4” of the terminal 10db that is the transmission source of the delay information.

  Next, the delay time management unit 60 of the management system 50 searches the terminal management DB 5003 (see FIG. 13) using the IP address “1.3.2.4” of the destination terminal (terminal 10db) as a search key. Then, the corresponding terminal ID “01db” is extracted. Also, the delay time management unit 60 stores the delay time “200 (ms)” indicated by the delay information in the delay time field portion of the record of the terminal ID “01db” in the session management table. Manage (step S86).

  Next, the quality determination unit 58 searches the quality management DB 5007 (see FIG. 18) using the delay time “200 (ms)” as a search key, and extracts the image quality “medium image quality” of the corresponding image data. The image quality is determined as “medium image quality” (step S87).

  Next, the transmission / reception unit 51 searches the relay device management DB 5001 (see FIG. 11) using the relay device ID “111a” associated with the terminal ID “01db” in the session management table as a search key. Thereby, the transmission / reception unit 51 extracts the IP address “1.2.1.2” of the relay device 30a corresponding to the search key (step S88). Then, the transmission / reception unit 51 transmits the quality “medium image quality” of the image data determined in step 87 to the relay device 30a via the communication network 2 by the management information session sei shown in FIG. Information is transmitted (step S89). This quality information includes the IP address “1.3.2.4” of the destination terminal (terminal 10db) used as the search key in step S86. Accordingly, in the relay device 30a, the change quality management unit 34 stores the IP address “1.3.2.4” of the destination terminal 10 (terminal 10db) and the image data to be relayed in the change quality management DB 3001. The image quality “medium image quality” is stored and managed in association (step S90).

  Next, the terminal 10aa continues to the relay device 30a through the image / audio data session sed in the same manner as in the above step S81, and the high-quality image data including the low resolution, medium resolution, and high resolution, and Audio data is transmitted (step S91). In the relay device 30a, as in step S82, the data quality confirmation unit 33 uses the IP address “1.3.2.4” of the destination terminal (terminal 10db) as a search key to change the quality management DB 3001 (see FIG. 10). ) The data quality confirmation unit 33 extracts the image quality “medium image quality” of the image data to be relayed corresponding to the search key, thereby confirming the image quality of the image data to be relayed (step S92). In this embodiment, the image quality of the confirmed image data is “medium image quality”, which is lower than the image quality “high image quality” of the image data received by the transmission / reception unit 31. Therefore, the data quality changing unit 35 changes the image quality of the image data by suppressing the image quality of the image data from “high image quality” to “medium image quality” (step S93).

  Then, the image / sound data session sed changes the image data in which the image quality of the image data is changed to “medium image quality” and the sound quality of the sound to the terminal 10 db via the communication network 2. Audio data that has not been transmitted is transmitted (step S94). As a result, the destination terminal (terminal 10db) receives the image data and audio data of medium quality composed of two of low resolution and medium resolution by the transmission / reception unit 11. Then, the display control unit 17 can combine the two resolution image data to display the image on the display 120, and the audio output unit 15b can output the audio based on the audio data.

  As described above, when a reception delay occurs at the destination terminal (terminal 10db) that receives the image data, the relay device 30a changes the quality of the image and gives a strange feeling to the person participating in the video conference. Can not be.

  Next, the process of displaying the translated text on the transmission destination terminal will be described using FIG. In the following, it is assumed that the terminal 10aa is a transmission source terminal and the terminal 10db is a transmission destination terminal. Further, the language (speaker language) of the terminal 10aa is Japanese, and the language (translation language) of the terminal 10db is English. In this case, Japanese speech (voice) by the user of the terminal 10aa is displayed as an English translation text on the display 120db of the terminal 10db.

  When the terminal 10aa and the terminal 10db are connected to hold a conference, the terminal 10aa activates an application that realizes the function of the service request unit 20 (step S101). Hereinafter, the function of this application will be described as the operation of the service request unit 20. The activation of the application corresponds to the start of operation of the service request unit 20.

  When starting the operation, the service request unit 20 first transmits a session request to the service providing system 130 (step S102). At this time, the service request unit 20 transmits information corresponding to the service providing system 130 together with the session request. For example, when the service providing system 130 is an external cloud service, it is conceivable to transmit the user ID and password of the terminal 10aa.

  Upon receiving the session request from the terminal 10aa, the authentication unit 134 of the service providing system 130 executes authentication processing using the received information (user ID and password) (step S103). When the authentication fails, the service providing system 130 returns a message to that effect to the terminal 10aa. Upon receiving this authentication result (authentication failure), the terminal 10aa displays the received authentication result on the display 120aa and notifies the user that the authentication has failed.

  On the other hand, when the authentication is successful, the authentication unit 134 returns a session establishment response (access token) to the terminal 10aa (step S104).

  When the service request unit 20 of the terminal 10aa receives the session establishment response, a session between the terminal 10aa and the service providing system 130 is established. When the session is established, the service request unit 20 reads the speaker language information and the translation language information received in advance from the management system 50 from the storage unit 1000 (step S105).

  Then, the service request unit 20 transmits the speaker language information indicating the speaker language (Japanese), which is the language of the terminal 10aa (sender terminal), and the text service start request to the service providing system 130 ( Step S106). If the voice data is stored in the storage unit 1000 at this time, the service request unit 20 may transmit the voice data to the service providing system 130 together with the text service start request.

  When receiving the text service start request, the text unit 132 of the service providing system 130 starts providing the text service to the terminal 10aa (step S107).

  Further, the service request unit 20 transmits the translation language information indicating the translation language (English) which is the language of the terminal 10db (transmission destination terminal) and the translation service start request to the service providing system 130 (step S108). If text data is stored in the storage unit 1000 at this time, the service request unit 20 may transmit the text data to the service providing system 130 together with the translation service start request.

  When receiving the translation service start request, the translation unit 133 of the service providing system 130 starts providing the translation service to the terminal 10aa (step S109).

  After that, when voice data is input, the service request unit 20 of the terminal 10aa reads the input voice data from the storage unit 1000 (step S110), and transmits the read voice data to the service providing system 130 (step S111). ).

  When receiving the voice data from the terminal 10aa, the text converting unit 132 of the service providing system 130 converts the received voice data into the text data of the speaker language (Japanese) (step S112). Then, the text conversion unit 132 returns the obtained text data to the terminal 10aa (step S113). That is, the text unit 132 provides a text service to the terminal 10aa.

  In addition, when receiving the text data from the service providing system 130, the service request unit 20 of the terminal 10aa stores the received text data in the storage unit 1000, and reads the stored text data from the storage unit 1000 (step S114). Then, the service request unit 20 transmits the read text data to the service providing system 130 (step S115).

  When receiving the text data from the terminal 10aa, the translation unit 133 of the service providing system 130 translates the received text data into a translation language (English) (step S116), and returns the obtained translated text data to the terminal 10aa. (Step S117). That is, the translation unit 133 provides a translation service to the terminal 10aa.

  When receiving the translation text data from the service providing system 130, the service request unit 20 of the terminal 10aa stores the received translation text data in the storage unit 1000, and reads the stored translation text data from the storage unit 1000 (step S118). . Then, the service request unit 20 transmits the read translated text data to the terminal 10db via the management system 50 (step S119).

  When receiving the translation text data from the terminal 10aa, the display control unit 17 of the terminal 10db displays the translation text corresponding to the received translation text data on the display 120db (step S120).

  The processes in steps S110 to S120 described above are executed each time voice data is input to the terminal 10aa after the provision of the text conversion service and the translation service is started. Thereby, the Japanese speech (voice) by the user of the terminal 10aa can be displayed on the display 120db of the terminal 10db as English translation text.

  In the example of FIG. 28, the processing when the terminal 10aa is the transmission source terminal has been described. However, the same processing as described above is possible when the terminal 10db is the transmission source terminal. Thereby, the English speech (voice) by the user of the terminal 10db can be displayed on the display 120aa of the terminal 10aa as Japanese translation text.

  In the example of FIG. 28, the processing when the users of the two terminals 10 hold the conference has been described, but the same applies to the case where the users of three or more terminals 10 hold the conference. Processing is possible.

  For example, consider a case where users of terminals 10aa, 10ab, and 10db hold a conference. It is assumed that the terminal 10aa is a transmission source terminal and the terminals 10ab and 10db are transmission destination terminals. Further, it is assumed that the language (speaker language) of the terminal 10aa is Japanese, and the languages (translation languages) of the terminals 10ab and 10db are Japanese and English, respectively.

  In this case, the terminal 10aa transmits translation language information indicating the translation language of the terminals 10ab and 10db to the service providing system 130 (step S108). When the translation unit 133 of the service providing system 130 receives text data from the terminal 10aa (step S115), the received text data is converted into the translation language (Japanese) of the terminal 10ab, the translation language (English) of the terminal 10db, (Step S116). Then, the translation unit 133 returns the obtained two translated text data to the terminal 10aa as translated text data of the terminals 10ab and 10db (step S117). The terminal 10aa transmits the translated text data of the terminal 10ab to the terminal 10ab, and transmits the translated text data of the terminal 10db to the terminal 10db (step S119). As a result, Japanese speech (speech) by the user of the terminal 10aa can be displayed on the display 120ab of the terminal 10ab as Japanese translation text, and can be displayed on the display 120db of the terminal 10db as English translation text. it can.

  If the language of the transmission source terminal (speaker language) matches the language of the transmission destination terminal (translation language) as in the terminals 10aa and 10ab, the translation unit 133 converts the text data into You may reply without translation. When there are a plurality of destination terminals having the same translation language, the translation unit 133 converts the translation text data obtained by one translation into the translation text data of the plurality of destination terminals having the same translation language. You may reply as In any case, the translation processing by the translation unit 133 can be reduced, and the load on the service providing system 130 can be reduced.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, even if the environment of the LAN 2 such as the IP address of the relay device 30 in the communication network 2 can be grasped, it is difficult to grasp the entire environment of the Internet 2i. It is. For this reason, first, the environment information that can be grasped is narrowed down to two or more of 30 relay devices that relay image data and audio data. Then, before actually transmitting and receiving image data and audio data among the plurality of terminals 10, advance transmission information is transmitted and received instead of image data and audio data. As a result, there is an effect that it can be narrowed down to one relay device 30 that can actually relay the pre-transmission information earliest.

  That is, by selecting the relay device 30 to which the upper two or more IP addresses close to any of the IP addresses of the terminal 10 are assigned, two or more candidates for the relay device 30 to be finally used are left. be able to. Thereafter, the transmission / reception information between the request source terminal and the destination terminal is actually transmitted / received between the requesting terminal and the destination terminal via each relay apparatus 30 as a candidate, so that the requirement required for transmission / reception among the two or more candidate relay apparatuses 30 is required. It is possible to narrow down to the relay device 30 that relays the pre-transmission information with the shortest time. Therefore, it is possible to realize transmission and reception of high-quality image data or audio data to the maximum in the current communication network 2 environment.

  In the present embodiment, when narrowing down the relay devices 30, not only the relay device 30 having an IP address close to the IP address of the terminal 10 that conducts the video conference is preferentially selected, but the maximum data transmission in each relay device 30 is also performed. In consideration of the speed, two or more relay devices 30 are selected. Thereby, the effect that the candidate of the relay apparatus 30 according to the environment of the actual communication network 2 can be narrowed down is produced.

  Furthermore, in this embodiment, when narrowing down the relay devices 30, the candidates for the relay devices 30 can be narrowed down further in accordance with the environment of the actual communication network 2 in order to narrow down the relay devices 30 whose operation state is online. it can.

  In this embodiment, the translation language for translating audio data is automatically determined as the most used language in the area where the transmission destination terminal is installed. For example, when the destination terminal is installed in the United States, the translation language is automatically determined to be English. In general, the language used by the user of the destination terminal is likely to be the most used language in the area where the destination terminal is installed. Therefore, by determining the translation language as in the present embodiment, the translation language can be automatically determined as the language used by the user of the destination terminal.

[Second Embodiment]
Hereinafter, the second embodiment of the present invention will be described with reference to FIG. FIG. 29 is a sequence diagram illustrating processing in which a terminal selects a relay device according to the second embodiment of the present invention. Note that since the present embodiment and the first embodiment are different only in the processing and operation in FIG. 29 and FIG. 25, only the difference in processing and operation will be described here, as shown below. To do.

<< Processing or Operation of Embodiment >>
In steps S61-1, S62-1, S61-2, and S62-2 in FIG. 25 of the first embodiment, the request source terminal (terminal 10aa) is sent from the management system 50 via each relay device (30a, 30b). ) Request to send advance request information. On the other hand, in the present embodiment, the two are different in that the prior request information is transmitted directly from the management system 50 to the request source terminal (terminal 10aa).

  That is, in this embodiment, as shown in FIG. 29, first, the management system 50 transmits the pre-transmission request information to the request source terminal (terminal 10aa) (step S161). In this case, the pre-transmission request information includes the session ID “se1”, the IP addresses (“1.2.1.2” and “1.2.2.2”) of the relay devices that are primarily narrowed down, and The IP address “1.3.2.4” of the destination terminal is included. Thereby, the request source terminal (terminal 10aa) belongs to which selection session, the relay device 30 that is the final narrowing candidate by the primary narrowing down, and the IP address of the destination terminal (terminal 10db) You can know what the address is. Further, the request source terminal (terminal 10aa) can grasp the IP address “1.1.1.2” of the management system 50 that is the transmission source of the advance transmission request information.

  Next, the request source terminal (terminal 10aa) transmits the advance transmission information to the relay devices (30a, 30b) (steps S162-1, 2). The advance transmission information includes the ping, the transmission date and time when the advance transmission information is transmitted from the request source terminal (terminal 10aa), and the session ID “se1”, as in Steps S63-1 and S63-2 in the first embodiment. include. The pre-transmission information further includes the IP address “1.3.2.4” of the destination terminal (terminal 10db). Thereby, each relay apparatus (30a, 30b) can grasp | ascertain that prior transmission information was sent in execution of the session in session ID "se1" for selection. Further, each relay device (terminals 30a and 30b) can grasp the IP address “1.2.1.3” of the request source terminal (terminal 10aa) that is the transmission source of this pre-transmission information. Further, each relay device (terminals 30a and 30b) can grasp the IP address “1.3.2.4” of the destination terminal (terminal 10db) that is the transmission destination of the pre-transmission information. Therefore, each relay apparatus (30a, 30b) can transmit (or transfer) the pre-transmission information to the acquired IP address of the destination terminal (terminal 10db).

  Note that steps S163-1, 2 to S168 in FIG. 29 are the same processes as S64-1, 2 to S69 in FIG. 25 in the first embodiment, and thus the description thereof is omitted.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, the same effects as the main effects of the first embodiment can be obtained.

  However, in this embodiment, since the requesting terminal is notified of the IP address of the destination terminal, it is inferior in terms of security compared to the first embodiment. However, for example, in the first embodiment, when a firewall is installed in order to ensure the confidentiality of the LAN 2a, the relay device 30b transmits in advance to the request source terminal (terminal 10aa). Request information may not be delivered. On the other hand, in this embodiment, since the advance transmission request information is not sent from the relay device 30 side to the terminal 10aa, the problem that the advance transmission request information cannot be delivered does not occur. In this case, the management system 50 is on the communication network outside the firewall, but the terminal 10aa and the management system 50 are already connected as shown in FIG. For this reason, the management system 50 can deliver the advance transmission request information to the terminal 10aa through the firewall.

[Supplement of Embodiment]
The relay device 30, the management system 50, the program providing system 90, the maintenance system 100, and the service providing system 130 in the above embodiment may be constructed by a single computer. Further, the relay device 30, the management system 50, the program providing system 90, the maintenance system 100, and the service providing system 130 may be constructed by a plurality of computers that are arbitrarily assigned by dividing each unit (function or means). Good. In addition, when the program providing system 90 is constructed by a single computer, the program transmitted by the program providing system 90 may be transmitted by dividing it into a plurality of modules. It may be transmitted. Further, when the program providing system 90 is constructed by a plurality of computers, a plurality of modules may be divided and transmitted from each computer.

  Further, a recording medium such as a CD-ROM in which a program such as a terminal program is stored, the HD 204, and a program providing system 90 including the HD 204 are provided to a user or the like as a program product (Program Product). Used when

  Furthermore, in the above-described embodiment, as an example of the image quality of the image data relayed by the relay device 30 using the changed quality management table shown in FIG. 10 and the quality management table shown in FIG. It managed by paying attention to the resolution of the data image. However, the quality is not limited to this. As another example of the quality, management may be performed by paying attention to the depth of image quality of the image data, the sampling frequency in the sound of the sound data, the bit length in the sound of the sound data, and the like. Further, the audio data may be transmitted and received by being divided into data of three types of resolution (high resolution, medium resolution, and low resolution).

  11, FIG. 13 and FIG. 15 manage the reception date and time, but the present invention is not limited to this, and at least the reception time of the reception date and time may be managed.

  Furthermore, in the above embodiment, the IP address of the relay device is managed in FIG. 11 and the IP address of the terminal in FIG. 13, but the present invention is not limited to this. If the relay device specifying information for specifying the relay device 30 on the communication network 2 or the terminal specifying information for specifying the terminal 10 on the communication network 2, each FQDN (Fully Qualified Domain Name) is managed. May be. In this case, an IP address corresponding to the FQDN is acquired by a known DNS (Domain Name System) server. It is expressed as “relay device connection destination information indicating a connection destination to the relay device 30 on the communication network 2” as well as “relay device specifying information for specifying the relay device 30 in the communication network 2”. Also good. Further, it may be expressed as “relay device destination information indicating a destination to the relay device 30 on the communication network 2”. In addition to “terminal specifying information for specifying the terminal 10 on the communication network 2”, “terminal connection destination information indicating a connection destination to the terminal 10 on the communication network 2” or “terminal on the communication network 2. It may be expressed as “terminal destination information indicating a destination to 10”.

  In the above embodiment, a video conference system has been described as an example of the transmission system 1. However, the present invention is not limited to this, and a telephone system such as an IP (Internet Protocol) telephone or an Internet telephone may be used. . Further, the transmission system 1 may be a car navigation system. In this case, for example, one of the terminals 10 corresponds to a car navigation device mounted on a car, and the other of the terminals 10 is mounted on a management terminal or management server of a management center that manages car navigation, or another car. This corresponds to a car navigation device. The transmission system 1 may be a mobile phone communication system. In this case, for example, the terminal 10 corresponds to a mobile phone.

  In the above embodiment, image data and audio data have been described as an example of content data. However, the present invention is not limited to this, and touch data may be used. In this case, the feeling that the user touched on one terminal side is transmitted to the other terminal side. Further, the content data may be smell data. In this case, the odor (odor) on one terminal side is transmitted to the other terminal side. The content data may be at least one of image data, audio data, tactile data, and olfactory data.

  Further, in the above embodiment, the case where a video conference is performed by the transmission system 1 has been described. However, the present invention is not limited to this, and a general conversation such as a meeting, a family or a friend, or information in one direction is performed. It may be used for presentation.

[Third Embodiment]
Hereinafter, the third embodiment of the present invention will be described with reference to FIG. FIG. 30 is a sequence diagram illustrating processing for displaying a translated text on a transmission destination terminal according to the third embodiment of the present invention. Note that the present embodiment and the first embodiment are different only in the processing and operation in FIG. 28 and FIG. 30, respectively, so only the differences in processing and operation will be described here as shown below. To do.

<< Processing or Operation of Embodiment >>
In FIG. 28 of the first embodiment, it is assumed that the transmission source terminal communicates with the service providing system 130 and receives service provision. On the other hand, in the present embodiment, it is assumed that the transmission destination terminal communicates with the service providing system 130 and receives service provision.

  The process of displaying the translated text on the transmission destination terminal will be described with reference to FIG. In the following, it is assumed that the terminal 10aa is a transmission source terminal and the terminal 10db is a transmission destination terminal. Further, the language (speaker language) of the terminal 10aa is Japanese, and the language (translation language) of the terminal 10db is English. In this case, Japanese speech (voice) by the user of the terminal 10aa is displayed as an English translation text on the display 120db of the terminal 10db.

  When the terminal 10aa and the terminal 10db are connected to hold a conference, the terminal 10db starts an application that realizes the function of the service request unit 20 (step S131). Hereinafter, the function of this application will be described as the operation of the service request unit 20. The activation of the application corresponds to the start of operation of the service request unit 20.

  When starting the operation, the service request unit 20 first transmits a session request to the service providing system 130 (step S132). At this time, the service request unit 20 transmits information corresponding to the service providing system 130 together with the session request. For example, when the service providing system 130 is an external cloud service, it is conceivable to transmit the user ID and password of the terminal 10db.

  When receiving the session request from the terminal 10db, the authentication unit 134 of the service providing system 130 executes authentication processing using the received information (user ID and password) (step S133). When the authentication fails, the service providing system 130 returns a message to that effect to the terminal 10db. Upon receiving this authentication result (authentication failure), the terminal 10db displays the received authentication result on the display 120db and notifies the user that the authentication has failed.

  On the other hand, when the authentication is successful, the authentication unit 134 returns a session establishment response (access token) to the terminal 10db (step S134).

  When the service request unit 20 of the terminal 10db receives the session establishment response, a session between the terminal 10db and the service providing system 130 is established. When the session is established, the service request unit 20 reads the speaker language information and the translation language information received in advance from the management system 50 from the storage unit 1000 (step S135).

  Then, the service request unit 20 transmits the speaker language information indicating the speaker language (Japanese), which is the language of the terminal 10aa (sender terminal), and the text service start request to the service providing system 130 ( Step S136). If the voice data is stored in the storage unit 1000 at this time, the service request unit 20 may transmit the voice data to the service providing system 130 together with the text service start request.

  When receiving the text service start request, the text unit 132 of the service providing system 130 starts providing the text service to the terminal 10db (step S137).

  In addition, the service request unit 20 transmits the translation language information indicating the translation language (English) that is the language of the terminal 10db (transmission destination terminal) and the translation service start request to the service providing system 130 (step S138). If text data is stored in the storage unit 1000 at this time, the service request unit 20 may transmit the text data to the service providing system 130 together with the translation service start request.

  When receiving the translation service start request, the translation unit 133 of the service providing system 130 starts providing the translation service to the terminal 10db (step S139).

  Thereafter, when the service request unit 20 of the terminal 10db receives voice data from the terminal 10aa via the management system 50 (step S140), the service request unit 20 transmits the received voice data to the service providing system 130 (step S141). At this time, the audio output unit 15b of the terminal 10db may output an audio signal related to the received audio data to the speaker 115 and output the audio from the speaker 115 by a loopback process.

  When receiving the voice data from the terminal 10db, the text converting unit 132 of the service providing system 130 converts the received voice data into text data in the speaker language (Japanese) (step S142). Then, the text unit 132 returns the obtained text data to the terminal 10db (step S143). That is, the text unit 132 provides the text service to the terminal 10db.

  When receiving the text data from the service providing system 130, the service request unit 20 of the terminal 10db stores the received text data in the storage unit 1000, and reads the stored text data from the storage unit 1000 (step S144). The service request unit 20 transmits the read text data to the service providing system 130 (step S145).

  When receiving the text data from the terminal 10db, the translation unit 133 of the service providing system 130 translates the received text data into the translated language (English) (step S146), and returns the obtained translated text data to the terminal 10db. (Step S147). That is, the translation unit 133 provides a translation service to the terminal 10db.

  When receiving the translation text data from the service providing system 130, the service request unit 20 of the terminal 10db stores the received translation text data in the storage unit 1000, and reads the stored translation text data from the storage unit 1000 (step S148). . The service request unit 20 displays the translation text corresponding to the read translation text data on the display 120db (step S149).

  The processes in steps S141 to S149 described above are executed each time voice data is received from the terminal 10aa after the provision of the text conversion service and the translation service is started. Thereby, the Japanese speech (voice) by the user of the terminal 10aa can be displayed on the display 120db of the terminal 10db as English translation text.

  In the example of FIG. 30, the processing when the terminal 10aa is the transmission source terminal has been described. However, the same processing as described above can be performed when the terminal 10db is the transmission source terminal. Thereby, the English speech (voice) by the user of the terminal 10db can be displayed on the display 120aa of the terminal 10aa as Japanese translation text.

  Further, in the example of FIG. 30, the processing when the users of the two terminals 10 hold the conference has been described, but the same applies to the case where the users of the three or more terminals 10 hold the conference. Processing is possible.

  For example, consider a case where users of terminals 10aa, 10ab, and 10db hold a conference. It is assumed that the terminal 10aa is a transmission source terminal and the terminals 10ab and 10db are transmission destination terminals. Further, it is assumed that the language (speaker language) of the terminal 10aa is Japanese, and the languages (translation languages) of the terminals 10ab and 10db are Japanese and English, respectively.

  In this case, the terminal 10aa transmits audio data to the terminals 10ab and 10db, respectively (step S140). The terminals 10ab and 10db that have received the audio data execute the processes of steps S141 to S149, respectively. As a result, Japanese speech (speech) by the user of the terminal 10aa can be displayed on the display 120ab of the terminal 10ab as Japanese translation text, and can be displayed on the display 120db of the terminal 10db as English translation text. it can.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, the same effects as the main effects of the first embodiment can be obtained.

  In the first embodiment, the transmission source terminal has performed all the communication for generating the translated text data of each transmission destination terminal. In the present embodiment, each transmission destination terminal has its own device. Communication for generating translated text data is performed. That is, in the present embodiment, communication for generating translated text data is shared by each destination terminal. For this reason, according to this embodiment, the load of each terminal 10 which comprises the transmission system 1 can be disperse | distributed, and the operation | movement of the transmission system 1 can be stabilized.

DESCRIPTION OF SYMBOLS 1 Transmission system 10 Transmission terminal 11 Transmission / reception part 16 Final narrowing down part 16a Measurement part 16b Calculation part 16c Final selection part 17 Display control part 18 Delay detection part 20 Service request part 30 Relay apparatus 31 Transmission / reception part 32 Status detection part 33 Data quality confirmation Unit 34 Change quality management unit 35 Data quality change unit 50 Transmission management system 51 Transmission / reception unit (an example of transmission means)
52 Terminal Authentication Unit 53 Status Management Unit 54 Terminal Extraction Unit 55 Terminal Status Acquisition Unit 56 Primary Refinement Unit 56a Selection Session ID Generation Unit 56b Terminal IP Address Extraction Unit 56c Primary Selection Unit 56c
56d Priority determining unit 57 Session managing unit 58 Quality determining unit 60 Delay time managing unit 61 Language determining unit (an example of language determining means)
70 router 90 program providing system 100 maintenance system 130 service providing system 131 transmission / reception unit 132 text conversion unit 133 translation unit 134 authentication unit 1000 storage unit (an example of storage unit)
3000 storage unit 3001 change quality management DB
5000 storage unit 5001 relay device management DB
5002 Terminal authentication management DB (an example of terminal management means)
5003 Terminal management DB
5004 Destination list management DB
5005 Session management DB
5006 Priority management DB
5007 Quality Management DB
5008 Language management DB (an example of language management means)

JP 2005-295015 A

Claims (8)

A transmission management system for determining a translation language for translating audio data communicated between a plurality of transmission terminals via a communication network,
For each terminal identification information for identifying the transmission terminal, terminal management means for managing an area where the transmission terminal is installed;
Language management means for managing a language set for each region;
Language determination means for determining the language set in the region where the transmission terminal as the transmission destination of the audio data is installed as the translation language of the audio data;
A transmission management system comprising: The transmission management system according to claim 1, wherein the language set for each region is a language most used for each region. 3. The transmission management system according to claim 1, further comprising a transmission unit configured to transmit translation language information indicating the translation language determined by the language determination unit to the transmission terminal serving as a transmission source of the audio data. 3. The transmission management system according to claim 1, further comprising a transmission unit configured to transmit translation language information indicating the translation language determined by the language determination unit to the transmission terminal serving as a transmission destination of the audio data. 5. The transmission management system according to claim 3, wherein the transmission unit transmits the translated language information to the transmission terminal after establishing a session with the transmission terminal and before starting communication of the audio data. The language determination unit determines the language set in the area where the transmission terminal as a transmission source of the voice data is installed as a speaker language that is a language of the voice data. 6. The transmission management system according to any one of 5 above. A transmission management method for determining a translation language for translating audio data communicated between a plurality of transmission terminals via a communication network,
For each terminal identification information for identifying the transmission terminal, managing the area where the transmission terminal is installed;
Managing a language set for each region;
Determining the language set in the region where the transmission terminal as a transmission destination of the audio data is installed as the translation language of the audio data;
A transmission management method comprising: A program for determining a translation language for translating audio data communicated between a plurality of transmission terminals via a communication network,
For each terminal identification information for identifying the transmission terminal, managing the area where the transmission terminal is installed;
Managing a language set for each region;
Determining the language set in the region where the transmission terminal as a transmission destination of the audio data is installed as the translation language of the audio data;
A program that causes a computer to execute.

Images