JP2018046410A - Communication system, information processing apparatus, communication method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication system which distributes a video corresponding to a moving state of a device having a moving function, to a second information processing apparatus which controls moving of the device, from a first information processing apparatus mounted on the device.SOLUTION: A communication system includes: a first information processing apparatus mounted on a device having a moving function, to control the moving function in accordance with predetermined control information; and a second information processing apparatus which receives a moving operation for the device and transmits the control information corresponding to the moving operation to the first information processing apparatus. The communication system includes an imaging unit for imaging video data of the surroundings of the device, a determination unit which determines a moving state of the device, and a determination unit which determines a processing method of the video data to be transmitted from the first information processing apparatus to the second information processing apparatus, so as to be different in accordance with the moving state of the device.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a communication system, an information processing apparatus, a communication method, and a program.

  For example, a video conference system that performs a video conference using a communication terminal between a plurality of bases via a communication network such as the Internet or a LAN (Local Area Network) has become widespread.

  As a technology related to this, in a video conference system, a technology is known in which video is transmitted at an optimal resolution, frame rate, and resolution depending on the type of video data (see, for example, Patent Document 1).

  In a telepresence robot system having a plurality of remote stations and a plurality of robots that can be moved by remote operation, a technique for controlling the robots from the remote stations is known (see, for example, Patent Document 2).

  For example, in a communication system such as a video conference system, a robot as disclosed in Patent Document 2 can be used as a conference information terminal. In such a case, an operator who operates an information terminal such as a robot that can be moved by remote operation (hereinafter referred to as a mobile terminal) performs remote operation of the mobile terminal while watching an image sent from the mobile terminal, for example. It is possible.

  In such a communication system, when the mobile terminal is moving, the video data transmitted from the mobile terminal is preferably distributed so that the operator can easily confirm the situation around the mobile terminal in real time. . On the other hand, when a video conference is being performed, it is desirable that the video data transmitted from the mobile terminal is distributed so as to easily confirm the participant's face, characters, and the like.

  However, in the conventional communication system, an image corresponding to the moving state of the device is displayed from the first information processing device mounted on the device having the moving function to the second information processing device that performs the moving operation of the device. It is difficult to deliver.

  The embodiment of the present invention has been made in view of the above problems, and is a second information processing apparatus that performs a moving operation of a device from a first information processing device mounted on a device having a moving function. On the other hand, it aims at providing the communication system which distributes the image | video according to the movement state of an apparatus.

  In order to solve the above problems, a communication system according to an embodiment of the present invention is mounted on a device having a moving function, and controls the moving function according to predetermined control information. And a second information processing apparatus that transmits the control information corresponding to the movement operation to the first information processing apparatus, wherein video data around the device is acquired. An imaging unit that captures an image, a determination unit that determines a movement state of the device, and a processing method of the video data that is transmitted from the first information processing device to the second information processing device are set to the movement state of the device. And a determining unit that determines a different processing method according to the method.

  According to the embodiment of the present invention, a first information processing device mounted on a device having a movement function is adapted to a second information processing device that performs a device movement operation according to the movement state of the device. A communication system for distributing video can be provided.

It is a figure which shows the structural example of the communication system which concerns on one Embodiment. It is a figure which shows the hardware structural example of the terminal which concerns on one Embodiment. It is a figure which shows the hardware structural example of the management apparatus which concerns on one Embodiment, and a relay apparatus. It is a figure which shows the hardware structural example of the mobile device which concerns on one Embodiment. It is a figure which shows the function structural example of the communication system which concerns on one Embodiment. It is a figure (1) which shows the example of the information managed with the communications system concerning one embodiment. It is a figure (2) which shows the example of the information managed with the communications system concerning one embodiment. It is a sequence diagram which shows the example of the process of the preparation stage of the communication system which concerns on one Embodiment. It is a sequence diagram which shows the example of the communication processing of the communication system which concerns on one Embodiment. It is a figure which shows the example of the display screen of the terminal which concerns on one Embodiment. It is a sequence diagram which shows the example of the process during the movement of the communication system which concerns on 1st Embodiment. It is a sequence diagram which shows the example of the process during the stop of the communication system which concerns on 1st Embodiment. It is a sequence diagram which shows the example of the process during the movement of the communication system which concerns on 2nd Embodiment. It is a sequence diagram which shows the example of the process during the stop of the communication system which concerns on 2nd Embodiment. It is a flowchart which shows the example of a process of the communication management system which concerns on 3rd Embodiment. It is a figure which shows the image of an example of the setting screen which concerns on 3rd Embodiment.

EMBODIMENT OF THE INVENTION The form for implementing this invention is demonstrated, referring drawings based on the following Examples. Examples described below are merely examples, and embodiments to which the present invention is applied are not limited to the following examples.
In all the drawings for explaining the embodiments, the same reference numerals are used for those having the same function, and repeated explanation is omitted.

<System configuration>
FIG. 1 is a diagram illustrating a configuration example of a communication system according to an embodiment. The communication system 1 includes a plurality of terminals (10A, 10B, 10C, 10D, 10E), a mobile device 20, a relay device 30, a management device 50, and the like. In the following description, “terminal 10” is used to indicate an arbitrary terminal among a plurality of terminals (10A, 10B, 10C, 10D, 10E). In addition, “request source terminal” is used to indicate the terminal 10 as a request source requesting the start of communication, and “destination terminal” is used to indicate the terminal 10 as the request destination.

  The terminal 10, the relay device 30, and the management device 50 are respectively connected to other terminals, devices, and systems via the communication network 2 so as to communicate with each other. The communication network 2 may include, for example, a LAN (Local Area Network), the Internet, a mobile phone network, or a dedicated line. In the following description, a system including the management device 50 and the relay device 30 is referred to as a communication management system 100.

  The terminal (information processing apparatus) 10 is, for example, a general-purpose information terminal such as a tablet terminal, a smartphone, or a PC (Personal Computer), or an information processing apparatus such as a dedicated video conference apparatus. The terminal 10 can perform a video conference, for example, by transmitting and receiving, for example, image data, audio data, and the like with one or more other terminals.

  The terminal 10B (first information processing apparatus) is mounted on the mobile device 20, and can communicate with the mobile device 20 wirelessly or by wire. In addition, the terminal 10B executes an application program (hereinafter referred to as an application) corresponding to the communication system 1, so that the mobile device 20 has a predetermined control information received from another terminal 10 (for example, the terminal 10A). The moving function can be controlled. Note that the terminal 10 </ b> B may be a movable terminal 10 integrated with the mobile device 20.

  The terminal 10A (second information processing apparatus), which is an example of the other terminal 10, executes a video conference with the terminal 10B by executing an application corresponding to the communication system 1, and moves the mobile device via the terminal 10B. 20 can be remotely operated.

  For example, the terminal 10A displays an operation button on the display screen of the video conference, receives a movement operation of the mobile device 20 equipped with the terminal 10B, and transmits control information corresponding to the received movement operation to the terminal 10B. Accordingly, the terminal 10A can move the moving device 20 on which the terminal 10B is mounted, for example, forward and backward, left and right.

  The mobile device 20 is driven by, for example, driving a plurality of wheels in accordance with control from the terminal 10B mounted on the mobile device 20, thereby “forward”, “reverse”, “right turn”, “left turn”. It is a device having a movement function for performing movement such as. The appearance of the moving device 20 shown in FIG. 1 is merely an example. The mobile device 20 may have another appearance as long as the mobile device 20 is a device that can move with the terminal 10B according to control from the terminal 10B mounted on the mobile device 20.

  The relay device 30 is an information processing device connected to the communication network 2 or a system including one or more information processing devices, and relays content data (for example, video data) transmitted / received between a plurality of terminals 10. .

  The management device 50 is an information processing device connected to the communication network 2 or a system including one or more information processing devices. The management device 50 performs, for example, login authentication from the terminal 10, management of the communication status of the terminal 10, management of a destination list, etc., and control of a session for performing communication between the plurality of terminals 10 via the relay device 30. Do.

  In addition, the management device 50 manages an operation right that is a right to operate the terminal 10 </ b> B and the mobile device 20. For example, the management device 50 manages the operation right of the terminal 10B so that the number of terminals 10 that can operate the terminal 10B among the plurality of terminals 10 participating in the same session (for example, video conference) as the terminal 10B is one or less. .

  In one embodiment, the session is realized by the relay device 30 relaying content data including video data and the like between the plurality of terminals 10.

  In the communication system 1, for example, a management information session sei for transmitting and receiving various types of management information is established between the terminal 10 </ b> A and the terminal 10 </ b> B via the management device 50. Using the management information session sei, the terminal 10A and the terminal 10B can transmit and receive various types of management data such as call control data for establishing a communication session, for example. In addition, a communication session sed (hereinafter simply referred to as a session) for transmitting content data such as video data and sound data is established between the terminal 10A and the terminal 10B via the relay device 30, for example. Is done.

  The terminal 10A can establish a session with, for example, the terminal 10B using the management information session sei, and can transmit and receive content data including video data and the like using the established session.

  In the above configuration, the user of the terminal 10A communicates with the terminal 10B, and remotely operates and moves the terminal 10B and the moving device 20 while watching, for example, an image around the terminal 10B transmitted from the terminal 10B. be able to. As a result, the user of the terminal 10A can move the terminal 10B and the moving device 20 close to any user, for example, and perform a video conference or the like.

  Further, in the communication system 1 according to the present embodiment, the moving state (for example, moving, stopped) of the terminal 10B or the mobile device 20 is determined, and the video data transmitted by the terminal 10B is changed to distribution data corresponding to the moving state. Process and distribute.

  For example, when the terminal 10B or the mobile device 20 is moving, the communication system 1 processes and distributes video data transmitted by the terminal 10B with priority on the frame rate. In addition, when the terminal 10B or the mobile device 20 is stopped, the communication system 1 processes and distributes video data transmitted by the terminal 10B with priority on image quality and resolution in order to perform a conference or the like with high image quality. To do.

  As described above, the communication system 1 according to the present embodiment switches the video data distribution method depending on whether the terminal 10B or the mobile device 20 is moving the video data transmitted by the terminal 10B.

  Thereby, according to the communication system 1 which concerns on this embodiment, according to the movement state of the moving apparatus 20 with respect to the terminal 10A which performs the movement operation of an apparatus from the terminal 10B mounted in the moving apparatus 20 which has a moving function. The video can be distributed.

  In this embodiment, a communication system 1 and a management device 50 are assumed assuming a video conference system as an example of a communication system, a video conference management system as an example of a communication management system, and a video conference terminal as an example of a communication terminal. The terminal 10 will be described. That is, the terminal 10 and the management apparatus 50 are not only applied to the video conference system but also applied to other communication systems. For example, the communication system 1 is a telephone system, and the terminal 10 may be an IP (Internet Protocol) telephone, an Internet telephone, a PC (Personal Computer), a notebook PC, a mobile terminal such as a smartphone or a tablet terminal, or the like.

  In the embodiment, “video conference” is used interchangeably with “video conference”.

<Hardware configuration>
(Device hardware configuration)
FIG. 2 is a diagram illustrating a hardware configuration example of a terminal according to an embodiment of the present invention. The terminal 10 includes a general computer configuration, for example, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a flash memory 104, and an SSD (Solid State Drive). 105 etc. The terminal 10 includes a media I / F (Interface) 107, an input unit 108, a display unit 109, a network I / F 111, a camera 112, an image sensor I / F 113, a microphone 114, a speaker 115, and a voice input / output I / F 116. Etc. Further, the terminal 10 includes an external device connection I / F 117, a short-range wireless communication unit 118, a bus 119, and the like.

  The CPU 101 is an arithmetic device that implements each function provided in the terminal 10 by reading a program and data from the ROM 102, the flash memory 104, and the like and executing processing. The ROM 102 is a nonvolatile memory that stores in advance a program used to start the CPU 101 such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory used as a work area for the CPU 101.

  The flash memory 104 is a storage device that stores, for example, an OS (Operating System), application programs, and various data. The SSD 105 controls reading and writing of various data with respect to the flash memory 104 according to the control of the CPU 101. The media I / F 107 controls reading or writing (storage) of data with respect to the recording medium 106 which is a storage medium such as a memory card.

  The input unit 108 is an input device for receiving an input operation from a user such as a touch panel, a keyboard, and a pointing device. The display unit 109 is a display device that performs various displays for the user. Note that the input unit 108 and the display unit 109 may be, for example, the display input unit 110 such as a touch panel display in which a touch panel and a display are integrated.

  The network I / F 111 is a communication interface for the terminal 10 to perform data transmission using the communication network 2. The camera 112 includes an image sensor for imaging a subject in accordance with the control of the CPU 101. The imaging element I / F 113 controls imaging by the camera 112 and converts the captured data into predetermined image data. The microphone 114 converts the collected sound into an electrical signal. The speaker 115 converts the sound signal into sound and outputs the sound. The voice input / output I / F 116 controls voice input / output through the microphone 114 and the speaker 115.

  The external device connection I / F 117 is an interface for connecting an external device such as a USB (Universal Serial Bus), for example. The external device includes, for example, the moving device 20 in FIG.

  The short-range wireless communication unit 118 is a communication interface for communicating with an external device (for example, the mobile device 20) by short-range wireless communication such as Bluetooth (registered trademark) or Bluetooth Low Energy (hereinafter referred to as BLE). It is. The bus 119 is commonly connected to each of the above components, and transmits an address signal, a data signal, various control signals, and the like.

(Hardware configuration of management device and relay device)
FIG. 3 is a diagram illustrating a hardware configuration example of a management device and a relay device according to an embodiment. The management device 50 has a general computer configuration, and includes, for example, a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) 505, a media drive 507, a display 508, and the like. The management device 50 includes a network I / F 509, a keyboard 511, a mouse 512, a CD-ROM drive 514, a bus 510, and the like.

  The CPU 501 is an arithmetic device that realizes each function provided in the management device 50 by, for example, reading a program or data stored in the ROM 502, the HD 504, or the like and executing processing. The ROM 502 is a non-volatile memory that stores in advance a program used to start the CPU 501 such as an IPL. A RAM 503 is a volatile memory used as a work area for the CPU 501.

  The HD 504 is a storage device that stores, for example, an OS, a program such as an application program, and various types of data. The HDD 505 controls reading or writing of various data with respect to the HD 504 according to the control of the CPU 501. The display 508 is a display device that displays various types of information such as a cursor, menu, window, character, or image.

  A network I / F 509 is a communication interface for performing data communication using the communication network 2. The keyboard 511 is an example of an input device for accepting input operations such as characters, numerical values, and various instructions by the user. A mouse 512 is an example of a pointing device that accepts operations such as selection and execution of various instructions by a user, selection of a processing target, movement of a cursor, and the like.

  The media drive 507 controls reading and writing (storage) of data with respect to a recording medium 506 such as a memory card, for example. The CD-ROM drive 514 controls reading or writing of data with respect to a disk 513 as an example of a removable storage medium. A bus 510 electrically connects the above-described components and transmits an address signal, a data signal, various control signals, and the like.

  Note that the hardware configuration of the computer is just an example. For example, the display 208, the keyboard 211, the mouse 212, and the like may be externally attached to the outside of the management device 50.

  The relay device 30 has the same hardware configuration as that of the management device 50. Note that the programs for the terminal 10, the relay device 30, and the management device 50 may be recorded and distributed on a computer-readable recording medium in a file that can be installed or executed. good. Examples of the recording medium include CD-R (Compact Disc Recordable), DVD (Digital Versatile Disk), Blu-ray Disc, and the like. Further, a recording medium such as a CD-ROM in which each program is stored, and the HD 504 in which these programs are stored can be provided as a program product (Japan or overseas).

(Hardware configuration of mobile device)
FIG. 4 is a diagram illustrating a hardware configuration example of the mobile device according to the embodiment. The mobile device 20 includes, for example, a CPU 401, a RAM 402, a ROM 403, an external device I / F 404, a short-range wireless communication unit 405, a wheel drive unit 406, a steering unit 407, and the like.

  The CPU 401 is an arithmetic device that implements each function of the mobile device 20 by executing a program stored in the ROM 403 or the like. A RAM 402 is a volatile memory used as a work area for the CPU 401. The ROM 403 is a non-volatile memory that stores a program of the mobile device 20 and the like. The ROM 403 may be a rewritable nonvolatile memory such as a flash ROM, for example.

The external device I / F 404 is a wired communication interface for performing a wired connection with the external device connection I / F 117 of the terminal 10 illustrated in FIG.

  The short-range wireless communication unit 405 is a wireless communication interface for performing wireless communication by the same wireless communication method (for example, BLE communication) as the short-range wireless communication unit 118 of the terminal 10 illustrated in FIG. The mobile device 20 only needs to be able to communicate with the terminal 10 via, for example, the external device I / F 404 or the short-range wireless communication unit 405.

  The wheel driving unit 406 is an example of a driving device that drives a wheel for moving the moving device 20. The wheel drive unit 406 includes, for example, a motor.

  The steering unit 407 is an example of a steering device that steers the moving device 20 that is moved by the wheel driving unit 406. For example, the steering unit 407 may change the direction of the wheels, or may change the direction of the moving device 20 by controlling the number of rotations, the speed, or the like of the left and right wheels. .

<Functional configuration>
FIG. 5 is a diagram illustrating a functional configuration example of a communication system according to an embodiment. In the example of FIG. 5, the communication system 1 includes a communication management system 100, a terminal 10A, and a terminal 10B that are communicably connected to each other via the communication network 2. The communication management system 100 corresponds to the management device 50 and the relay device 30 in FIG.

(Functional configuration of terminal 10A)
The terminal 10A is an example of the terminal 10 that is not connected to a device having a moving function such as the mobile device 20.

  The terminal 10A includes, for example, a transmission / reception unit 11, an operation input reception unit 12, a communication control unit 13, an imaging unit 14, an audio input unit 15a, an audio output unit 15b, a display control unit 16, a storage / read unit 17, and control information transmission. Part 41 and the like. The terminal 10A may further include a movement state determination unit B42, a processing method determination unit C43, and the like. Each of these units is realized by any one of the constituent elements shown in FIG. 2 operating according to a command from the CPU 101 according to the program for the terminal 10 expanded from the flash memory 104 onto the RAM 103. Function or means to function. Further, the terminal 10 includes a storage unit 18 realized by the RAM 103 illustrated in FIG. 2 and the flash memory 104 illustrated in FIG.

(Functional configuration of terminal 10B)
The terminal 10B is an example of the terminal 10 having a control function for controlling a device having a moving function such as the mobile device 20, for example. The terminal 10B is, for example, the transmission / reception unit 11, the operation input reception unit 12, the communication control unit 13, the imaging unit 14, the audio input unit 15a, the audio output unit 15b, the display control unit 16, and the storage / reading unit, similarly to the terminal 10A. 17 and a storage unit 18 and the like.

  Further, the terminal 10B includes, for example, an operation instruction receiving unit 31, an operation instruction authenticating unit 32, a device control unit 33, an inter-device communication unit 34, a movement state determining unit A35, a processing method determining unit A36, and the like. The function of the processing method determination unit A36 may be realized by the processing method determination unit B59 of the communication management system 100. In this case, the terminal 10B may not have the processing method determination unit A36.

(Each functional configuration of terminal 10)
Next, each functional configuration common to the terminal 10A and the terminal 10B will be described.

  The transmission / reception unit 11 is realized by a command from the CPU 101 illustrated in FIG. 2 and the network I / F 111 illustrated in FIG. 2, and other terminals, devices, or systems and various data via the communication network 2. (Or information) is sent and received. The transmission / reception unit 11 starts receiving state information indicating the state of each terminal 10 as a destination candidate from the communication management system 100 before starting communication with the destination terminal. This status information indicates not only the operating status of each terminal 10 (online or offline status) but also a detailed status such as whether communication is possible or communication is in progress when online.

  The operation input receiving unit 12 is realized by a command from the CPU 101 shown in FIG. 2, the input unit 108 shown in FIG. 2, and the like, and receives various inputs by the user. For example, when the user performs an operation of turning on the power of the terminal 10, the operation input receiving unit 12 illustrated in FIG. 5 receives the operation and controls the power on.

  The communication control unit 13 is realized by a command from the CPU 101 shown in FIG. For example, the communication control unit 13 receives login request information indicating that the login is requested from the transmission / reception unit 11 to the communication management system 100 via the communication network 2 and the current time of the request source terminal. Automatically sends the IP address. When the user performs an operation to turn off the terminal 10, the operation input accepting unit 12 turns off the power after the transmission / reception unit 11 transmits state information indicating that the power is turned off to the communication management system 100. To do. As a result, the communication management system 100 can recognize that the terminal 10 has been turned off from being turned on.

  In addition, the communication control unit 13 performs various communication controls such as establishment and disconnection of a communication session for transmitting / receiving content data to / from another terminal 10 via the relay device 30. Note that the communication control unit 13 according to the present embodiment includes the terminal ID (Identification) of the terminal 10 in the control information (for example, start request information, start response information, and the like described later) transmitted to the management apparatus 50. Send.

  The terminal ID is an example of identification information of an account that can participate in a session in which content data is transmitted using the terminal 10. The terminal ID may be, for example, a user ID that is user identification information, an application ID that is application identification information, or a contract ID that is identification information of a contractor of the terminal 10.

  The imaging unit 14 is configured to execute predetermined commands from the CPU 101 illustrated in FIG. 2 and imaging data obtained by imaging the subject, which is realized by the camera 112 and the imaging element I / F 113 illustrated in FIG. 2. Converted to image (video) data and output. For example, the imaging unit 14 captures video data around the terminal 10.

  The voice input unit 15a is realized by the command from the CPU 101 shown in FIG. 2 and the voice input / output I / F 116 shown in FIG. 2, and the user's voice is converted into a voice signal by the microphone 114. Thereafter, the audio signal is converted into predetermined audio data and output. The audio output unit 15b is realized by a command from the CPU 101 shown in FIG. 2 and the audio input / output I / F 116 shown in FIG. 2, and converts audio data into an audio signal and outputs it to a speaker. Audio is output from the speaker 115.

  The display control unit 16 is realized by a command from the CPU 101 shown in FIG. For example, the display control unit 16 causes the display unit 109, the display input unit 110, and the like to display a video included in the content data received by the terminal 10, a device operation screen for accepting an operation on the mobile device 20, and the like. Further, the display control unit 16 can transmit information on the destination list received from the communication management system 100 to the display unit 109 and cause the display unit 109 to display the destination list.

  The storage / reading unit 17 is realized by an instruction from the CPU 101 shown in FIG. 2 and the SSD 105 shown in FIG. 2 or the like, or realized by an instruction from the CPU 101. The storage / reading unit 17 performs a process of storing various data in the storage unit 18 and reading out various data stored in the storage unit 18.

  The storage unit 18 stores, for example, the above-described terminal ID and authentication information such as a password corresponding to the terminal ID. Also, the storage unit 18 stores various information (for example, operation right, transfer right, etc.) notified from the management device 50 regarding the operation right of the terminal 10B and the mobile device 20. Furthermore, the storage unit 18 overwrites and stores image data and audio data received when performing communication with the destination terminal each time it is received. Among these, the image is displayed on the display unit 109 or the like by the image data before being overwritten, and the sound is output from the speaker 115 by the sound data before being overwritten.

  Note that the terminal ID and the relay device ID described later are examples of identification information such as language, characters, symbols, and various signs used to uniquely identify the terminal 10, the relay device 30, and the like. Indicates. Further, the terminal ID and the relay device ID may be identification information in which at least two of the language, characters, symbols, and various indicia are combined.

(Each functional configuration of terminal 10B)
Subsequently, each functional configuration included in the terminal 10B will be described.

  The operation instruction receiving unit 31 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, and another device 10 (for example, the terminal 10 </ b> A) transmits a device (mobile device) to the terminal 10 </ b> B via the transmission / reception unit 11. 20) The operation instruction information requesting the control is received. The operation instruction information includes, for example, the terminal ID of the terminal 10 that has transmitted the operation instruction information, control information indicating the requested control content, and the like.

  The operation instruction authenticating unit 32 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, and the operation instruction information received by the operation instruction receiving unit 31 is information from the terminal 10 having the operation right of the terminal 10B. Authenticate if there is. For example, the operation instruction authentication unit 32 stores in the storage unit 18 the terminal ID of the terminal 10 having the operation right of the terminal 10B notified from the communication management system 100. Further, when the operation instruction receiving unit 31 receives the operation instruction information, the operation instruction authenticating unit 32 has the terminal ID included in the received operation instruction information and the terminal 10 having the operation right of the terminal 10 stored in the storage unit 18. Is compared with the terminal ID of, and authentication is permitted if they match.

  The device control unit 33 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, received by the operation instruction receiving unit 31, and included in the operation instruction information permitted to be authenticated by the operation instruction authentication unit 32. The mobile device 20 is controlled based on the control information.

  The inter-device communication unit 34 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, the short-range wireless communication unit 118 illustrated in FIG. 2, the external device connection I / F 117, or the like.

  The movement state determination unit A (an example of a determination unit) 35 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, and determines the movement state of the movement device 20 (and the terminal 10B).

  The movement state determination unit A35 is, for example, a “moving” state (first state) in which the moving device 20 is moving, or a “stopped” state (second state) in which the moving device 20 is not moving. ).

  Alternatively, for example, the movement state determination unit A35 determines whether the movement speed of the movement device 20 is equal to or higher than a predetermined speed (another example of the first state), or the movement speed of the movement device 20 is lower than the predetermined speed (second It is possible to determine whether the state is another example). Here, the following description will be given on the assumption that the movement state determination unit A35 determines whether the movement device 20 is in the “moving” state or the “stopping” state.

  There are several possible methods for the movement state determination unit A35 to determine the movement state of the moving device 20.

  As an example, the movement state determination unit A <b> 35 can determine the movement state of the mobile device 20 by acquiring information indicating the movement state from the mobile device 20 using the inter-device communication unit 34. At this time, the movement state determination unit A35 may determine that the movement state has changed when the acquired movement state continues for a predetermined time.

  When the terminal 10B and the mobile device 20 are the integrated terminal 10, the movement state determination unit A35 acquires information indicating the movement state of the integrated terminal 10 from the travel control unit 22 described later. The movement state of the integrated terminal 10 may be determined.

  As another example, the movement state determination unit A35 may determine the movement state of the mobile device 20 based on control information received from the terminal 10A for operating the mobile device 20 or the like. For example, when the control information received from the terminal 10A is information instructing the movement of the moving device 20, the moving state determination unit A35 may determine that the moving device 20 has shifted to the “moving” state. good. Further, the movement state determination unit A35 may determine that the mobile device 20 has shifted to the “stopped” state when the control information received from the terminal 10A is information that instructs the mobile device 20 to stop. good. At this time, the movement state determination unit A35 may determine that the movement state has changed when the state of the received control information continues for a predetermined time.

  The processing method determination unit A (an example of a determination unit) 36 is realized by, for example, an instruction from the CPU 101 illustrated in FIG. The processing method determination unit A36 determines the processing method of the video data transmitted from the terminal 10B to the terminal 10A and captured by the imaging unit 14 around the moving device 20 as a different processing method according to the moving state of the moving device 20. .

  For example, when the movement state of the moving device 20 determined by the movement state determination unit A35 is the “moving” state (first state), the processing method determination unit A36 prioritizes the frame rate as the video data processing method. The determined processing method (first processing method) is determined.

  On the other hand, when the movement state of the moving device 20 determined by the movement state determination unit A35 is the “stopped” state (second state), the processing method determination unit A36 determines the image data processing method, such as image quality, resolution, and the like. Is determined as a processing method (second processing method) giving priority.

  For example, when the communication speed of the distribution data distributed to the terminal 10A is constant, a higher frame can be obtained by reducing the resolution of the video data (for example, changing the resolution from 540 × 720 dots to 270 × 360 dots). Video data can be transmitted at a rate.

  Similarly, when the communication speed of the distribution data distributed to the terminal 10A is constant, the color depth (an example of image quality) of the video data is lowered (for example, the color depth is changed from 24 bits to 8 bits). The video data can be transmitted at a higher frame rate.

  On the other hand, when the communication speed of the distribution data distributed to the terminal 10A is constant, the resolution (or image quality) is increased by lowering the frame rate of the video data (for example, changing from 30 fps to 15 fps). Video data can be transmitted with.

  The processing method determination unit A36 determines whether to process the video data with priority on the frame rate or to process the video data with priority on the image quality / resolution according to the moving state of the mobile device 20 (and the terminal 10B). (change.

(Functional configuration of mobile device)
The mobile device 20 includes, for example, an inter-device communication unit 21 and a travel control unit 22.

  The inter-device communication unit 21 is realized by, for example, a command from the CPU 401 illustrated in FIG. 4, the external device I / F 404, the short-range wireless communication unit 405, or the like. Here, it is assumed that the inter-device communication unit 21 communicates with the terminal 10B using the short-range wireless communication unit 405.

  The traveling control unit 22 is realized by, for example, a command from the CPU 401 illustrated in FIG. The traveling control unit 22 moves the moving device 20 (for example, forward, backward, left-handed) by controlling the wheel driving unit 406 and the steering unit 407 of FIG. 4 according to the control of the device control unit 33 of the terminal 10B, for example. Turn, turn right, etc.).

(Each functional configuration of terminal 10A)
Subsequently, each functional configuration included in the terminal 10A will be described.

  The control information transmission unit 41 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, and includes control information corresponding to a moving operation performed on the device operation screen displayed by the display control unit 16. The operation instruction information is transmitted to the terminal 10B.

  The movement state determination unit B (another example of the determination unit) 42 is realized by, for example, a command from the CPU 101 illustrated in FIG. 2, and determines the movement state of the mobile device 20 (and the terminal 10B). For example, the movement state determination unit B42 determines the movement state of the movement device 20 based on the control information transmitted from the control information transmission unit 41 to the terminal 10B in order to operate the movement device 20.

  For example, when the control information transmission unit 41 transmits control information instructing movement to the terminal 10B, the movement state determination unit B42 determines that the terminal 10B has shifted to the “moving” state, and stops the terminal 10B. When the control information to be instructed is transmitted, it is determined that the state has shifted to the “stopped” state.

  The processing method determination unit C (another example of the determination unit) 43 is realized by, for example, an instruction from the CPU 101 illustrated in FIG. The processing method determination unit C43 changes the processing method of the video data captured by the imaging unit 14 to be transmitted from the terminal 10A to the terminal 10B according to the movement state of the mobile device 20 determined by the movement state determination unit B42. decide. Note that the method of determining the processing method of the video data by the processing method determination unit C43 may be the same as the processing method determination unit A36 of the terminal 10B, for example.

(Functional configuration of communication management system)
The communication management system 100 includes, for example, one or more information processing devices such as the management device 50 and the relay device 30 illustrated in FIG. Here, for ease of explanation, the following description will be given on the assumption that the communication management system 100 is configured by one information processing apparatus. However, this is merely an example, and each functional configuration described below may be provided in a distributed manner in a plurality of information processing devices (for example, the management device 50, the relay device 30, and the like).

  The communication management system 100 includes a transmission / reception unit 51, a terminal authentication unit 52, a terminal management unit 53, a destination list management unit 54, a session management unit 55, a device operation right management unit 56, a data relay unit 57, a video data processing unit 58, a processing A method determining unit B59, a storage / reading unit 60, and the like are included. Each of these units is a function or a means that functions when any one of the components shown in FIG. 3 is operated by an instruction from the CPU 201 according to a program expanded from the HD 204 to the RAM 203. . Further, the communication management system 100 has a storage unit 5000 realized by the HD 204 or the like shown in FIG.

  Note that the function of the processing method determination unit B59 may be realized by the processing method determination unit A36 of the terminal 10B.

(Each functional configuration of the communication management system)
Next, each functional configuration of the communication management system 100 will be described in detail.

  The transmission / reception unit 51 is executed by the command from the CPU 201 shown in FIG. 3 and the network I / F 209 shown in FIG. Or information).

  The terminal authentication unit 52 is realized by a command from the CPU 201 shown in FIG. For example, the terminal authentication unit 52 determines whether the combination of the terminal ID and the password included in the login request information received via the transmission / reception unit 51 is included in the authentication management DB (Database) 5002. Thus, the terminal 10 is authenticated.

  The terminal management unit 53 is realized by a command from the CPU 201 illustrated in FIG. 3, and the terminal management DB 5003 stores the destination name, operation status, reception date and time of request information, etc. for each terminal ID, and the IP of the request source terminal. Store and manage addresses in association with each other. For example, the terminal management unit 53 sets the terminal management DB 5003 online based on the status information sent from the terminal 10 to turn off the power when the user turns off the power of the terminal 10. Change the indicated operating status to offline. The terminal management DB 5003 will be described later.

  Also, the terminal management unit 53 searches the destination list management DB 5004 using the terminal ID of the request source terminal that requested the login as a key, and extracts the terminal ID of the destination terminal that can communicate with the request source terminal. Furthermore, the terminal management unit 53 searches the destination list management DB 5004 using the terminal ID of the request source terminal that has requested login as a key, and registers the terminal ID of the request source terminal as a destination terminal candidate. The terminal ID of the request source terminal is also extracted.

  Further, the terminal management unit 53 searches the terminal management DB 5003 described above using the extracted terminal ID of the destination terminal candidate as a search key, and reads the operating state for each extracted terminal ID. As a result, the terminal management unit 53 can acquire the operating state of the candidate destination terminal that can communicate with the request source terminal that has requested the login. Further, the terminal management unit 53 searches the terminal management DB 5003 described above using the extracted terminal ID as a search key, and also acquires the operating state of the request source terminal that has requested the login.

  The destination list management unit 54 is realized by a command from the CPU 201 illustrated in FIG. 3, and adds or deletes the terminal ID of the destination terminal for each terminal ID of each request source terminal in the destination list management DB 5004. The destination list management DB 5004 will be described later.

  The session management unit 55 is realized by a command from the CPU 201 shown in FIG. 3 and controls a session managed by the communication management system 100. The session control includes, for example, control for establishing a session, control for causing the terminal 10 to participate in the established session, control for disconnecting the session, generation of a session ID, and the like. In addition, the session management unit 55 associates the request source terminal ID of the terminal 10 that requested the start of the session, the destination terminal ID of the destination terminal 10, and the like with the session ID that is the session identification information. Remember and manage. The session management DB 5005 will be described later.

  The device operation right management unit 56 is realized by a command from the CPU 201 illustrated in FIG. 3. For example, the device operation right management unit 56 controls a device 10 such as the mobile device 20 (hereinafter, may be referred to as a device control terminal). Manages operation rights that are the right to perform operations. For example, the device operation right management unit 56 changes the setting that the terminal 10 capable of operating the device such as the mobile device 20 through the communication with the device control terminal from the setting that is the terminal 10A to the setting that is the terminal 10C.

  Preferably, the device operation right management unit 56 also manages the transfer right, which is the right to transfer the operation right of the device control terminal to another terminal 10. In addition, management of the transfer right is not essential.

  Preferably, the device operation right management unit 56 is a terminal 10 of a person in charge of a predetermined device operation such as a user who manages the device control terminal or a user who is used to operating the device control terminal. It also manages information (for example, terminal ID) of “device operation charge terminal”. It is not essential to manage the device operation terminal.

  The data relay unit (distribution unit) 57 is realized by a command from the CPU 201 illustrated in FIG. 3, and a plurality of terminals 10 that participate in the same session based on a session ID that is identification information for identifying a session. Relay content data (video data, etc.) between them.

  For example, the data relay unit 57 receives the video data transmitted from the terminal 10B, and distributes the distribution data generated by the video data processing unit 58 using the received video data to the terminal 10A. Further, the data relay unit 57 may receive video data transmitted from the terminal 10A, and distribute the distribution data generated by the video data processing unit 58 using the received video data to the terminal 10B. .

  The video data processing unit (generation unit) 58 is realized by a command from the CPU 201 shown in FIG. The video data processing unit 58 processes the video data according to the video data processing method determined by the processing method determination unit A36 of the terminal 10B or the processing method determination unit B59 of the communication management system 100, and the terminal 10A (or terminal 10B). Generate distribution data to be distributed to.

  For example, when the video data processing method is determined to be the frame rate priority processing method, the video data processing unit 58 has a higher frame rate than when the video data processing method is determined to be the image quality / resolution priority processing method. Then, the video data is processed (for example, encoded). Also, the video data processing unit 58 has a better image quality when the video data processing method is determined to be the image quality / resolution priority processing method than when the video data processing method is determined to be the frame rate priority processing method. Alternatively, the video data is processed (for example, encoded) at a high resolution.

  In the present embodiment, the video data processing method is not particularly limited, but as an example, the video data processing unit 58 is an H.264 standard. Video data can be processed using video data that is scalable video coded by H.264 SVC (Scalable Video Cording).

  H. H.264 SVC is a video coding standard. The video data encoded by H.264 SVC includes a base layer and an enhancement layer, which are the minimum necessary video elements. This extension layer includes, for example, a spatial layer that contributes to higher definition of the image, a time layer that contributes to the frame rate, an S / N ratio layer that contributes to an improvement in the S / N (Signal / Noise) ratio, and the like. .

  For example, the video data processing unit 58 receives the H.264 from the terminal 10B. By receiving video data encoded by H.264 SVC and generating distribution data using a base layer and a time layer included in the video data, distribution data giving priority to the frame rate can be generated.

  Similarly, the video data processing unit 58 can generate distribution data giving priority to resolution by generating distribution data using a base layer and a spatial layer included in the video data. Further, the video data processing unit 58 generates distribution data with priority on S / N (an example of image quality) by generating distribution data using a base layer and an S / N ratio layer included in the video data. be able to.

  H. The generation of distribution data using video encoded by H.264 SVC is an example of a method of generating distribution data by the video data processing unit 58.

  As another example, the video data processing unit 58 includes processing method management information including information on the frame rate and resolution when priority is given to the frame rate, and information on the frame rate and resolution when priority is given to image quality and resolution. It may be stored in the storage unit 5000 in advance. In this case, the video data processing unit 58 generates distribution data giving priority to the frame rate or delivery data giving priority to the image quality / resolution in accordance with the processing method management information.

  The processing method determination unit B (another example of the determination unit) 59 is realized by, for example, an instruction from the CPU 201 illustrated in FIG. The processing method determination unit B59 determines the processing method of the video data by the video data processing unit 58 as a different processing method according to the information indicating the movement state of the mobile device 20 received from the terminal 10B (or the terminal 10A). Note that the method for determining the processing method of the video data may be the same as the determination method by the processing method determination unit A36 of the terminal 10B, for example.

  The storage / reading unit 60 is realized by the command from the CPU 201 and the HDD 205 shown in FIG. 3 or by the command from the CPU 201. The storage / reading unit 60 stores various data to the storage unit 5000 and reads various data from the storage unit 5000.

<Example of information managed by communication system>
Here, an example of each piece of information managed by the communication system 1 will be described.

  6 and 7 are diagrams illustrating examples of information managed by the communication system according to the embodiment.

(Device operation right management table)
The device operation right management DB 5001 stored in the storage unit 5000 of the communication management system 100 includes, for example, a device operation right management table 601 as shown in FIG. In the device operation right management table 601, for example, “transfer right possessing terminal”, “operation right possessing terminal”, and “apparatus operation terminal” are associated with the terminal ID of the device control terminal that controls the device such as the mobile device 20. Information such as “responsible terminal” is managed.

  The “transfer right possessing terminal” information is information (for example, terminal ID) for identifying the terminal 10 having the right to transfer the device operation right (hereinafter referred to as transfer right) that is the right to operate the device control terminal. is there. For example, the device operation right management unit 56 stores and manages the terminal ID of the terminal 10 that first established a session with the device control terminal as a transfer right possessing terminal. In addition, when the device operation right is returned from the terminal 10 to which the device operation right is transferred, the device operation right management unit 56 stores and manages the terminal 10 having the transfer right as an operation right holding terminal. Note that the “transfer right possessing terminal” information is optional and not required.

  The “operation right possessing terminal” information is information (for example, a terminal ID) for identifying the terminal 10 having the device operation right that is the right to operate the device control terminal. For example, the device operation right management unit 56 stores and manages the terminal ID of the terminal 10 that currently has the device operation right of the device control terminal as an “operation right holding terminal”.

  The information on the “device operation charge terminal” includes, for example, the terminal 10 used by a predetermined person in charge of device operation such as a user who manages the device control terminal or a user who is used to operating the device control terminal. Information for identification (for example, a terminal ID). For example, a terminal ID of a person in charge determined in advance for each device control terminal is stored in the “device operation charge terminal” information. This information is used, for example, to facilitate selection of the terminal 10 used by the person in charge of device operation on the destination selection screen described later with reference to FIG. Note that the information on “device operation charge terminal” is optional and not essential.

  Note that “unused” in the information of the device operation right management table 601 indicates that the device control terminal is not currently used (not participating in a session). Further, “unsupported terminal” in the information of the device operation right management table 601 indicates that it is not a device control terminal. The device operation right management DB 5001 and the device operation right management table 601 are an example of a storage unit that stores information of a terminal having an operation right that is a right to operate a device control terminal.

(Authentication management table)
The authentication management DB 5002 stored in the storage unit 5000 of the communication management system 100 includes an authentication management table 602 as shown in FIG. 6B, for example. In the authentication management table 602, the terminal ID of the terminal 10 managed by the communication management system 100 and the password corresponding to the terminal ID are managed in association with each other. For example, the authentication management table 602 shown in FIG. 6B indicates that the password of the terminal 10 whose terminal ID is “01aa” is “aaa”.

(Terminal management table)
The terminal management DB 5003 stored in the storage unit 5000 of the communication management system 100 includes a terminal management table 603 as shown in FIG. 6C, for example. In this terminal management table 603, for each terminal ID of the terminal 10, a destination name when each terminal 10 is a destination, an operating state of each terminal 10, and a reception date and time when the communication management system 100 receives login request information described later , And the IP address of the terminal 10 are managed in association with each other. For example, in the terminal management table 603 shown in FIG. 6C, the terminal 10 with the terminal ID “01aa” has the terminal name “Japan Tokyo Office AA terminal” and the operating state is “online (communicable)”. ) ". The terminal 10 with the terminal ID “01aa” has the date and time when the login request information is received by the communication management system 100 as “13:40 on April 10, 20xx” and the IP address is “1.2. 1.3 ".

(Destination list management table)
The destination list management DB 5004 stored in the storage unit 5000 of the communication management system 100 includes, for example, a destination list management table 701 as shown in FIG. In the destination list management table 701, all terminal IDs of destination terminals registered as destination terminal candidates are managed in association with terminal IDs of request source terminals that request the start of communication in a video conference. For example, in the destination list management table 701 shown in FIG. 7A, a destination terminal candidate that can request the start of communication from a request source terminal whose terminal ID is “01ab” is a terminal ID “ These terminals are “01aa”, “01ca”, and “01cb”. This destination terminal candidate is updated by being added or deleted by the destination list management unit 54 in response to a request for addition or deletion from the arbitrary request source terminal to the communication management system 100.

(Session management table)
The session management DB 5005 stored in the storage unit 5000 of the communication management system 100 includes a session management table 702 as shown in FIG. 7B, for example. In this session management table 702, for each session ID that is identification information of a session, the relay device ID of the relay device 30 used for relay, the terminal ID of the request source terminal, the terminal ID of the destination terminal, and the start date and time of the session Etc. are managed. For example, in the session management table 702 shown in FIG. 7B, the session with the session ID “se2” is performed between the request source terminal ID “01ad” and the destination communication “01ca”. It is shown. Also, it is indicated that the session with the session ID “se2” is started at “20xx / 4/10 13:11:11” via the relay device with the relay device ID “111b”.

(Processing method management information)
As an example, the video data processing unit 58 included in the communication management system 100 processes video data based on processing method management information 703 as shown in FIG.

  In the example of FIG. 7C, the video data processing unit 58 generates distribution data with a resolution of “270 × 360” dots and a frame rate of “30 fps” when processing the video data with priority on the frame rate. ing. In this case, for example, the video data processing unit 58 generates distribution data by reducing the resolution of the distribution data so that the frame rate of the distribution data approaches “30 fps”. Note that the video data processing unit 58 may set the resolution higher than “270 × 360” dots when the frame rate reaches a value of “30 fps”.

  Further, it is shown that the video data processing unit 58 generates distribution data at a resolution of “540 × 720” dots and a frame rate of “15 fps” when processing video data with priority on image quality and resolution. In this case, the video data processing unit 58 generates the distribution data by reducing the frame rate of the distribution data so that the resolution of the distribution data approaches “540 × 720”, for example. Note that the video data processing unit 58 may set the frame rate to a value higher than “15 fps” when the resolution reaches the value “540 × 720”.

  Note that the processing method management information 703 and the processing method of the video data using the processing method management information 703 shown in FIG. 7C are examples, and the processing method of the video data by the video data processing unit 58 is another method. It may be.

<Process flow>
Next, a processing flow of the communication method by the communication system 1 will be described.

(Preparation process)
FIG. 8 is a sequence diagram illustrating an example of processing in a preparation stage of the communication system according to the embodiment. Here, as an example, processing in the preparation stage before the session is started by the terminal 10A, the terminal 10B, and the terminal 10C will be described. In the following description, it is assumed that the terminal ID of the terminal 10A is “01aa”, the terminal ID of the terminal 10B is “01ab”, and the terminal ID of the terminal 10C is “01ac”. In FIG. 8, it is assumed that various types of management information are transmitted and received by the management information session sei described above.

  First, when the user of the terminal 10A that is the request source terminal performs an operation of turning on the power of the terminal 10A, for example, the operation input receiving unit 12 illustrated in FIG. 5 receives the operation of turning on the power. Then, the terminal 10A is turned on (step S21). And the communication control part 13 transmits the login request information which requests | requires a login from the transmission / reception part 11 to the communication management system 100 via the communication network 2 by said power-on (step S22). Note that the login request information is transmitted by an operation to turn on the terminal 10A, for example, and is transmitted by, for example, an operation to the input unit 108 by a user or activation of an application. There may be.

  Further, the login request information includes a terminal ID (request source terminal ID) and a password for identifying the terminal 10A as the request source. These terminal ID and password are information read from the storage unit 18 via the storage / readout unit 17, for example. Further, when the login request information is transmitted from the terminal 10A to the communication management system 100, the communication management system 100 as the receiving side can grasp the IP address of the terminal 10A as the transmitting side.

  Next, the terminal authentication unit 52 of the communication management system 100 searches the above-described authentication management table 602 using the terminal ID and password included in the login request information received via the transmission / reception unit 51 as search keys. The terminal authentication unit 52 performs authentication based on whether or not the combination of the terminal ID and the password included in the login request information received from the terminal 10A is included in the authentication management table 602 (step S23).

  When the terminal authentication unit 52 determines that the login request is from the terminal 10 </ b> A having a valid usage right, the terminal management unit 53 stores the terminal ID “01aa” of the terminal 10 </ b> A recorded in the terminal management table 603. Change the operating status to “Online (communicable)”. At this time, the terminal management unit 53 updates the reception date and time, and updates the IP address of the terminal 10 as necessary (step S24). Accordingly, the terminal management table 603 includes the terminal ID “01aa” of the terminal 10A, the operation state “online (communication possible)”, the reception date and time “20xx.4.1.13: 40”, and the IP address “10A of the terminal 10A”. "1.2.1.3" is managed in association with each other.

  The transmission / reception unit 51 of the communication management system 100 is a request source terminal that has requested the login of the authentication result information indicating the authentication result obtained by the terminal authentication unit 52 via the communication network 2. It transmits to the terminal 10A (step S25). Here, a case where the terminal authentication unit 52 determines that the terminal has a valid use authority will be described below.

  The terminal management unit 53 of the communication management system 100 searches the destination list management table 701 using the terminal ID “01aa” of the request source terminal (terminal 10A) that requested the login as a search key. As a result, the terminal management unit 53 extracts candidate terminal IDs of destination terminals that can communicate with the request source terminal (terminal 10A) (step S26). Here, as an example, “01ab”, “01ac”, and “01ad” are extracted as the terminal IDs of the destination terminals corresponding to the terminal ID “01aa” of the request source terminal (terminal 10A).

  Next, the terminal management unit 53 searches the terminal management table 603 using the extracted terminal IDs of candidate destination terminals (“01ab”, “01ac”, “01ad”) as search keys. As a result, by reading out the operating state for each extracted terminal ID, each operating state of the terminal ID (“01ab”, “01ac”, “01ad”) is acquired (step S27).

  Next, the transmission / reception unit 51 sends the destination state information including the respective operation states of the destination terminal candidate terminal IDs (“01ab”, “01ac”, “01ad”) to the request source terminal (terminal 10A). Transmit (step S28). Thereby, the request source terminal (terminal 10A) grasps the current operating state of the terminal IDs (“01ab”, “01ac”, “01ad”) that are candidates for the destination terminal of the request source terminal (terminal 10A). be able to.

  Further, the terminal management unit 53 of the communication management system 100 searches the destination list management table 701 using the terminal ID “01aa” of the request source terminal (terminal 10A) that requested the login as a search key. Thereby, the terminal management unit 53 extracts another request source terminal ID in which the terminal ID “01aa” of the request source terminal (terminal 10A) is registered as a destination terminal candidate (step S29). In the destination list management table 701 shown in FIG. 7A, the terminal IDs of the other request source terminals to be extracted are “01ab”, “01ac”, and “01ad”.

  Next, the terminal management unit 53 of the communication management system 100 searches the terminal management table 603 using the terminal ID “01aa” of the request source terminal (terminal 10A) that requested the login as a search key. Thereby, the terminal management unit 53 acquires the operating state of the request source terminal (terminal 10A) that has requested login (step S30).

  The transmission / reception unit 51 sets the operation status to “online (communicable)” in the terminal management table 603 among the terminal IDs (“01ab”, “01ac”, “01ad”) extracted in step S29. Terminal IDs (“01ab”, “01ac”) are extracted. Further, the transmission / reception unit 51 sends the terminal ID “01aa” of the requesting terminal (terminal 10A) and the operation state “online (terminal) to the terminals 10B and 10C corresponding to the extracted terminal IDs (“ 01ab ”and“ 01ac ”). Destination state information including "communicable" is transmitted (steps S31-1, 2).

  When the transmission / reception unit 51 transmits the destination status information to the terminals 10B and 10C, the IP address of the terminal managed in the terminal management table 603 is set based on each terminal ID (“01ab”, “01ac”). refer. As a result, the terminal ID “01aa” of the request source terminal (terminal 10A) and the operation status are respectively transmitted to the destination terminals (terminals 10B and 10C) that can communicate with the request source terminal (terminal 10A) that requested the login. Can communicate “online”.

  On the other hand, the other terminals 10 also perform the same processing as the processing in steps S22 to S31-1, 2 according to, for example, a power-on operation by the user.

(Communication processing)
FIG. 9 is a sequence diagram illustrating an example of communication processing of the communication system according to the embodiment. Here, an example of communication processing for starting communication between the terminal 10A and the terminal 10B that is a device control terminal capable of controlling the mobile device 20 will be described.

  In step S901, when the terminal 10A receives a selection operation of the destination terminal (terminal 10B) by the user of the terminal 10A, in step S902, the terminal 10A transmits start request information requesting the communication management system 100 to start a session. The start request information includes, for example, a request source terminal ID that is the terminal ID of the terminal 10A that is the request source terminal, a destination terminal ID that is the terminal ID of the terminal 10B that is the destination terminal, and the like. The start request information includes information such as the IP address (request source IP address) of the terminal 10A.

  In step S903, the terminal management unit 53 of the communication management system 100 that has received the start request information from the terminal 10A, based on the terminal ID “01aa” of the request source terminal (terminal 10A) included in the start request information. Update. For example, the terminal management unit 53 changes the operation state information corresponding to the terminal ID “01aa” of the terminal 10A to “online (during communication)” and updates the reception date / time information.

  In step S904, the session management unit 55 of the communication management system 100 transmits start request information for requesting the start of the session to the terminal 10B that is the destination terminal. The start request information includes, for example, the request source terminal ID of the terminal 10A that is the request source terminal.

  In step S <b> 905, the terminal 10 </ b> B that has received the start request information from the communication management system 100 transmits start response information to the communication management system 100. The start response information includes, for example, the destination terminal ID of the terminal 10B.

  In step S906, the terminal management unit 53 of the communication management system 100 that has received the start response information from the terminal 10B updates the terminal management DB 5003 based on the terminal ID “01ab” of the terminal 10B included in the start response information. For example, the terminal management unit 53 changes the operating state information corresponding to the terminal ID “01ab” of the terminal 10B to “online (during communication)” in the terminal management table 603 as illustrated in FIG. Update the reception date information.

  In step S907, the session management unit 55 of the communication management system 100 creates a session ID that is identification information for identifying a session. Further, the session management unit 55 associates the created session ID with the request source terminal ID (terminal ID of the terminal 10A) and the destination terminal ID (terminal ID of the terminal 10B), as shown in FIG. 7B. Such a session management table 702 is stored.

  In step S908, the device operation right management unit 56 of the communication management system 100 sets the terminal 10A that is the request source terminal as the terminal 10 that has the operation right and the transfer right of the terminal 10B that is the device control terminal. Store in DB5001. For example, in the device operation right management table 601 in FIG. 6A, the device operation right management unit 56 sets the “transfer right holding terminal” and the “operation right holding terminal” corresponding to the device control terminal “01ab” as terminals. The terminal ID “01aa” of 10A is stored.

  In step S909a, the session management unit 55 of the communication management system 100 transmits start instruction information for instructing the terminal 10A to start a session. Similarly, in step S909b, the session management unit 55 of the communication management system 100 transmits start instruction information that instructs the terminal 10B to start a session.

  The start instruction information includes, for example, information related to the session ID and the transfer right and operation right of the terminal 10B. The information regarding the transfer right and operation right of the terminal 10B is information corresponding to, for example, “transfer right holding terminal”, “operation right holding terminal”, etc. in the device operation right management table 601 in FIG.

  Through the above processing, the communication management system 100, the terminal 10A, and the terminal 10B have information regarding the operation right and transfer right of the terminal 10B.

  As an option, the start instruction information transmitted to the terminal 10A may include information corresponding to the “device operation charge terminal” in the device operation right management table 601 in FIG.

  In step S910a, terminal 10A establishes a communication session with data relay unit 57 of communication management system 100 based on the received start instruction information. Similarly, in step S910b, terminal 10B establishes a communication session with data relay unit 57 of communication management system 100 based on the received start instruction information.

  Thereby, in step S911, the terminal 10A and the terminal 10B can participate in a communication session having the same session ID and transmit and receive content data such as video data, for example, to perform a video conference.

  In addition, the terminal 10A can transmit and receive information regarding the operation of the device (for example, the mobile device 20) to and from the terminal 10B using the established communication session. Note that transmission / reception of information related to device operation may be performed using a management information session sei without using a communication session, or via the communication network 2 or the like. What is transmitted / received between 10B may be sufficient.

  In step S912, when the user of the terminal 10A performs input on the operation screen of the device, operation instruction information corresponding to the input operation is transmitted to the terminal 10B. The operation instruction information includes, for example, the terminal ID of the terminal 10A, control information corresponding to the operation content of the user, and the like.

  In step S913, when the operation instruction reception unit 31 of the terminal 10B receives the operation instruction information transmitted from the terminal A, the operation instruction authentication unit 32 performs an authentication process on the received operation instruction information. For example, the operation instruction authentication unit 32 compares the terminal ID included in the operation instruction information with the terminal ID of the terminal having the operation right of the terminal 10B notified from the communication management system 100, and when both match, Allows authentication of operation instruction information. In addition, the operation instruction authentication unit 32 notifies the device control unit 33 of operation instruction information for which authentication is permitted.

  In step S914, the operation instruction authenticating unit 32 of the terminal 10B notifies the authentication result to the terminal 10A. Here, the following description will be given assuming that the authentication result in step S913 is OK (permitted).

  In step S915, the device control unit 33 of the terminal 10B controls the device (the mobile device 20) via the inter-device communication unit 34 based on the control information included in the notified operation instruction information.

  FIG. 10 is a diagram illustrating an example of a display screen of the terminal according to the first embodiment. The display screen 1001 of the terminal 10A that is the device operation terminal includes, for example, an image 1002 captured by the imaging unit 14 of the terminal 10B, an image 1003 captured by the imaging unit 14 of the terminal 10A, a device operation screen 1004, and the like.

  The user of the terminal 10A can hold a video conference with the user on the terminal 10B side while viewing the images 1002 and 1003. Further, the user of the terminal 10A can remotely operate the mobile device 20 via the terminal 10B using the device operation screen 1004.

  In the example of FIG. 10, the device operation screen 1004 displays a switch 1005 for switching on / off the device operation, a switch 1006 for switching the moving speed of the moving device 20, and the like.

  The user of the terminal 10A can select whether or not to operate the mobile device 20 using a switch 1005 that switches on / off the device operation. As an example, the device operation screen 1004 may be displayed by selecting the device operation icon 1011 and may be hidden by turning off the switch 1005.

  Further, the user of the terminal 10 </ b> A can switch the moving speed of the moving device 20 to high speed or low speed by using a switch 1006 that switches the moving speed of the moving device 20.

  The device operation screen 1004 displays a forward button 1007, a left turn button 1008, a right turn button 1009, a reverse button 1010, and the like.

  The user of the terminal 10 </ b> A can make the moving device “forward”, “turn left”, “turn right”, or “retreat” by selecting these buttons.

  For example, even when the user of the terminal 10A cannot directly view the terminal 10B and the mobile device 20, the user can remotely operate the terminal 10B and the mobile device 20 based on the image 1002 captured by the imaging unit 14 of the terminal 10B. Can be moved.

  Next, communication processing according to the movement state of the mobile device 20 and the terminal 10B in the communication system 1 according to the present embodiment will be described.

[First Embodiment]
(Processing while moving)
FIG. 11 is a sequence diagram illustrating an example of processing during movement of the communication system according to the first embodiment. This process shows an example of the process of the communication system 1 when the mobile device 20 and the terminal 10B are moving according to the control information from the terminal 10A, for example. It is assumed that the mobile device 20 can be remotely operated from the terminal 10A via the terminal 10B, for example, by the communication process shown in FIG. 9 at the start of the process shown in FIG.

  In step S1101, the operation input accepting unit 12 of the terminal 10A accepts a moving operation (for example, forward movement) by the operator using, for example, a device operation screen 1004 as shown in FIG.

  In steps S1102 and S1103, the control information transmitting unit 41 of the terminal 10A communicates operation instruction information including control information (forward) corresponding to the moving operation received by the operation input receiving unit 12 and the terminal ID of the terminal 10A. The data is transmitted to the terminal 10B via the management system 100.

  In step S1104, the operation instruction reception unit 31 of the terminal 10B receives operation instruction information from the terminal 10A, and the operation instruction authentication unit 32 performs an authentication process on the received operation instruction information. When the authentication of the operation instruction information is permitted, the device control unit 33 of the terminal 10B starts the advance of the moving device 20 according to the control information (advance) included in the notified operation instruction information. As a result, the mobile device 20 and the terminal 10B are in the “moving” state.

  In step S1105, the movement state determination unit A35 of the terminal 10B determines that the movement state of the mobile device 20 is the “moving” state. For example, the movement state determination unit A35 of the terminal 10B acquires information indicating the current movement state from the device control unit 33 of the terminal 10B or the travel control unit 22 of the movement device 20, and determines the movement state of the movement device 20. To do.

  In step S1106, the movement state determination unit A35 of the terminal 10B notifies the communication management system 100 of movement information indicating that the movement apparatus 20 is moving.

  In step S1107, when the processing method determining unit B59 of the communication management system 100 receives movement information indicating that the mobile device 20 is moving from the terminal 10B, the video data transmitted from the terminal 10B is prioritized in frame rate. Decide to process.

  As described above, in the first embodiment, the communication management system 100 determines the video data processing method. Note that an example of processing in the case of determining the video data processing method in the terminal 10B will be described in the second embodiment.

  In step S <b> 1108, the transmission / reception unit 11 of the terminal 10 </ b> B transmits the video data around the mobile device 20 and the terminal 10 </ b> B captured by the imaging unit 14 to the communication management system 100.

  Preferably, the transmission / reception unit 11 of the terminal 10B uses the video data captured by the imaging unit 14 as H.264, for example. The video is encoded with H.264 SVC and transmitted to the communication management system 100.

  In step S1109, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10B with priority on the frame rate, and generates distribution data to be distributed to the terminal 10A.

  As an example, the video data processing unit 58 generates distribution data with a resolution of “270 × 360” dots and a frame rate of “30 fps”, for example, according to the processing method management information as shown in FIG.

  As another example, as described above, the video data processing unit 58 is the H.264 standard. By generating the distribution data using the base layer and the time layer included in the video data encoded by H.264 SVC, the distribution data giving priority to the frame rate is generated.

  In step S1110, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10A.

  In step S <b> 1111, the display control unit 16 of the terminal 10 </ b> A causes the display unit 109 to display video around the mobile device 20 and the terminal 10 </ b> B using the distribution data distributed from the communication management system 100.

  When the mobile device 20 and the terminal 10B are in the “moving” state, the communication system 1 executes, for example, processing 1131 shown in steps S1105 to S1111 to frame video data transmitted from the terminal 10B to the terminal 10A. Process and distribute with priority on rate.

  Thereby, according to the communication system 1 according to the present embodiment, when the mobile device 20 and the terminal 10B are moving, the video data transmitted from the terminal 10B is transmitted by the operator around the mobile device 20 and the terminal 10B. It is distributed so that it is easy to check the situation in real time.

  In addition, when the mobile device 20 and the terminal 10B are in the “moving” state, the communication system 1 performs processing 1132 shown in steps S1112 to S1118 in parallel (or before and after) processing 1131 shown in steps S1105 to S1111. It may be something that executes.

  In step S1112, the movement state determination unit B42 of the terminal 10A determines that the mobile device 20 and the terminal 10B are in the “moving” state. For example, the movement state determination unit B42 determines the movement state of the mobile device 20 based on the control information transmitted from the control information transmission unit 41 to the terminal 10A in order to operate the mobile device 20 in step S1101.

  In step S1113, the movement state determination unit B42 of the terminal 10A notifies the communication management system 100 of movement information indicating that the movement apparatus 20 is moving.

  In step S1114, when the processing method determination unit B59 of the communication management system 100 receives movement information indicating that the mobile device 20 is moving from the terminal 10A, the video data transmitted from the terminal 10A is prioritized in frame rate. Decide to process.

  In step S <b> 1115, the transmission / reception unit 11 of the terminal 10 </ b> A transmits video data around the terminal 10 </ b> A captured by the imaging unit 14 to the communication management system 100.

  In step S1116, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10A with priority on the frame rate, and generates distribution data to be distributed to the terminal 10B.

  In step S1117, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10B.

  In step S1118, the display control unit 16 of the terminal 10B causes the display unit 109 to display a video around the terminal 10A using the distribution data distributed from the communication management system 100.

(Processing during suspension)
FIG. 12 is a sequence diagram illustrating an example of processing during suspension of the communication system according to the first embodiment. This process shows an example of the process of the communication system 1 when the mobile device 20 and the terminal 10B stop moving according to the control information from the terminal 10A, for example. The basic process flow is the same as the moving process shown in FIG. 11, and thus detailed description of the process is omitted here.

  In step S1201, the operation input accepting unit 12 of the terminal 10A accepts a moving operation (stop) by the operator using, for example, a device operation screen 1004 as shown in FIG.

  In steps S1202 and S1203, the control information transmission unit 41 of the terminal 10A communicates the operation instruction information including the control information (stop) corresponding to the moving operation received by the operation input receiving unit 12 and the terminal ID of the terminal 10A. The data is transmitted to the terminal 10B via the management system 100.

  In step S1204, when the operation instruction receiving unit 31 of the terminal 10B receives the operation instruction information from the terminal 10A and authentication is permitted, the device control unit 33 controls the control information (stop) included in the notified operation instruction information. ), The movement of the moving device 20 is stopped. As a result, the mobile device 20 and the terminal 10B are in the “stopped” state.

  In step S1205, the movement state determination unit A35 of the terminal 10B determines that the movement state of the mobile device 20 is the “stopped” state.

  In step S1206, the movement state determination unit A35 of the terminal 10B notifies the communication management system 100 of movement information indicating that the moving device 20 is stopped.

  In step S1207, when the processing method determination unit B59 of the communication management system 100 receives the movement information indicating that the mobile device 20 is stopped from the terminal 10B, the image data transmitted from the terminal 10B is prioritized for image quality / resolution. It decides to process with.

  In step S <b> 1208, the transmission / reception unit 11 of the terminal 10 </ b> B transmits video data around the mobile device 20 and the terminal 10 </ b> B captured by the imaging unit 14 to the communication management system 100.

  In step S1209, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10B with priority on image quality / resolution, and generates distribution data to be distributed to the terminal 10A.

  As an example, the video data processing unit 58 generates distribution data having a resolution of “540 × 720” dots and a frame rate of “15 fps”, for example, according to the processing method management information as shown in FIG.

  As another example, as described above, the video data processing unit 58 is the H.264 standard. By generating the distribution data using the base layer and the spatial layer included in the video data encoded by H.264 SVC, the distribution data giving priority to the resolution is generated. Alternatively, as described above, the video data processing unit 58 prioritizes S / N (an example of image quality) by generating distribution data using the base layer and the S / N ratio layer included in the video data. Generate distribution data.

  In step S1210, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10A.

  In step S <b> 1211, the display control unit 16 of the terminal 10 </ b> A causes the display unit 109 to display video around the mobile device 20 and the terminal 10 </ b> B using the distribution data distributed from the communication management system 100.

  When the mobile device 20 and the terminal 10B are in the “stopped” state, the communication system 1 executes the processing 1231 shown in steps S1205 to S1211, for example, to process the video data transmitted from the terminal 10B to the terminal 10A.・ Process and distribute with priority on resolution.

  Thereby, according to the communication system 1 according to the present embodiment, when the mobile device 20 and the terminal 10B are stopped, the video data transmitted from the terminal 10B can easily confirm the participant's face, characters, and the like. So that it can be easily confirmed.

  In addition, when the mobile device 20 and the terminal 10B are in the “stopped” state, the communication system 1 performs processing 1232 shown in steps S1212 to S1218 in parallel (or before and after) the processing 1231 shown in steps S1205 to S1211. It may be something that executes.

  In step S1212, the movement state determination unit B42 of the terminal 10A determines that the mobile device 20 and the terminal 10B are in the “stopped” state.

  In step S1213, the movement state determination unit B42 of the terminal 10A notifies the communication management system 100 of movement information indicating that the moving device 20 is stopped.

  In step S1214, when the processing method determination unit B59 of the communication management system 100 receives movement information indicating that the mobile device 20 is stopped from the terminal 10A, the image data transmitted from the terminal 10A is prioritized for image quality / resolution. It decides to process with.

  In step S <b> 1215, the transmission / reception unit 11 of the terminal 10 </ b> A transmits video data around the terminal 10 </ b> A captured by the imaging unit 14 to the communication management system 100.

  In step S1216, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10A with priority on image quality and resolution, and generates distribution data to be distributed to the terminal 10B.

  In step S1217, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10B.

  In step S1118, the display control unit 16 of the terminal 10B causes the display unit 109 to display a video around the terminal 10A using the distribution data distributed from the communication management system 100.

[Second Embodiment]
In the second embodiment, an example of processing when the terminal 10 determines the video data processing method will be described.

(Processing while moving)
FIG. 13 is a sequence diagram illustrating an example of processing during movement of the communication system according to the second embodiment. The basic process flow is the same as the moving process according to the first embodiment shown in FIG. 11, and thus detailed description of the process is omitted here.

  In step S1301, the operation input accepting unit 12 of the terminal 10A accepts a moving operation (for example, forward movement) by the operator using, for example, a device operation screen 1004 as shown in FIG.

  In steps S1302 and S1303, the control information transmitting unit 41 of the terminal 10A communicates operation instruction information including control information (forward) corresponding to the moving operation received by the operation input receiving unit 12 and the terminal ID of the terminal 10A. The data is transmitted to the terminal 10B via the management system 100.

  In step S1304, when the operation instruction receiving unit 31 of the terminal 10B receives the operation instruction information from the terminal 10A and authentication is permitted, the device control unit 33 controls the control information (forward) that is included in the notified operation instruction information. ), The moving device 20 starts to advance. As a result, the mobile device 20 and the terminal 10B are in the “moving” state.

  In step S1305, the movement state determination unit A35 of the terminal 10B determines that the movement state of the mobile device 20 is the “moving” state.

  In step S1306, when the movement state determination unit A35 determines that the movement state of the mobile device 20 is the “moving” state, the processing method determination unit A36 of the terminal 10B converts the video data transmitted by the terminal 10B to the frame rate. Decide to process with priority.

  In step S1307, the processing method determination unit A36 of the terminal 10B notifies the communication management system 100 that the processing method of the video data transmitted by the terminal 10B is frame rate priority.

  In step S1308, the video data processing unit 58 of the communication management system 100 sets the video data transmitted from the terminal 10B to be processed with priority on the frame rate in accordance with the video data processing method notified from the terminal 10B.

  In this manner, in the second embodiment, the terminal 10B determines the video data processing method. Thereby, for example, the processing load of the communication management system 100 when the number of terminals 10 is increased can be reduced.

  In step S <b> 1309, the transmission / reception unit 11 of the terminal 10 </ b> B transmits the video data around the mobile device 20 and the terminal 10 </ b> B captured by the imaging unit 14 to the communication management system 100.

  In step S1310, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10B with priority on the frame rate, and generates distribution data to be distributed to the terminal 10A.

  In step S1311, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10A.

  In step S1312, the display control unit 16 of the terminal 10A causes the display unit 109 to display video around the mobile device 20 and the terminal 10B using the distribution data distributed from the communication management system 100.

  By the above processing, as in the first embodiment, when the mobile device 20 and the terminal 10B are moving, the video data transmitted from the terminal 10B is the situation of the surroundings of the mobile device 20 and the terminal 10B by the operator. Is distributed so that it can be easily confirmed in real time.

  In addition, when the mobile device 20 and the terminal 10B are in the “moving” state, the communication system 1 performs processing 1332 shown in steps S1313 to S1320 in parallel (or before and after) processing 1331 shown in steps S1305 to S1312. It may be something that executes.

  In step S1313, the movement state determination unit B42 of the terminal 10A determines that the mobile device 20 and the terminal 10B are in the “moving” state.

  In step S1314, when the movement state determination unit B42 determines that the movement state of the mobile device 20 is the “moving” state, the processing method determination unit C43 of the terminal 10A converts the video data transmitted by the terminal 10A to the frame rate. Decide to process with priority.

  In step S1315, the processing method determination unit C43 of the terminal 10A notifies the communication management system 100 that the processing method of the video data transmitted by the terminal 10A is frame rate priority.

  In step S1316, the video data processing unit 58 of the communication management system 100 sets the video data transmitted from the terminal 10A to be processed with priority on the frame rate according to the video data processing method notified from the terminal 10A.

  In step S <b> 1317, the transmission / reception unit 11 of the terminal 10 </ b> A transmits video data around the terminal 10 </ b> A captured by the imaging unit 14 to the communication management system 100.

  In step S1318, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10A with priority on the frame rate, and generates distribution data to be distributed to the terminal 10B.

  In step S1319, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10B.

  In step S1320, the display control unit 16 of the terminal 10B causes the display unit 109 to display a video around the terminal 10A using the distribution data distributed from the communication management system 100.

(Processing during suspension)
FIG. 14 is a sequence diagram illustrating an example of processing during suspension of the communication system according to the second embodiment. Note that the basic processing flow is the same as the stopped processing according to the first embodiment shown in FIG. 12, and therefore detailed description of the processing is omitted here.

  In step S1401, the operation input accepting unit 12 of the terminal 10A accepts a moving operation (stop) by the operator using, for example, a device operation screen 1004 as shown in FIG.

  In steps S1402 and S1403, the control information transmitting unit 41 of the terminal 10A communicates operation instruction information including control information (stop) corresponding to the moving operation received by the operation input receiving unit 12 and the terminal ID of the terminal 10A. The data is transmitted to the terminal 10B via the management system 100.

  In step S1404, when the operation instruction receiving unit 31 of the terminal 10B receives the operation instruction information from the terminal 10A and authentication is permitted, the device control unit 33 controls the control information (stop) included in the notified operation instruction information. ), The movement of the moving device 20 is stopped.

  In step S1405, the movement state determination unit A35 of the terminal 10B determines that the movement state of the mobile device 20 is the “stopped” state.

  In step S1406, when the movement state determination unit A35 determines that the movement state of the mobile device 20 is the “stopped” state, the processing method determination unit A36 of the terminal 10B converts the video data transmitted by the terminal 10B to the image quality / Decide to process with priority on resolution.

  In step S1407, the processing method determination unit A36 of the terminal 10B notifies the communication management system 100 that the processing method of the video data transmitted by the terminal 10B is image quality / resolution priority.

  In step S1408, the video data processing unit 58 of the communication management system 100 sets the video data transmitted from the terminal 10B to be processed with priority on image quality / resolution in accordance with the video data processing method notified from the terminal 10B.

  In step S <b> 1409, the transmission / reception unit 11 of the terminal 10 </ b> B transmits the video data around the mobile device 20 and the terminal 10 </ b> B captured by the imaging unit 14 to the communication management system 100.

  In step S1410, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10B with priority on image quality and resolution, and generates distribution data to be distributed to the terminal 10A.

  In step S1411, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10A.

  In step S1412, the display control unit 16 of the terminal 10A causes the display unit 109 to display video around the mobile device 20 and the terminal 10B using the distribution data distributed from the communication management system 100.

  By the above processing, as in the first embodiment, when the mobile device 20 and the terminal 10B are stopped, the video data transmitted from the terminal 10B can easily check the participant's face, characters, etc. It is distributed so that it can be easily confirmed.

  In addition, when the mobile device 20 and the terminal 10B are in the “stopped” state, the communication system 1 performs processing 1432 shown in steps S1413 to S1420 in parallel (or before and after) processing 1431 shown in steps S1405 to S1412. It may be something that executes.

  In step S1413, the movement state determination unit B42 of the terminal 10A determines that the mobile device 20 and the terminal 10B are in the “stopped” state.

  In step S1414, when the movement state determination unit B42 determines that the movement state of the mobile device 20 is the “stopped” state, the processing method determination unit C43 of the terminal 10A converts the video data transmitted by the terminal 10A to the image quality / Decide to process with priority on resolution.

  In step S1415, the processing method determination unit C43 of the terminal 10A notifies the communication management system 100 that the processing method of the video data transmitted by the terminal 10A is image quality / resolution priority.

  In step S1416, the video data processing unit 58 of the communication management system 100 sets the video data transmitted from the terminal 10A to be processed with priority on image quality / resolution in accordance with the video data processing method notified from the terminal 10A.

  In step S <b> 1417, the transmission / reception unit 11 of the terminal 10 </ b> A transmits the video data around the terminal 10 </ b> A captured by the imaging unit 14 to the communication management system 100.

  In step S1418, the video data processing unit 58 of the communication management system 100 processes the video data received from the terminal 10A with priority on image quality and resolution, and generates distribution data to be distributed to the terminal 10B.

  In step S1419, the data relay unit 57 of the communication management system 100 distributes the distribution data generated by the video data processing unit 58 to the terminal 10B.

  In step S1420, the display control unit 16 of the terminal 10B uses the distribution data distributed from the communication management system 100 to display the video around the terminal 10A on the display unit 109.

[Third Embodiment]
In the communication system 1, it is desirable that processing for automatically processing video data according to the movement state of the mobile device 20 and the terminal 10B can be switched between permitted / inhibited (or valid / invalid) by menu setting or the like. This is because the user does not necessarily desire automatic processing of video data.

  In the third embodiment, an example of processing in a case where whether processing for automatically processing video data is permitted or prohibited by a menu setting will be described.

  FIG. 15 is a flowchart illustrating an example of processing of the communication management system according to the third embodiment.

  In step S1501, the processing method determination unit B59 of the communication management system 100 determines whether or not the automatic processing function for the distribution video is permitted. Whether the automatic processing function of the distribution video is permitted or prohibited is assumed to be set in advance by the user or the administrator using a setting screen 1600 as shown in FIG. 16, for example.

  FIG. 16 is a diagram illustrating an example of a setting screen according to the third embodiment. In the example of FIG. 16, the setting screen 1600 includes a setting item 1601 of “automatic distribution video processing function”, so that permission 1602 or rejection 1603 can be selected.

  For example, the terminal 10 displays a setting screen 1600 as shown in FIG. 16 on the display unit 109 and notifies the communication management system 100 of the setting contents set by the user or the administrator. Alternatively, the user or the administrator may perform settings by displaying a setting screen 1600 provided by the communication management system 100 using a web browser or the like.

  If it is determined in step S1501 that the distribution video automatic processing function is not permitted, the processing method determination unit B59 shifts the processing to step S1502. On the other hand, when it is determined that the automatic processing function of the distribution video is permitted, the processing method determination unit B59 shifts the processing to step S1503.

  In step S1502, the video data processing unit 58 of the communication management system 100 processes the video data according to a predetermined processing method. Alternatively, the video data processing unit 58 may not process the video data.

  In step S1503, the processing method determination unit B59 of the communication management system 100 determines whether or not the terminal 10 that is the target of the automatic processing of the distribution video is moving.

  When the target terminal 10 is moving, the video data processing unit 58 of the communication management system 100 processes the video data received from the target terminal 10 with priority on the frame rate.

  On the other hand, when the target terminal 10 is not moving (when stopped), the video data processing unit 58 of the communication management system 100 processes the video data received from the target terminal 10 with priority on image quality and resolution. To do.

  In step S1506, the data relay unit 57 of the communication management system 100 distributes the processed distribution data to the destination terminal 10.

  With the above processing, the user or administrator can switch permission / prohibition of processing for automatically processing video data of the communication system 1 by menu setting or the like.

  As described above, according to each of the above-described embodiments, communication that distributes video corresponding to the moving state of the mobile device 20 from the terminal 10B mounted on the mobile device 20 to the terminal 10A that performs the moving operation of the mobile device 20 A system 1 can be provided.

  For example, when the communication system 1 includes the management device 50 and the relay device 30 as illustrated in FIG. 1, the data relay unit 57 and the video data processing unit 58 are included in the relay device 30, for example.

  The system configuration described in each of the above embodiments is an example, and the communication system 1 can have various system configurations.

  For example, the processing method determination unit that determines the processing method of the video data of the terminal 10B may be included in the terminal 10B, or may be included in, for example, the management device 50 or the relay device 30 in FIG. .

  Further, the video data processing unit 58 that processes the video data of the terminal 10B may be included in the relay device 30 of FIG. 1 or may be included in the terminal 10B, for example.

  Furthermore, each function included in the mobile device 20 may be included in the terminal 10B. Alternatively, each function of the terminal 10B may be included in the mobile device 20.

1 Communication System 10 Terminal (Information Processing Device)
10A terminal (second information processing apparatus)
10B terminal (first information processing apparatus)
DESCRIPTION OF SYMBOLS 14 Image pick-up part 16 Display control part 20 Mobile device 30 Relay apparatus 35 Movement state judgment part A (an example of a judgment part)
36 Machining method determination unit A (an example of a determination unit)
42 Movement state determination unit B (another example of determination unit)
43 Processing Method Determination Unit C (Another Example of Determination Unit)
50 management device 57 data relay unit (distribution unit)
58 Video data processing unit (generation unit)
59 Processing Method Determination Unit B (Another Example of Determination Unit)
100 Communication management system

JP 2013-048356 A Special table 2012-523890 gazette

Claims (10)

A first information processing apparatus mounted on a device having a moving function and controlling the moving function according to predetermined control information; a moving operation to the device is received; and the control information corresponding to the moving operation is A communication system including a second information processing device that transmits to one information processing device,
An imaging unit for imaging video data around the device;
A determination unit for determining a movement state of the device;
A determination unit that determines a processing method of the video data to be transmitted from the first information processing device to the second information processing device to a different processing method according to a movement state of the device;
A communication system. The determination unit
The communication system according to claim 1, wherein it is determined whether the device is in a first state where the device is moving or whether the device is in a second state where the device is not moving. The determination unit
The communication according to claim 1, wherein it is determined whether the moving speed of the device is in a first state where the moving speed is equal to or higher than a predetermined speed or whether the moving speed of the device is a second state where the moving speed is less than the predetermined speed. system. The determination unit
When the device is in the first state, the processing method of the video data is determined as the first processing method giving priority to the frame rate,
The communication system according to claim 2 or 3, wherein when the device is in the second state, the processing method of the video data is determined to be a second processing method giving priority to image quality or resolution. A generator that processes the video data according to the processing method and generates distribution data to be distributed to the second information processing apparatus;
The generator is
When the video data processing method is determined as the first processing method, the distribution data is generated at a higher frame rate than when the video data processing method is determined as the second processing method,
When the video data processing method is determined to be the second processing method, the distribution data is generated with better image quality or higher resolution than when the video data processing method is determined to be the first processing method. The communication system according to claim 4. A distribution unit that distributes the distribution data generated by the generation unit to the second information processing apparatus;
A display control unit that displays video around the device on the display unit of the second information processing device based on distribution data distributed from the distribution unit;
The communication system according to claim 5. The communication system is:
A communication management system for managing communication between the first information processing apparatus and the second information processing apparatus;
The communication management system includes:
The communication system according to claim 5 or 6, comprising the determination unit or the generation unit. A first information processing apparatus included in the communication system according to any one of claims 1 to 6,
A communication unit that communicates with a second information processing apparatus that receives a movement operation to a device having a movement function and transmits the control information corresponding to the movement operation to the first information processing apparatus;
An imaging unit for imaging video data around the device;
A determination unit for determining a movement state of the device;
A determination unit that determines a processing method of the video data to be transmitted from the first information processing device to the second information processing device to a different processing method according to a movement state of the device;
An information processing apparatus. A first information processing apparatus mounted on a device having a moving function and controlling the moving function according to predetermined control information; a moving operation to the device is received; and the control information corresponding to the moving operation is A first information processing apparatus of a communication system including a second information processing apparatus that transmits to the first information processing apparatus;
Imaging video data around the device;
Determining a movement state of the device;
Determining a processing method of the video data to be transmitted from the first information processing device to the second information processing device as a different processing method according to a movement state of the device;
A program for running A first information processing apparatus mounted on a device having a moving function and controlling the moving function according to predetermined control information; a moving operation to the device is received; and the control information corresponding to the moving operation is A communication method in a communication system including a second information processing device that transmits to one information processing device,
The first information processing apparatus imaging video data around the device;
The first information processing apparatus determining a movement state of the device;
A communication management system that manages communication between the first information processing apparatus or the first information processing apparatus and the second information processing apparatus receives the second information from the first information processing apparatus. Determining a processing method of the video data to be transmitted to a processing device as a different processing method according to a movement state of the device;
Including a communication method.

Images