JP2017069796A - Inter-device communication system, information transmitter and information receiver - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the reproduction of a moving image according to a communication environment.SOLUTION: An inter-device communication system includes: a communication state determination unit which determines a communication state of a session between an information transmitter and an information receiver, on the basis of the determination criterion information of a session communication state; a parameter calculation unit which calculates a coding parameter of moving image data communicated through the session, on the basis of the session communication state determined by the communication state determination unit; a generation unit which generates the moving image data on the basis of the coding parameter calculated by the calculation unit; a transmitter unit which transmits the moving image data, which is generated by the generation unit, from the information transmitter to the information receiver through the session; a receiver unit which receives the moving image data transmitted from the transmitter unit; a display control unit which displays the moving image based on the moving image data received by the receiver unit; and a parameter change unit which changes the coding parameter calculated by the calculation unit, on the basis of the communication state of the moving image data received by the receiver unit and the session communication state determined by the communication state determination unit.SELECTED DRAWING: Figure 9

Description

  Embodiments described herein relate generally to an inter-device communication system, an information transmission device, and an information reception device.

  2. Description of the Related Art Conventionally, there has been known a communication system that enables video data to be transmitted and received between devices using wired communication or wireless communication and that a moving image can be reproduced on a receiving device (see, for example, Patent Document 1). .

JP 2000-261574 A

  Depending on the situation of the communication environment, the moving image may not be played smoothly. However, in the conventional communication system as described above, even if the communication environment is not suitable for moving image reproduction, the moving image has to be transmitted and received as it is. That is, in the conventional communication system as described above, it is not possible to reproduce a moving image according to the communication environment.

  It is an object of the present invention to provide an inter-device communication system, an information transmission device, and an information reception device that can reproduce a moving image according to a communication environment.

  In one embodiment of the present invention, a communication status determination unit that determines a communication status of a session between an information transmission device and an information reception device based on determination criterion information of the communication status of the session, and the communication status determination unit Based on the communication status of the session determined by the parameter, a parameter calculation unit that calculates an encoding parameter of moving image data communicated by the session, and a moving image based on the encoding parameter calculated by the parameter calculation unit A moving image data generating unit for generating data, a moving image data transmitting unit for transmitting the moving image data generated by the moving image data generating unit from the information transmitting device to the information receiving device through the session, and the moving image A moving image data receiving unit that receives the moving image data transmitted by the image data transmitting unit, and the moving image data received by the moving image data receiving unit. Based on a display control unit that displays a moving image based on image data, a communication state of the moving image data received by the moving image data reception unit, and a communication state of the session determined by the communication state determination unit, An inter-device communication system comprising: a parameter changing unit that changes the encoding parameter calculated by the parameter calculating unit.

In the inter-device communication system according to an embodiment of the present invention, the communication status of the moving image data received by the moving image data receiving unit includes a packet arrival status of the moving image data. The communication status includes the status of the available bandwidth of the session communication,
The parameter changing unit changes the coding parameter based on the arrival status of the moving image data packet and the available bandwidth status.

  In the inter-device communication system according to an embodiment of the present invention, the available bandwidth status of the session communication includes an estimated available bandwidth status estimated based on the session communication, and the parameter The changing unit changes the encoding parameter based on the arrival status of the moving image data packet and the estimated available bandwidth status.

  In the inter-device communication system according to an embodiment of the present invention, the packet arrival status includes the number of lost packets of the moving image data, and the encoding parameter is reproduced by the moving image data. A parameter indicating the quality of the moving image is included, and the parameter changing unit has a predetermined number of packets lost in the moving image data equal to or less than a predetermined first threshold value, and a usable second bandwidth of the session communication is a predetermined second value. When the threshold value is exceeded, the parameter indicating the quality of the moving image is changed in the high quality direction.

  Further, in the inter-device communication system according to an embodiment of the present invention, when the number of lost packets of the moving image data is equal to or less than a third threshold value that is greater than the first threshold value, the parameter changing unit is , Changing the encoding parameter by changing a parameter indicating the quality of the moving image in a low quality direction, and the moving image data generation unit reproduces the reproduction based on the encoding parameter changed by the parameter changing unit. The moving image data with reduced quality of the moving image is generated, and the display control unit displays the moving image based on the moving image data with reduced quality of the reproduced moving image, and the number of lost packets of the moving image data Is greater than the third threshold value, the display control unit may replace the moving image with a still image based on the moving image data received by the moving image data receiving unit. To display.

  Moreover, one Embodiment of this invention produces | generates the test | inspection information used for determination of the communication condition of the said session based on the 1st criteria information of the communication condition of the session between an own apparatus and an information receiver. A test information generating unit; a test information transmitting unit for transmitting the test information generated by the test information generating unit to the information receiving device; a second test information transmitted; and a second communication status of the session. A moving image data generating unit that generates moving image data using an encoding parameter calculated by a parameter calculating unit based on a communication status of the session determined by the information receiving device based on the determination criterion information; A moving image data transmitting unit configured to transmit the moving image data generated by the image data generating unit from the own device to the information receiving device through the session, and generating the moving image data When the coding parameter is changed according to the communication status of the moving image data transmitted by the session, the moving image data transmitted by the session based on the changed coding parameter Is an information transmitting device for generating

  In the information transmitting apparatus according to the embodiment of the present invention, the communication status of the moving image data received by the moving image data receiving unit includes a packet arrival status of the moving image data, and the communication of the session The situation includes the situation of the available bandwidth of the communication of the session, and the moving image data generation unit includes a parameter changing unit that changes the encoding parameter calculated by the parameter calculating unit. The moving image data is generated using the encoding parameter changed based on the packet arrival status and the available bandwidth status.

  In the information transmitting apparatus according to the embodiment of the present invention, the status of the available bandwidth of the session communication includes an estimated available bandwidth status estimated based on the session communication, and the moving image The data generation unit generates the moving image data by using the coding parameter changed by the parameter changing unit based on the arrival status of the moving image data packet and the estimated available bandwidth.

  In the information transmitting apparatus according to the embodiment of the present invention, the packet arrival status includes the number of lost packets of the moving image data, and the encoding parameter includes a moving image reproduced by the moving image data. A parameter indicating the quality of the video, and the moving image data generating unit is configured such that the parameter changing unit has a number of lost packets of the moving image data equal to or less than a predetermined first threshold value, and an available bandwidth of communication of the session When the value exceeds a predetermined second threshold value, the moving image data is generated using the encoding parameter obtained by changing the parameter indicating the quality of the moving image in the high quality direction.

  In the information transmitting apparatus according to the embodiment of the present invention, the moving image data generating unit may be configured such that the parameter changing unit has a third threshold in which the number of lost packets of moving image data is greater than the first threshold value. The moving image data in which the quality of the reproduced moving image is reduced is generated by using the encoding parameter changed by changing the parameter indicating the quality of the moving image in the low quality direction when the value is less than the value.

  In addition, according to an embodiment of the present invention, inspection information generated by the information transmission device based on first determination criterion information on a communication state of a session between the own device and the information transmission device is transmitted to the information transmission device. The communication status determination unit that determines the communication status of the session based on the test information received by the test information, the test information received by the test information reception unit, and the second determination criterion information of the communication status of the session And moving image data generated by the information transmitting device using the encoding parameter calculated by the parameter calculating unit based on the communication status of the session determined by the communication status determining unit, from the information transmitting device An image data receiving unit, a display control unit for displaying a moving image based on the moving image data received by the moving image data receiving unit, and the moving image data receiving unit A parameter changing unit that changes the encoding parameter calculated by the parameter calculating unit based on a communication state of image data and a communication state of the session determined by the communication state determining unit; The data receiving unit receives the moving image data generated by the information transmitting device using the encoding parameter changed by the parameter changing unit from the information transmitting device, and the display control unit is configured by the parameter changing unit. The information receiving apparatus displays a moving image based on the moving image data generated using the changed encoding parameter.

  In the information receiving apparatus according to the embodiment of the present invention, the communication status of the moving image data received by the moving image data receiving unit includes a packet arrival status of the moving image data, and the communication of the session The situation includes the status of the available bandwidth of the communication of the session, the moving image data generation unit, the parameter changing unit, the status of the arrival of the moving image data packet, the status of the available bandwidth, Based on the above, the encoding parameter is changed.

  Further, in the information reception device of one embodiment of the present invention, the status of the available bandwidth of the session communication includes the status of the estimated available bandwidth estimated based on the communication of the session, and the parameter change The unit changes the encoding parameter based on the arrival status of the moving image data packet and the estimated available bandwidth status.

  In the information receiving apparatus according to the embodiment of the present invention, the packet arrival status includes the number of lost packets of the moving image data, and the encoding parameter includes a moving image reproduced by the moving image data. A parameter indicating the quality of the session, wherein the parameter changing unit has a number of lost packets of the moving image data equal to or less than a predetermined first threshold, and an available bandwidth of the session communication is a predetermined second threshold. When the value is exceeded, the parameter indicating the quality of the moving image is changed in the high quality direction.

  Further, in the information receiving apparatus of one embodiment of the present invention, when the number of lost packets of the moving image data is equal to or less than a third threshold value that is larger than the first threshold value, the parameter changing unit is The encoding parameter is changed by changing a parameter indicating the quality of the moving image in a low quality direction, and the moving image data receiving unit is configured to change the moving image based on the encoding parameter changed by the parameter changing unit. Receiving the moving image data generated by the data generation unit and having a reduced quality of the reproduced moving image, the display control unit displaying a moving image based on the moving image data having the reduced quality of the reproduced moving image; When the number of lost packets of the moving image data is larger than the third threshold, the display control unit replaces the moving image before the moving image data receiving unit has received it. Displaying a still image based on the moving image data.

  According to the present invention, it is possible to provide an inter-device communication system, an information transmission device, and an information reception device that can reproduce a moving image according to a communication environment.

It is a schematic diagram which shows an example of the outline | summary of the information communication function which the communication system between apparatuses of 1st Embodiment provides. It is a schematic diagram which shows an example of a structure of the terminal device of this embodiment. It is a schematic diagram which shows an example of starting operation of the information communication function which the communication system between apparatuses of this embodiment provides. It is a schematic diagram which shows an example of a structure of the communication system between apparatuses of this embodiment. It is a block diagram which shows an example of a function structure of the communication system between apparatuses of this embodiment. It is a flowchart which shows an example of operation | movement of the communication system between apparatuses of this embodiment. It is a flowchart which shows an example of the operation | movement of the test | inspection of the criteria information performed by the communication condition determination part of this embodiment. It is a flowchart which shows an example of the operation | movement of the update of the criteria information by the communication system between apparatuses of this embodiment. It is a block diagram which shows an example of a function structure of the communication system between apparatuses of 2nd Embodiment. It is a flowchart which shows an example of operation | movement of the communication system between apparatuses in this embodiment. It is a flowchart which shows an example of the change operation | movement of the encoding parameter by the parameter change part of this embodiment. It is a block diagram which shows an example of a function structure of the communication system between apparatuses of 3rd Embodiment. It is a figure which shows an example of the moving image data memorize | stored in the moving image data storage part of this embodiment. It is a flowchart which shows an example of operation | movement of the communication system between apparatuses in this embodiment. It is a flowchart which shows an example of the operation | movement of control of the moving image data by the moving image data control part of this embodiment.

[First Embodiment]
Hereinafter, a first embodiment of the inter-device communication system 1 according to the present embodiment will be described with reference to the drawings.
FIG. 1 is a schematic diagram illustrating an example of an outline of an information communication function provided by the inter-device communication system 1. The inter-device communication system 1 of the present embodiment provides an information communication function between a plurality of devices. In the inter-device communication system 1, the types of these devices are not limited. For example, the inter-device communication system 1 provides an information communication function between terminal devices or between a server device and a terminal device. That is, the devices referred to here include a terminal device and a server device. In this example, a case where a plurality of devices are all mobile terminal devices will be described. Here, the mobile terminal device is a communication device that can communicate with each other by voice or text, and is, for example, a mobile phone or a smartphone. This portable terminal device can be connected to a high-speed data communication line capable of transmitting and receiving voice information and image information through the same line. The high-speed data communication line is, for example, an LTE (Long Term Evolution) communication network. This high-speed data communication line provides a service (VoLTE; Voice over LTE) that enables voice calls in addition to transmission and reception of digital information such as images and characters. In the inter-device communication system 1, the communication method and the like are not limited to these. In the following description, this mobile terminal device is simply referred to as a terminal device 10. Further, as necessary, the terminal device 10 on the transmission side is described as an information transmission device or terminal device 10-1, and the terminal device 10 on the reception side is described as an information reception device or terminal device 10-2.

The inter-device communication system 1 provides an information communication function between a plurality of terminal devices 10. This information communication function is a function of communicating image information or audio information among a plurality of terminal devices 10. As an example of the information communication function, the inter-device communication system 1 may provide an information sharing function for bidirectionally sharing image information or audio information between a plurality of terminal devices 10.
Here, a case where the transmission-side terminal device 10 transmits moving image data to the reception-side terminal device 10 will be described as an example. In this example, the inter-device communication system 1 displays a moving image displayed on the transmission-side terminal device 10 on the reception-side terminal device 10.

  In the specific example illustrated in FIG. 1, the user of the terminal device 10-1 writes an arrow indicating the destination on the map displayed on the screen of the terminal device 10-1. By this operation, the terminal device 10-1 generates a moving image P10 in which a map and an arrow are combined. The terminal device 10-1 transmits the moving image data of the moving image P10 to the terminal device 10-2. Thereby, the moving image P11 corresponding to the moving image P10 is displayed on the screen of the terminal device 10-2.

As an example of the moving image data transmission function, when the terminal device 10-1 includes a camera capable of capturing a moving image, the moving image captured by the camera is transmitted to the terminal device 10-2. You may do.
Further, as an example of the moving image data transmission function, moving image data generated by a moving image providing server (not shown) may be transmitted to the terminal device 10.
Note that the generation of moving image data here refers to the case where moving image content is generated, such as a moving image captured in real time with a camera, or the case where the resolution or frame rate of moving image data prepared in advance is changed. And are included. The generation of moving image data includes changing the data amount of moving image data transmitted and received between apparatuses.

  In the following description, the function of causing the terminal device 10-2 to display the screen displayed on the terminal device 10-1 among the information communication functions is also referred to as a display screen sharing function. The configuration of the terminal device 10 will be described with reference to FIG.

[Configuration of terminal device]
FIG. 2 is a schematic diagram illustrating an example of the configuration of the terminal device 10 according to the present embodiment. The terminal device 10 includes a display unit 100, an operation detection unit 110, an imaging unit 120, a sound reproduction unit 130, a sound collection unit 140, a communication unit 150, and a control unit 160.

The display unit 100 includes a liquid crystal display and displays images such as characters and photographs.
The operation detection unit 110 includes a transparent touch panel sensor that covers the display unit 100 and detects a user operation.
The imaging unit 120 includes a camera that can capture still images and moving images. The camera includes an imaging element such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor and an optical member such as a lens, and captures an image around the terminal device 10.

The sound reproducing unit 130 includes a speaker and reproduces sound based on input sound information.
The sound collection unit 140 includes a microphone, collects sounds around the terminal device 10, and converts the collected sounds into sound information. The inter-device communication system 1 can also share the sound collected by the sound collection unit 140 between the terminal devices 10.

  The communication unit 150 includes a communication control IC (Integrated Circuit), and performs communication with other devices. The terminal device 10 communicates with other devices by the communication operation by the communication unit 150. In the following description, description of the configuration and operation of the communication unit 150 is omitted.

  The control unit 160 includes a CPU (Central Processing Unit) and controls each unit included in the terminal device 10. The terminal device 10 performs operations such as display on the display unit 100, detection of an operation by the operation detection unit 110, communication with another device by the communication unit 150, and the like by the control of each unit by the control unit 160.

[Overview of selecting information communication function]
Next, an outline of an operation for the user to activate the information communication function will be described with reference to FIG.
FIG. 3 is a schematic diagram illustrating an example of an activation operation of the information communication function provided by the inter-device communication system 1 of the present embodiment. Here, a case where user A (Mr. A) and user B (Mr. B) use the display screen sharing function using the terminal device 10 will be described as an example. In the description of this example, the user A uses the terminal device 10-1. User B uses terminal device 10-2.

  The user A and the user B use the display screen sharing function by sequentially performing the operations shown in FIGS. 3 (A) to 3 (E). The user A and the user B are making a voice call via the LTE communication network. FIG. 3A shows an example of an image displayed on each terminal device 10 when the user A and the user B are making a voice call via the LTE communication network.

  At this time, the image PMB is displayed on both the display unit 100 of the terminal device 10-1 of the user A and the display unit 100 of the terminal device 10-2 of the user B. This image PMB indicates that the terminal device 10 on the communication partner side supports the information communication function. In this example, the display of the image PMB indicates that both the terminal device 10-1 and the terminal device 10-2 are compatible with the display screen sharing function.

  User A taps the image PMB of the shared menu button displayed on the display unit 100 of the terminal device 10-1. When the operation detection unit 110 of the terminal device 10-1 detects this tap operation, a launcher image P20 is displayed (see FIG. 3B). The launcher image P20 is an example of the menu image described above. In this example, the launcher image P20 includes an icon image P21 of a display screen sharing function. When the user taps the position on the display unit 100 where the icon image P21 is displayed, the function associated with the icon image P21 is activated.

  When the operation detection unit 110 of the terminal device 10-1 detects this tap operation, a function explanation image P22 of the display screen sharing function is displayed (see FIG. 3C). When the operation detection unit 110 of the terminal device 10-1 detects this tap operation, a screen sharing start consent image P23 is displayed on the display unit 100 of the sharing destination terminal device 10-2. The consent image P23 is an image displayed on the requesting terminal device 10 for information sharing, and prompts the user to perform an operation indicating whether or not to accept the information sharing request by the requesting terminal device 10. It is an image. In this example, the consent image P23 is an image prompting the user B to perform a tap operation indicating whether or not to accept the screen sharing request from the user A.

  Next, the user B taps the image P24 of the consent button of the consent image P23. When the operation detection unit 110 of the terminal device 10-2 detects this tap operation, an image P25 indicating that it is connected is displayed on each display unit 100 of the terminal device 10-2 and the terminal device 10-1 (FIG. 3 ( See D)). At this time, an inter-terminal communication session for display screen sharing is established between the terminal device 10-1 and the terminal device 10-2.

  Next, when a session for inter-terminal communication for display screen sharing is established between the terminal device 10-1 and the terminal device 10-2, an image P26 displayed on the display unit 100 of the terminal device 10-1 is displayed. Is displayed as an image P27 on the display unit 100 of the terminal device 10-2 (see FIG. 3E). That is, display screen sharing is started between the terminal device 10-1 and the terminal device 10-2.

[Configuration of inter-terminal communication system]
Next, the configuration of the inter-device communication system 1 that provides the above-described information communication function will be described with reference to FIG.
FIG. 4 is a schematic diagram illustrating an example of the configuration of the inter-device communication system 1 of the present embodiment. The inter-device communication system 1 includes the following devices in addition to the plurality of terminal devices 10 (in this example, the terminal device 10-1 and the terminal device 10-2). That is, the inter-device communication system 1 includes a signaling server 20, an authentication server 30, a push server 40, and a relay server 50.

  The signaling server 20 manages a session of information sharing communication performed between the terminal devices 10. Specifically, the signaling server 20 can communicate with a plurality of terminal devices 10, the authentication server 30, and the push server 40. When the terminal device 10 is compatible with the information communication function, the signaling server 20 performs registration (initial registration) of information of the terminal device 10. Further, when there is a request for information sharing from the terminal device 10, the signaling server 20 performs signaling processing for establishing a communication session with the sharing destination terminal device 10. Further, when there is a request for information sharing from the terminal device 10, the signaling server 20 returns a response as to whether or not the terminal device 10 that is the sharing request destination supports the information communication function. To do.

The signaling server 20 includes a terminal information reception unit, a suitability determination unit, a determination result transmission unit, and a session construction unit as functional units.
Among these, the terminal information receiving unit receives terminal information from the terminal device 10. The terminal information includes terminal identification information for identifying the terminal device 10 and application identification information for identifying an application operating on the terminal device 10.
The suitability determination unit determines suitability between the first application operating in the terminal device 10-1 and the second application operating in the terminal device 10-2 based on the terminal information.
The determination result transmission unit transmits the determination result by the suitability determination unit to the terminal device 10-1 in response to the determination request transmitted by the first application operating in the terminal device 10-1.
The session construction unit constructs a session between the first application that transmitted the communication start request and the second application in response to the communication start request transmitted by the first application based on the terminal information.

  Further, the signaling server 20 determines whether or not the terminal device 10 is a legitimate device at the start of session construction, that is, secure determination. Whether or not the terminal device 10 is a legitimate device can be determined by various means. In this example, the signaling server 20 makes a secure determination based on the telephone number and the IP address transmitted from the terminal device 10. More specifically, the signaling server 20 is based on whether or not the telephone number and IP address transmitted from the terminal apparatus 10 match the telephone number and IP address previously assigned to the terminal apparatus 10. Make a secure decision.

  In addition, the signaling server 20 stores registration information. The signaling server 20 uses this registration information to perform session construction and secure determination. The signaling server 20 makes a secure determination at a predetermined frequency after the session is established. The signaling server 20 establishes a session or maintains an established session only when it is determined that the terminal device 10 is a legitimate device. Specifically, when the signaling server 20 determines that the terminal device 10 is a legitimate device, the signaling server 20 issues a token to the terminal device 10. This token is permission information for the terminal devices 10 to communicate with each other via a high-speed communication line. This token has an expiration date from the time of issue. The terminal device 10 can share screen information with other terminal devices 10 when this valid token is issued. Further, the terminal device 10 cannot share screen information with other terminal devices 10 when a token has not been issued and when the validity period of the issued token has expired. In this way, the signaling server 20 manages a session between the terminal devices 10.

  The authentication server 30 stores the telephone number of the terminal device 10 and the Internet protocol address (IP address) in association with each other. When there is a request for an IP address corresponding to the telephone number from the signaling server 20, the authentication server 30 returns an IP address corresponding to the telephone number of the terminal device 10 to the signaling server 20.

  If the combination of the telephone number and the IP address for a certain terminal device 10 does not match the combination of the telephone number and the IP address stored in the authentication server 30, this terminal device 10 is not a legitimate device. there is a possibility. The signaling server 20 acquires a telephone number and an IP address from a certain terminal device 10, and compares the telephone number and the IP address stored in the authentication server to see if the combination of the telephone number and the IP address matches. Determine. As a result of this comparison, when it is determined that the combination of the telephone number and the IP address matches, the signaling server 20 issues a token to the terminal device 10. Further, as a result of this comparison, when it is determined that the combination of the telephone number and the IP address does not match, the signaling server 20 does not issue a token to the terminal device 10.

  The push server 40 stores a plurality of push IDs. The push ID is identification information for identifying an application corresponding to the information communication function that operates on the terminal device 10. The push ID stored in the push server 40 is associated with the application operating on the terminal device 10 by the signaling server 20. Specifically, when a new downloaded application is activated for the first time in the terminal device 10, the terminal device 10 requests the push server 40 to issue a push ID. The push server 40 issues a push ID for the new application of the terminal device 10 based on this request, and transmits the issued push ID to the terminal device 10. The terminal device 10 transmits information associating the new application and the issued push ID as registration information to the signaling server 20. The signaling server 20 stores the registration information transmitted from the terminal device 10. Thereby, the push ID stored in the push server 40 and the application operating on the terminal device 10 are associated by the signaling server 20.

  Further, when the application is upgraded in the terminal device 10, the push server 40 issues a new push ID along with the upgrade. That is, the push server 40 is updated with the update of the application version of the application running on the terminal device 10.

  Further, when the signaling server 20 requests the requesting terminal device 10 to start the session construction, the push server 40 uses the push technology to obtain the telephone number of the requesting terminal device 10 by the push technique. 10 is transmitted. This push technology is a technology for delivering a communication request to the terminal device 10 from the server device side (in this example, the signaling server 20 and the push server 40 side) using an IP address. In the following description, transmitting information by the push technique is also referred to as simply pushing information.

The relay server 50 is a relay server that performs information sharing communication between a plurality of terminal devices 10 when peer-to-peer connection is not possible in information sharing communication between the plurality of terminal devices 10. A peer-to-peer connection is also referred to as a P2P connection. When a P2P session of information sharing communication between a plurality of terminal devices 10 cannot be established even after a predetermined time, a session via the relay server 50 is established.
Next, a procedure until session establishment by each device described above will be briefly described.

[Initial Settings (Registration)]
Upon receiving the push ID issuance request transmitted by the application of the terminal device 10, the push server 40 transmits the push ID corresponding to the activated application to the terminal device 10 (steps S2-1 and S2-2). ).
The terminal device 10 transmits registration information to the signaling server 20 (steps S4-1 and S4-2).
The signaling server 20 stores the registration information transmitted in step S2-1 and step S2-2, and ends the registration process.

[Personal authentication]
The signaling server 20 authenticates the user of the terminal device 10 in step S6 and step S7. Specifically, the signaling server 20 transmits a telephone number to the authentication server 30 (step S6). This telephone number is a telephone number included in the registration information transmitted in step S20.
The authentication server 30 receives the telephone number transmitted in step S6, determines whether this telephone number is a telephone number registered in advance, and transmits this determination result to the signaling server 20 (step). S7).

[Session construction]
The requesting terminal device 10-1 transmits an information sharing request to the signaling server 20 (step S10). The signaling server 20 receives a request for information sharing from the terminal device 10. This request for information sharing includes the telephone number of the requested terminal device 10. Next, the signaling server 20 compares the received telephone number with registration information stored in advance, and determines whether or not the received telephone number is available.
If the signaling server 20 determines that the received telephone number is available, the signaling server 20 notifies the requesting terminal device 10 that the information communication function is available (step S13).

  Next, when receiving the notification that the information communication function is available, the requesting terminal device 10 transmits a session construction request to the signaling server 20 (step S14). This request for session construction includes the telephone number and IP address of the terminal device 10 on the transmission request side and the telephone number of the terminal device 10 on the request side.

  Next, when the signaling server 20 receives the session construction request transmitted in step S130, the signaling server 20 transmits the session construction information to the push server 40 (step S15). This session construction information includes the telephone number of the requesting terminal device 10 and the push ID. At this time, the signaling server 20 acquires the push ID by searching the registration information using the telephone number of the requested terminal device 10 as a search key. This push ID corresponds to an information sharing application that operates in the requested terminal device 10.

Upon receiving the transmitted session construction information, the push server 40 pushes an information sharing application activation request to the requested terminal device 10 based on the push ID included in the session construction information (step) S16). The information pushed by the push server 40 at this time includes the telephone number of the requesting terminal device 10.
The requested terminal device 10 displays an information sharing approval image and waits for the user's approval operation. The requested terminal device 10 transmits session construction approval information to the signaling server 20 when the user performs an operation of approval (step S19).
Upon receiving the session construction approval information transmitted in step S19, the signaling server 20 constructs a session between the requesting terminal device 10 and the requested terminal device 10 and ends the session construction processing. To do.

[Functional configuration of communication system between terminals]
Next, an example of a functional configuration of the inter-device communication system 1 will be described with reference to FIG.
FIG. 5 is a block diagram illustrating an example of a functional configuration of the inter-device communication system 1 according to the present embodiment. The signaling server 20 includes an update information supply unit 201. In addition, each part of the terminal information receiving unit, the suitability determining unit, the determination result transmitting unit, and the session constructing unit included in the signaling server 20 is not shown.
The update information supply unit 201 supplies update information for updating the determination criterion information to each terminal device 10. This determination criterion information is information indicating a criterion for determining the communication status of a session.

The control unit 160 of the terminal device 10 includes the following units as functional units.
That is, the control unit 160 of the terminal device 10-1 includes a first update information reception unit 1601, a first determination criterion information update unit 1602, a first determination criterion information storage unit 1603, an inspection information generation unit 1604, an inspection An information transmission unit 1605, a parameter reception unit 1606, a moving image data generation unit 1607, a moving image data storage unit 1608, and a moving image data transmission unit 1609 are provided.
In addition, the control unit 160 of the terminal device 10-2 includes a second update information reception unit 1621, a second determination criterion information update unit 1622, a second determination criterion information storage unit 1623, an inspection information reception unit 1625, and a communication. A situation determination unit 1626, a parameter calculation unit 1627, a parameter transmission unit 1628, a moving image data reception unit 1629, and a display control unit 1630 are provided.

The first update information receiving unit 1601 receives update information supplied by the update information supply unit 201 of the signaling server 20.
The first determination criterion information storage unit 1603 stores determination criterion information.
The first determination criterion information update unit 1602 updates the determination criterion information stored in the first determination criterion information storage unit 1603 with the update information received by the first update information reception unit 1601.
The configuration of the second update information receiving unit 1621, the second determination criterion information updating unit 1622, and the second determination criterion information storage unit 1623 includes a first update information receiving unit 1601, a first determination criterion information updating unit 1602, and Since it is the same as the 1 criterion information storage unit 1603, the description thereof is omitted.
By providing the first update information receiving unit 1601 and the first determination criterion information updating unit 1602, the inter-device communication system 1 allows the signaling server 20 to collectively update the determination criterion information of each terminal device 10. Can do.

  The inspection information generation unit 1604 generates inspection information based on the first determination criterion information stored in the first determination criterion information storage unit 1603. As an example of the inspection information, there is an inspection packet used for a known available bandwidth estimation procedure such as a packet train method. The packet size of the inspection packet, the transmission interval of the inspection packet, the number of inspection packets transmitted, and the like are determined by the first determination criterion information stored in the first determination criterion information storage unit 1603. That is, the first determination criterion information is information for determining parameters used for the available bandwidth estimation. In the following description, the available bandwidth estimated by the available bandwidth estimation is also referred to as an estimated available bandwidth. Further, the bandwidth of the available bandwidth is also referred to as the available bandwidth status or simply the available bandwidth.

The inspection information transmission unit 1605 transmits the inspection information generated by the inspection information generation unit 1604 to the terminal device 10-2.
The inspection information receiving unit 1625 receives the inspection information transmitted by the inspection information transmitting unit 1605.
The communication status determination unit 1626 determines the communication status of the session based on the second determination criterion information stored in the second determination criterion information storage unit 1623 and the inspection information received by the inspection information reception unit 1625. In this example, the communication status determination unit 1626 determines the communication status by estimating the available bandwidth of the session based on these pieces of information. Hereinafter, the determination result of the communication status by the communication status determination unit 1626 is referred to as estimated usable bandwidth information.

The parameter calculation unit 1627 calculates the encoding parameter of the moving image data based on the estimated available bandwidth information that is the determination result of the communication status determination unit 1626. The encoding parameter of the moving image data includes a parameter indicating the quality of the moving image reproduced by the moving image data. Specifically, the encoding parameters include a moving image frame rate, bit rate, resolution, quantization parameter, and the like. The parameter indicating the quality of the moving image can be changed in the high quality direction and the low quality direction. Here, changing the parameter indicating the quality of the moving image in the high quality direction includes, for example, increasing the frame rate, bit rate, or resolution of the moving image, and reducing the quantization parameter. Also, changing the parameter indicating the quality of the moving image in the low quality direction includes, for example, reducing the frame rate, bit rate, or resolution of the moving image, and increasing the quantization parameter.
In the following description, a frame rate and a bit rate of moving images are collectively referred to as an update rate. Also, a case where the update rate is increased will be described as an example of changing the parameter indicating the quality of the moving image in the high quality direction. As an example of changing the parameter indicating the quality of the moving image in the low quality direction, a case of reducing the update rate will be described.
The parameter transmission unit 1628 transmits the encoding parameter calculated by the parameter calculation unit 1627 to the terminal device 10-1.

  In this example, the terminal device 10-2, that is, the control unit 160 of the receiving-side terminal device 10 is described as including the parameter calculation unit 1627, but is not limited thereto. The parameter calculation unit 1627 may be provided in the terminal device 10-1, that is, the control unit 160 of the transmission-side terminal device 10. In this case, the communication status determination unit 1626 transmits the estimated available bandwidth information to the terminal device 10-1 instead of the encoding parameter.

The parameter receiving unit 1606 receives the encoding parameter transmitted by the parameter transmitting unit 1628.
The moving image data generating unit 1607 generates moving image data based on the encoding parameter received by the parameter receiving unit 1606. Specifically, the moving image data generation unit 1607 generates moving image data based on each piece of information such as the frame rate, bit rate, resolution, and image size of the moving image included in the encoding parameter.
The moving image data storage unit 1608 includes a buffer memory, and temporarily stores the moving image data generated by the moving image data generation unit 1607 in the order of generation.
The moving image data transmission unit 1609 reads the moving image data stored in the moving image data storage unit 1608 in the storage order, that is, the generation order of moving image data. The moving image data transmission unit 1609 transmits the read moving image data to the terminal device 10-2.

The moving image data receiving unit 1629 receives moving image data transmitted by the moving image data transmitting unit 1609.
The display control unit 1630 displays a moving image based on the moving image data received by the moving image data receiving unit 1629 on the display unit 100.

[Operation of inter-terminal communication system]
Next, the operation of the inter-device communication system 1 of the present embodiment will be described with reference to FIG.
FIG. 6 is a flowchart showing an example of the operation of the inter-device communication system 1 of the present embodiment. In this example, a case will be described in which moving image data is supplied from the terminal device 10-1 to the terminal device 10-2, and the moving image is reproduced in the terminal device 10-2.

  The terminal device 10-1 and the terminal device 10-2 construct a session (Step S100, Step S200). Next, the terminal device 10-1 transmits inspection information to the terminal device 10-2 by the constructed session. First, the inspection information generation unit 1604 of the terminal device 10-1 generates inspection information based on the first determination criterion information stored in the first determination criterion information storage unit 1603 (step S110). The inspection information transmission unit 1605 transmits the inspection information generated in step S110 to the terminal device 10-2 (step S120).

  The inspection information receiving unit 1625 of the terminal device 10-2 receives the inspection information transmitted by the inspection information transmitting unit 1605 in step S120 (step S210). The communication status determination unit 1626 reads the second determination criterion information stored in the second determination criterion information storage unit 1623 (step S220). The communication status determination unit 1626 inspects the determination criterion information based on the inspection information received in step S210 and the second determination criterion information read in step S220 (step S230). Details of the inspection of the criterion information performed by the communication status determination unit 1626 will be described with reference to FIG.

  FIG. 7 is a flowchart illustrating an example of the operation of checking the criterion information performed by the communication status determining unit 1626 of the present embodiment. Here, the inspection of the determination criterion information is an inspection of whether or not the determination criterion information stored in each terminal device 10 matches. Specifically, each terminal device 10 stores determination criterion information. The terminal device 10-1 on the transmission side generates inspection information based on the first determination criterion information stored in the own device. Further, the receiving-side terminal device 10-2 determines the communication status based on the second determination criterion information stored in the own device and the inspection information generated based on the first determination criterion information. That is, the terminal device 10-2 determines the communication status on the assumption that the first determination criterion information and the second determination criterion information match.

  Here, the signaling server 20 may update the criterion information stored in each terminal device 10. In the process of this update, there are cases where the terminal device 10 that has already been updated to the new criterion information and the terminal device 10 that has not yet been updated to the new criterion information. That is, there is a case where old and new determination criterion information is mixed in the terminal device 10-1 on the transmission side and the terminal device 10-2 on the reception side. In this case, if both of the terminal devices 10 determine the communication status using the determination criterion information of their own devices, there may be a case where the determination accuracy is deteriorated or the determination itself cannot be performed. . Therefore, the terminal device 10 inspects the criterion information prior to determining the communication status.

  The communication status determination unit 1626 acquires first determination criterion information included in the inspection information from the inspection information received in Step S210 (Step S231). The communication status determination unit 1626 acquires the second determination criterion information read from the second determination criterion information storage unit 1623 in step S220 (step S232). The communication status determination unit 1626 determines whether or not the first determination criterion information matches the second determination criterion information (step S233). Here, both the first determination criterion information and the second determination criterion information include information indicating the update status of the determination criterion information, for example, version information. If the version information matches, the communication status determination unit 1626 determines that the first determination criterion information matches the second determination criterion information (step S233; YES), and the second determination The reference information is used for determining the communication status (step S234). If the version information does not match, the communication status determination unit 1626 determines that the first determination criterion information and the second determination criterion information do not match (step S233; NO), and The 1 determination criterion information is used for determining the communication status (step S235).

  Returning to FIG. 6, the communication status determination unit 1626 determines the communication status based on the test information received by the test information reception unit 1625 and the criterion information selected in step S230 (step S240). Various known methods can be applied to the determination of the communication status. For example, when the inspection information is a packet train inspection packet, the communication status determination unit 1626 performs communication using a known method such as an inspection packet reception interval or an available bandwidth estimation based on the number of unreceived packets. Determine the situation.

  Next, the parameter calculation unit 1627 calculates an encoding parameter based on the communication status determined by the communication status determination unit 1626 (step S250). The parameter transmission unit 1628 transmits the encoding parameter calculated by the parameter calculation unit 1627 in step S250 to the terminal device 10-1 (step S260).

  The parameter receiving unit 1606 receives the encoding parameter transmitted by the parameter transmitting unit 1628 (step S130). The moving image data generation unit 1607 generates moving image data based on the encoding parameter received by the parameter receiving unit 1606 (step S140). The moving image data transmission unit 1609 transmits the moving image data generated by the moving image data generation unit 1607 to the terminal device 10-2 (step S150).

  The moving image data receiving unit 1629 receives the moving image data transmitted by the moving image data transmitting unit 1609 (step S270). The display control unit 1630 displays a moving image based on the moving image data received by the moving image data receiving unit 1629 on the display unit 100 (step S280).

The terminal device 10-1 and the terminal device 10-2 repeatedly execute steps S140 to S280 until the session is ended (steps S160 and S290). Thereby, the communication system 1 between apparatuses displays the moving image based on the moving image data which the terminal device 10-1 produced | generated on the display part 100 of the terminal device 10-2.
Note that the terminal device 10-1 and the terminal device 10-2 may repeatedly execute steps S110 to S280 until the session is terminated. Here, in step S110 to step S130 and in step S210 to step S260, the terminal device 10-1 and the terminal device 10-2 perform estimation of the available bandwidth of the session. That is, the terminal device 10-1 and the terminal device 10-2 perform estimation of the usable bandwidth while transmitting / receiving moving image data until the session is terminated.

Next, with reference to FIG. 8, the operation of updating the criterion information will be described.
FIG. 8 is a flowchart showing an example of the operation of updating the criterion information by the inter-device communication system 1 of the present embodiment. In the present embodiment, the determination criterion information is updated after the above-described session ends. The signaling server 20 transmits update information to each terminal device 10 for which the session has ended. The first update information receiving unit 1601 of the terminal device 10-1 receives the update information transmitted by the signaling server 20 (step S1010). The first determination criterion information update unit 1602 stores the update information received by the first update information reception unit 1601 in the first determination criterion information storage unit 1603 by writing it into the first determination criterion information storage unit 1603. The criterion information is updated (step 1020).
Also in the terminal device 10-2, the determination stored in the second determination criterion information storage unit 1623 by the second update information receiving unit 1621 and the second determination criterion information updating unit 1622 in the same manner as the terminal device 10-1. The reference information is updated (Step 2010, Step 2020).

  As described above, the inter-device communication system 1 according to the present embodiment determines the communication status at the start timing of the moving image data transmission / reception session, and therefore can supply moving image data according to the communication status. The inter-device communication system 1 determines the communication status while transmitting / receiving moving image data after the start of the transmission / reception session. Further, the inter-device communication system 1 can collectively update the determination criterion information used for determining the communication status from the signaling server 20 to each terminal device 10. Thereby, even after the terminal device 10 is sold and reaches the user's hand, the criterion information can be updated. Here, transmission / reception of data having a relatively large amount of information such as moving image data is relatively easily affected by the quality of the communication environment. In addition, since the communication environment changes day by day due to various factors, it is difficult to uniquely determine determination criterion information for determining the quality of the communication environment. Since the signaling server 20 constructs a session of a communication environment for transmitting and receiving moving image data, it is easy to obtain statistical information of the communication environment. Based on the statistical information obtained by the signaling server 20, the signaling server 20 performs update control of the determination criterion information, whereby the determination criterion information can be updated according to a change in the communication environment. That is, according to the inter-device communication system 1, it is possible to reproduce a moving image according to the communication environment.

[Second Embodiment]
Hereinafter, a second embodiment of the inter-device communication system 1 according to the present embodiment will be described with reference to FIGS. 9 to 11.
FIG. 9 is a block diagram illustrating an example of a functional configuration of the inter-device communication system 1 according to the second embodiment. The inter-device communication system 1 of the present embodiment differs from the first embodiment described above in that the terminal device 10-2 can change the encoding parameter according to the reception status of moving image data. In addition, about the structure and operation | movement same as 1st Embodiment mentioned above, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

The encoding parameter is calculated by the parameter calculation unit 1627 based on the communication state of the moving image data received by the moving image data reception unit 1629 and the communication state of the session determined by the communication state determination unit 1626.
The terminal device 10-2 includes a parameter changing unit 1631. The parameter changing unit 1631 changes the encoding parameter calculated by the parameter calculating unit 1627. That is, the parameter changing unit 1631 updates the generation parameter of the moving image data transmitted / received by the session from the start to the end of the session. That is, the parameter changing unit 1631 dynamically updates the encoding parameter during transmission / reception of moving image data.

  The parameter changing unit 1631 changes the encoding parameter based on the communication state of the moving image data by the moving image data transmitting unit 1609 and the moving image data receiving unit 1629. Specifically, the parameter changing unit 1631 monitors the arrival status of moving image data packets. The packet arrival status includes situations such as the degree of packet delay, the number of lost packets, and the packet loss rate. In the following description, an example will be described in which the parameter changing unit 1631 changes the encoding parameter based on the number of lost moving image data packets.

  FIG. 10 is a flowchart showing an example of the operation of the inter-device communication system 1 in the present embodiment. In FIG. 10, illustration of each step from step S100 to step S130 and from step S200 to step S260 in the first embodiment described above is omitted.

The parameter changing unit 1631 monitors the packet arrival status (step S2210). In this example, the parameter changing unit 1631 monitors the packet arrival status by counting the number of lost packets per unit time of moving image data after the start of the session.
Next, the parameter changing unit 1631 changes the encoding parameter based on the number of lost packets counted in step S2210 (step S2220). In this example, a case where the parameter changing unit 1631 changes the update rate of moving image data among parameters indicating the quality of a moving image included in the encoding parameter will be described, but other parameters may be changed. The encoding parameter changing operation by the parameter changing unit 1631 will be described with reference to FIG.

  FIG. 11 is a flowchart illustrating an example of a coding parameter changing operation by the parameter changing unit 1631 of the present embodiment. In this example, at the start of the session, the communication status determination unit 1626 estimates the available bandwidth of the session in advance. Information indicating the available bandwidth estimated by the communication status determination unit 1626 is also referred to as estimated available bandwidth information.

The parameter changing unit 1631 acquires estimated usable bandwidth information (step S2221). Next, the parameter changing unit 1631 acquires the number of lost packets counted in step 2210 (step S2222).
If the parameter changing unit 1631 determines that the number of lost packets is equal to or less than the first threshold th1 (step S2223; YES), the process proceeds to step S2224. Here, the first threshold value may be set such that the number of lost packets is 0 (zero). In this case, the parameter changing unit 1631 advances the process to step S2224 when the number of lost packets is 0 (zero).
Next, the parameter changing unit 1631 determines whether or not the estimated available bandwidth exceeds the second threshold th2 (step S2224). If the parameter changing unit 1631 determines that the estimated available bandwidth exceeds the second threshold th2 (step S2224; YES), the parameter changing unit 1631 increases the update rate.

  That is, the parameter changing unit 1631 determines that there is a margin in the communication state when the number of lost packets is relatively small and the estimated available bandwidth is relatively large, and increases the update rate of the moving image data. That is, the parameter changing unit 1631 can transmit / receive higher-quality moving image data by increasing the update rate of moving image data when there is a margin in the communication state.

  In addition, the second threshold th2 of the estimated usable bandwidth may be associated with an estimated video MOS (Mean Opinion Score) estimated from the stream. Here, the estimated video MOS is automatically estimated from the stream without a human judgment, for example, by a video MOS estimation unit (not shown). The estimated video MOS is indicated by, for example, five levels: 5: Excellent, 4: Good, 3: Fair, 2: Poor, 1: Bad. It is. The second threshold th2 is set so that the estimated video MOS can maintain 3 or more, for example, when the update rate is increased. As described above, since the second threshold th2 is set in association with the estimated video MOS, even when the parameter changing unit 1631 increases the update rate of the moving image data, the quality of the moving image exceeds a predetermined value. Can be maintained in quality.

  In Step S2223, if parameter changing unit 1631 determines that the number of lost packets exceeds first threshold th1 (Step S2223; NO), it advances the process to Step S2226.

If the parameter changing unit 1631 determines that the number of lost packets is equal to or less than the fourth threshold th4 (step S2226; YES), the parameter changing unit 1631 proceeds to step S2227 and maintains the update rate.
If the parameter changing unit 1631 determines that the number of lost packets is equal to or less than the third threshold th3 (step S2228; YES), the parameter changing unit 1631 proceeds to step S2229 to reduce the update rate.

  Here, the third threshold th3 is set larger than the fourth threshold th4. That is, the parameter changing unit 1631 maintains the update rate when it is determined that the number of lost packets is relatively small, and reduces the update rate when it is determined that the number of lost packets is relatively large. Thus, the parameter changing unit 1631 can prevent the update rate from changing frequently by maintaining the update rate when the number of lost packets is relatively small.

  In addition, when the update rate of moving image data is high (large), the session may become congested, and the moving image may not be reproduced normally. When the number of lost packets is relatively large, the parameter changing unit 1631 can suppress the session from becoming congested by reducing the update rate. That is, the parameter changing unit 1631 can prevent the moving image from being normally reproduced by reducing the update rate.

Note that if the parameter changing unit 1631 determines that the number of lost packets exceeds the third threshold th3 (step S2228; NO), the process proceeds to step S2230. In step S2230, the parameter changing unit 1631 outputs an instruction to the display control unit 1630 to stop moving image display and switch to still image display.
When the number of lost packets exceeds the third threshold th3, even if the update rate of the moving image data is reduced, the moving image may not be reproduced normally. The parameter changing unit 1631 displays a still image instead of the moving image when it is highly possible that the moving image cannot be normally reproduced or when it is detected that the moving image cannot be normally reproduced. Here, the fact that a moving image cannot be reproduced normally includes a case where image information is lost in the decoding process of the moving image. As described above, when information is lost in the process of decoding a moving image, the reproduced image is corrupted. That is, the fact that the moving image cannot be normally reproduced includes a case where the image to be reproduced is corrupted. In addition, the still image instead of the moving image here is a still image decoded before the moving image cannot be normally reproduced. That is, the still image referred to here is a still image in which the image is not corrupted because it has been normally decoded. In this way, the parameter changing unit 1631 can maintain the quality of the display image by displaying a still image that does not collapse when the moving image is corrupted. That is, according to the inter-device communication system 1, it is possible to reproduce a moving image according to the communication environment.

[Third Embodiment]
Hereinafter, a third embodiment of the inter-device communication system 1 according to the present embodiment will be described with reference to FIGS.
FIG. 12 is a block diagram illustrating an example of a functional configuration of the inter-device communication system 1 according to the third embodiment. The inter-device communication system 1 according to the present embodiment is different from the above-described embodiments in that the terminal device 10-1 can control generation and transmission of moving image data according to the transmission state of moving image data. And different. In addition, about the same structure and operation | movement as each embodiment mentioned above, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

The terminal device 10-1 includes an untransmitted data detection unit 1610 and a moving image data control unit 1611.
The untransmitted data detection unit 1610 detects untransmitted data that has not been transmitted by the moving image data transmission unit 1609 among the moving image data stored in the moving image data storage unit 1608.
The moving image data control unit 1611 controls moving image data transmitted by the moving image data transmission unit 1609 based on the quantity of untransmitted data detected by the untransmitted data detection unit 1610. Here, the quantity of untransmitted data is the number or amount of untransmitted data stored in the moving image data storage unit 1608. As an example, the quantity of untransmitted data includes the number of frames of untransmitted moving image data and the data amount of untransmitted moving image data.
An example of the moving image data stored in the moving image data storage unit 1608 will be described with reference to FIG.

FIG. 13 is a diagram illustrating an example of moving image data stored in the moving image data storage unit 1608 of the present embodiment. The moving image data generation unit 1607 sequentially generates moving image data in time series. This moving image data includes an I frame and a P frame. This I frame is a frame generated without using inter-frame prediction. A P frame is a frame generated using inter-frame prediction. In this example, it is assumed that only the I frame and the P frame are included, but the frame generated by the moving image data generation unit 1607 may include a frame of another format. For example, the frame generated by the moving image data generation unit 1607 may include a B frame generated by selecting any one of forward prediction, backward prediction, or bidirectional prediction.
Here, since the basic information for decoding the moving image is included in the I frame, it is possible to decode the I frame into the moving image using only the frame. On the other hand, since the P frame includes only difference information from other frames, it can be decoded into a moving image by combining with I frames and other P frames.

In the moving image data storage unit 1608, the frames generated by the moving image data generation unit 1607 are stored in the order of frame generation from the frame 01, which is a P frame, the frame 10, which is an I frame, from the old frame to the new frame. ing.
The moving image data transmission unit 1609 reads out the moving image data stored in the moving image data storage unit 1608 in order from the old frame and transmits it to the terminal device 10-2. The moving image data transmission unit 1609 updates the read frame to “read”. The moving image data generation unit 1607 writes the newly generated frame at the position where the “read” frame is stored. That is, the moving image data storage unit 1608 is a buffer memory for reading and transmitting moving image data generated by the moving image data generation unit 1607 in the order of generation.

With reference to FIG. 14, an example of operation | movement of the communication system 1 between apparatuses in this embodiment is demonstrated.
FIG. 14 is a flowchart showing an example of the operation of the inter-device communication system 1 in the present embodiment. In FIG. 14, the steps S100 to S130, the steps S200 to S260, the steps S160 and S290 in the first embodiment described above are not shown.

The moving image data generation unit 1607 writes the generated frames of moving image data in the moving image data storage unit 1608 in the order of generation (step S1310). The moving image data transmission unit 1609 reads moving image data frames from the moving image data storage unit 1608 in the order of generation (step S1320), and transmits the frames to the terminal device 10-2 in the order of reading (step S150).
The moving image data receiving unit 1629 of the terminal device 10-2 receives moving image data (step S270). The display control unit 1630 displays a moving image based on the moving image data on the display unit 100 (step S280).

  The untransmitted data detection unit 1610 detects untransmitted moving image data among the frames of moving image data stored in the moving image data storage unit 1608 (step S1330). More specifically, the untransmitted data detection unit 1610 counts the number of frames of untransmitted moving image data.

  The moving image data control unit 1611 controls the moving image data transmitted by the moving image data transmission unit 1609 based on the amount of untransmitted moving image data detected by the untransmitted data detection unit 1610 (step S1340). An example of control of moving image data by the moving image data control unit 1611 will be described with reference to FIG.

  FIG. 15 is a flowchart illustrating an example of an operation of controlling moving image data by the moving image data control unit 1611 of the present embodiment. The moving image data control unit 1611 controls moving image data based on the amount of untransmitted moving image data detected by the untransmitted data detection unit 1610 and a predetermined threshold value. Specifically, the moving image data control unit 1611 determines that the number of untransmitted frames counted by the untransmitted data detection unit 1610 is equal to or less than the fifth threshold th5 (step S1341; YES). The update rate of the moving image data is maintained (step S1342). If the moving image data control unit 1611 determines that the number of untransmitted frames counted by the untransmitted data detection unit 1610 exceeds the fifth threshold th5 (step S1341; NO), the process proceeds to step S1343. Proceed to

  When the moving image data control unit 1611 determines that the number of untransmitted frames counted by the untransmitted data detection unit 1610 is equal to or less than the sixth threshold th6 (step S1343; YES), the moving image data update is performed. The rate is reduced (step S1344). Here, the sixth threshold th6 is set to a value larger than the fifth threshold th5. When the amount of untransmitted data is relatively small, the moving image data control unit 1611 prevents the update rate from changing frequently by maintaining the update rate. Also, the moving image data control unit 1611 suppresses the overflow of moving image data in the moving image data storage unit 1608, that is, the transmission buffer memory by reducing the update rate when the amount of untransmitted data is relatively large. To do. Thereby, the moving image data control part 1611 can suppress that the quality of the moving image reproduced | regenerated in the terminal device 10-2 falls.

  Also, when the moving image data control unit 1611 determines that the number of untransmitted frames counted by the untransmitted data detection unit 1610 exceeds the sixth threshold th6 (step S1343; NO), the moving image data storage is performed. Part or all of the moving image data stored in the unit 1608 is erased. That is, the moving image data control unit 1611 clears the transmission buffer memory (step S1345). In other words, the moving image data control unit 1611 has a large amount of untransmitted data or a moving image in the transmission buffer memory to such an extent that overflow of moving image data in the transmission buffer memory cannot be suppressed only by reducing the update rate. When the delay amount increases due to the accumulation of data, the transmission in the generation order of the moving image data is given up. In other words, the moving image data control unit 1611 clears the transmission buffer memory when the amount of untransmitted moving image data is large and deterioration of the quality of moving images cannot be avoided. Here, after a certain amount of time has passed, the communication status may change, and the available bandwidth may be restored to the extent that moving image data can be transmitted and received. Since the moving image data control unit 1611 clears the transmission buffer memory, it is possible to prevent untransmitted moving image data that causes a decrease in moving image quality from being transmitted when the available bandwidth is restored. That is, according to the inter-device communication system 1, it is possible to reproduce a moving image according to the communication environment.

Further, when clearing the transmission buffer memory, the moving image data control unit 1611 can erase a part of the moving image data stored in the moving image data storage unit 1608 and not erase the other part. Specifically, as shown in FIG. 13, the moving image data control unit 1611 is a frame after frame 20 that is the latest I frame among the frames of moving image data stored in the moving image data storage unit 1608. Do not erase. In this case, the moving image data control unit 1611 includes the P frame generated before the frame 20, which is the latest I frame among the frames of the moving image data stored in the moving image data storage unit 1608, and Erase the I frame.
As described above, since the I frame includes basic information for decoding a moving image, the I frame can be decoded into a moving image using only the frame. By not erasing the latest I frame, the moving image data control unit 1611 can reproduce a moving image more quickly than when erasing all moving image data.

  If the moving image data control unit 1611 determines that the sixth threshold value th6 is exceeded, the newly generated moving image data is stored in the moving image data storage unit 1608 instead of clearing the transmission buffer memory. It can also be discarded so that it does not occur. With this configuration, it is possible to prevent the quality of the moving image from being deteriorated due to the change of the storage order of moving image data due to clearing of the transmission buffer memory.

  As mentioned above, although embodiment of this invention and its deformation | transformation were demonstrated, these embodiment and its deformation | transformation were shown as an example and are not intending limiting the range of invention. These embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and equivalents thereof.

  Each of the above devices has a computer inside. The process of each device described above is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer reading and executing the program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Communication system between apparatuses, 10 ... Terminal device, 20 ... Signaling server, 30 ... Authentication server, 40 ... Push server, 50 ... Relay server

Claims (15)

A communication status determination unit that determines a communication status of a session between the information transmission device and the information reception device based on determination criterion information of the communication status of the session;
A parameter calculation unit that calculates an encoding parameter of moving image data communicated by the session based on the communication status of the session determined by the communication status determination unit;
A moving image data generating unit that generates moving image data based on the encoding parameter calculated by the parameter calculating unit;
A moving image data transmission unit that transmits the moving image data generated by the moving image data generation unit from the information transmission device to the information reception device by the session;
A moving image data receiving unit for receiving the moving image data transmitted by the moving image data transmitting unit;
A display control unit for displaying a moving image based on the moving image data received by the moving image data receiving unit;
The encoding parameter calculated by the parameter calculation unit is changed based on the communication status of the video data received by the video image reception unit and the communication status of the session determined by the communication status determination unit. A parameter changing section;
An inter-device communication system. The communication status of the video data received by the video data receiving unit includes the status of packet arrival of the video data,
The communication status of the session includes the status of available bandwidth of the session communication,
The parameter changing unit
The inter-device communication system according to claim 1, wherein the encoding parameter is changed based on a packet arrival status of the moving image data and a status of the available bandwidth. The available bandwidth status of the session communication includes the estimated available bandwidth status estimated based on the session communication,
The parameter changing unit
The inter-device communication system according to claim 2, wherein the coding parameter is changed based on a packet arrival status of the moving image data and a status of the estimated available bandwidth. The packet arrival status includes the number of lost packets of the moving image data,
The encoding parameter includes a parameter indicating the quality of a moving image reproduced by the moving image data,
The parameter changing unit
When the number of lost packets of the moving image data is equal to or less than a predetermined first threshold and the available bandwidth of the session communication exceeds a predetermined second threshold, a parameter indicating the quality of the moving image is increased. The inter-device communication system according to claim 2 or 3, wherein the communication direction is changed in a quality direction. When the number of lost packets of the moving image data is equal to or smaller than a third threshold value that is greater than the first threshold value,
The parameter changing unit
By changing the parameter indicating the quality of the moving image in the low quality direction, the encoding parameter is changed,
The moving image data generation unit
Based on the encoding parameter changed by the parameter changing unit, the moving image data with reduced quality of the reproduced video is generated,
The display control unit
Display the video based on the moving image data, the quality of the video to be played is reduced,
When the number of lost packets of the moving image data is larger than the third threshold value,
The display control unit
The inter-device communication system according to claim 4, wherein the moving image data receiving unit displays a still image based on the received moving image data instead of the moving image. A test information generating unit that generates test information used for determining the communication status of the session based on the first determination criterion information of the communication status of the session between the own device and the information receiving device;
An inspection information transmission unit that transmits the inspection information generated by the inspection information generation unit to the information receiving device;
Using the encoding parameter calculated by the parameter calculation unit based on the communication status of the session determined by the information receiving device based on the transmitted inspection information and second determination criterion information on the communication status of the session A moving image data generating unit for generating moving image data;
A moving image data transmitting unit that transmits the moving image data generated by the moving image data generating unit from the own device to the information receiving device through the session;
With
The moving image data generation unit
When the encoding parameter is changed according to the communication status of the moving image data transmitted by the session, the moving image data transmitted by the session is generated based on the changed encoding parameter. Information transmitting device. The communication status of the moving image data includes the arrival status of the moving image data packet in the information receiving device,
The communication status of the session includes the status of available bandwidth of the session communication,
The moving image data generation unit
The parameter changing unit that changes the encoding parameter calculated by the parameter calculating unit uses the encoding parameter changed based on the status of packet arrival of the moving image data and the status of the available bandwidth. The information transmitting device according to claim 6, wherein moving image data is generated. The available bandwidth status of the session communication includes the estimated available bandwidth status estimated based on the session communication,
The moving image data generation unit
The said parameter change part produces | generates the said moving image data using the said encoding parameter changed based on the condition of the packet arrival of the said moving image data, and the condition of the said estimation available bandwidth. Information transmission device. The packet arrival status includes the number of lost packets of the moving image data,
The encoding parameter includes a parameter indicating the quality of a moving image reproduced by the moving image data,
The moving image data generation unit
The quality of the moving image when the parameter changing unit has a number of lost packets of the moving image data equal to or less than a predetermined first threshold value and an available bandwidth of the session communication exceeds a predetermined second threshold value. The information transmitting apparatus according to claim 7 or 8, wherein the moving image data is generated using the encoding parameter obtained by changing a parameter indicating the high quality direction. The moving image data generation unit
The parameter changing unit changes a parameter indicating the quality of the moving image in a lower quality direction when the number of lost packets of the moving image data is equal to or less than a third threshold value that is larger than the first threshold value. The information transmitting apparatus according to claim 9, wherein the moving image data in which the quality of a moving image to be reproduced is lowered is generated using the changed encoding parameter. A test information receiving unit that receives test information generated by the information transmitting device based on first determination criterion information of a communication status of a session between the own device and the information transmitting device;
A communication status determination unit that determines the communication status of the session based on the test information received by the test information reception unit and second determination criterion information of the communication status of the session;
Moving image data generated from the information transmitting device using the encoding parameter calculated by the parameter calculating unit based on the communication status of the session determined by the communication status determining unit, from the information transmitting device A receiver,
A display control unit for displaying a moving image based on the moving image data received by the moving image data receiving unit;
The encoding parameter calculated by the parameter calculation unit is changed based on the communication status of the video data received by the video image reception unit and the communication status of the session determined by the communication status determination unit. A parameter changing section;
With
The moving image data receiving unit
The moving image data generated by the information transmitting device using the encoding parameter changed by the parameter changing unit is received from the information transmitting device,
The display control unit
An information receiving apparatus that displays a moving image based on the moving image data generated using the encoding parameter changed by the parameter changing unit. The communication status of the video data received by the video data receiving unit includes the status of packet arrival of the video data,
The communication status of the session includes the status of available bandwidth of the session communication,
The parameter changing unit
The information receiving apparatus according to claim 11, wherein the encoding parameter is changed based on a packet arrival status of the moving image data and a status of the available bandwidth. The available bandwidth status of the session communication includes the estimated available bandwidth status estimated based on the session communication,
The parameter changing unit
The information receiving apparatus according to claim 12, wherein the encoding parameter is changed based on a state of arrival of the moving image data packet and a state of the estimated available bandwidth. The packet arrival status includes the number of lost packets of the moving image data,
The encoding parameter includes a parameter indicating the quality of a moving image reproduced by the moving image data,
The parameter changing unit
When the number of lost packets of the moving image data is equal to or less than a predetermined first threshold and the available bandwidth of the session communication exceeds a predetermined second threshold, a parameter indicating the quality of the moving image is increased. The information receiving apparatus according to claim 12 or 13, wherein the information receiving apparatus is changed to a quality direction. When the number of lost packets of the moving image data is equal to or smaller than a third threshold value that is greater than the first threshold value,
The parameter changing unit
By changing the parameter indicating the quality of the moving image in the low quality direction, the encoding parameter is changed,
The moving image data receiving unit
Receiving the moving image data generated by the moving image data generating unit based on the encoding parameter changed by the parameter changing unit and having a reduced quality of the reproduced moving image;
The display control unit
Display the video based on the moving image data, the quality of the video to be played is reduced,
When the number of lost packets of the moving image data is larger than the third threshold value,
The display control unit
The information receiving apparatus according to claim 14, wherein the moving image data receiving unit displays a still image based on the received moving image data instead of the moving image.

Images