JP2016072874A - End-to-end communication system, end-to-end communication method, and terminal device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an end-to-end communication system capable of enhancing operability when sharing information between terminal devices.SOLUTION: An end-to-end communication system includes a server device that transmits an information sharing request to a second information sharing function application in response to an information sharing request transmitted by a first information sharing function application while a voice communication session is established between a first terminal device and a second terminal device, and constructs an information sharing session in response to an approval response transmitted by the second information sharing function application. The second information sharing function application has a display unit display an approval information to have a user select whether the user approves the information sharing request on the basis of the information sharing request transmitted by the server device, and switches from a small-voice communication mode to drive a first speaker for small voice to a loud-voice communication mode to drive a second speaker for loud voice in response to an operation unit accepting operation to approve the information sharing request.SELECTED DRAWING: Figure 9

Description

  Embodiments described herein relate generally to an inter-terminal communication system, an inter-terminal communication method, and a terminal device.

  Conventionally, data communication by which voice data and image data can be mutually processed in real time between callers using these portable terminal devices by transmitting and receiving voice data and image data to and from each other. A system is known (see, for example, Patent Document 1).

JP 2000-261574 A

  In the data communication system as described above, there is a case where information such as an image is desired to be displayed on both display screens between portable terminal devices during communication of real-time data such as a call between portable terminal devices. In this case, the operability of the conventional portable terminal device may not be sufficient.

  It is an object of the present invention to provide an inter-terminal communication system, an inter-terminal communication method, and a terminal device that can improve operability when information is shared between terminal devices.

  (1) In order to solve the above-described problem, an inter-terminal communication system according to an aspect of the present invention includes a first terminal device that executes a first information sharing function application, and a second terminal that executes a second information sharing function application. In response to an information sharing request transmitted from the first information sharing function application while a voice call session is established between the two terminal devices, the first terminal device, and the second terminal device. A server device that transmits an information sharing request to the two information sharing function applications and constructs an information sharing session in response to the acceptance response transmitted from the second information sharing function application, and the second information sharing function The application displays a consent image for selecting whether to accept the information sharing request based on the information sharing request transmitted from the server device. The loudspeaker communication that drives the second loudspeaker from the earpiece communication mode that drives the first speaker for earpiece in response to the operation unit accepting the operation for accepting the information sharing request. Switch to mode.

  (2) One aspect of the present invention is the inter-terminal communication system according to (1) described above, wherein the second information sharing function application receives the information sharing request according to the first speaker or the A configuration in which sound is generated in the second speaker or vibration is generated in the vibration unit may be used.

  (3) One aspect of the present invention is the inter-terminal communication system according to (2), in which the second information sharing function application generates sound or vibrates in the first speaker or the second speaker. A configuration may be used in which the consent image is displayed in response to the proximity sensor detecting that the object is separated from the first speaker after generating vibration in the part.

  (4) According to one aspect of the present invention, in the inter-terminal communication system according to any one of (1) to (3), the second information sharing function application performs an operation of accepting an information sharing request. After the predetermined time received by the operation unit, the loudspeaker communication mode is switched, and during the predetermined time, a volume adjustment image for adjusting the volume of the second speaker is displayed on the display unit, and the volume adjustment image is received. A configuration may be used in which the volume of the second speaker in the loudspeaker communication mode is adjusted based on the operation performed.

  (5) According to one aspect of the present invention, in the inter-terminal communication system according to (4), when the second information sharing application includes audio data in the information shared by the information sharing session, The volume adjustment image includes an image for adjusting the volume of the voice communicated by the voice call session and an image for adjusting the volume of the voice communicated by the information sharing session, displayed on the display unit, and the volume adjustment image The volume of voice communicated by the voice call session and the volume of voice communicated by the information sharing session are adjusted based on the received operation, and voice data is included in the information shared by the information sharing session If not, the volume adjustment image should include an image for adjusting the volume of the voice communicated by the voice call session. Wherein is displayed on the display unit, to adjust the volume of sound to be communicated by the voice call session based on the operation received to the volume control picture, a structure may be employed.

  (6) One aspect of the present invention is the inter-terminal communication system according to any one of (1) to (5) described above, wherein the second information sharing function application is switched to the voice communication mode. A configuration may be used in which the proximity sensor is switched from the loudspeaker communication mode to the earpiece communication mode in response to the proximity sensor detecting that the object is in proximity to the first speaker.

  (7) One aspect of the present invention is the inter-terminal communication system according to any one of (1) to (6) described above, in which the second information sharing function application A configuration may be used in which an icon indicating that the second speaker is operating or an icon indicating that the second speaker of the first terminal device is operating is displayed on the display unit.

  In order to solve the above-described problem, in the inter-terminal communication method according to an aspect of the present invention, the server device executes the first information sharing function application and the second information sharing function application. An information sharing request is transmitted to the second information sharing function application in response to the information sharing request transmitted from the first information sharing function application while a voice call session is established with the second terminal device The server device constructs an information sharing session in response to the acceptance response transmitted from the second information sharing function application, and the second information sharing function application transmits the information transmitted from the server device. An acceptance image for selecting whether to accept an information sharing request based on a sharing request is displayed on the display unit, and the second information sharing function Amplification communication that drives the second speaker for loudspeaking from the earpiece communication mode that drives the first speaker for earpiece in response to an operation that the application accepts an information sharing request accepted by the operation unit Switch to mode.

  In order to solve the above problems, a terminal device according to an aspect of the present invention is received by a display unit that displays information, an operation unit that receives an operation, a communication unit that transmits and receives data, and the communication unit. Executes an information sharing function application, a first speaker for ear pads that outputs sound based on sound data, a second speaker for sound output that outputs sound based on sound data received by the communication unit An information sharing request based on an information sharing request transmitted from the other terminal device and received by the communication unit while a voice call session is established between the application executing unit and the other terminal device. An approval image for selecting whether to accept is displayed on the display unit, and in response to an operation for accepting an information sharing request being accepted by the operation unit, the first scan is performed. To switch from the earpiece communication mode to drive the over months to loudspeaker communication mode for driving the second speaker has a control unit.

  ADVANTAGE OF THE INVENTION According to this invention, the operativity at the time of sharing information between terminal devices can be improved.

It is a schematic diagram which shows an example of a structure of the terminal device of this embodiment. It is a schematic diagram which shows an example of the share menu button displayed on the display part of the terminal device of this embodiment. It is a schematic diagram which shows an example of the outline | summary of the information sharing function which the communication system between terminals of this embodiment provides. It is a schematic diagram which shows an example of selection operation of the information sharing function which the communication system between terminals of this embodiment provides. It is a schematic diagram which shows an example of a structure of the communication system between terminals of this embodiment. It is a table | surface which shows an example of the registration information which the signaling server of this embodiment memorize | stores. It is a flowchart which shows an example of the operation | movement procedure of the registration of this embodiment. It is a flowchart which shows an example of the operation | movement procedure of session construction of this embodiment. It is a flowchart which shows the share consent process in the operation | movement procedure of session construction of this embodiment. It is a flowchart which shows another example of the share consent process in the operation | movement procedure of session construction of this embodiment. It is a flowchart which shows another example of the share consent process in the operation | movement procedure of session construction of this embodiment. It is a figure which shows the image for volume adjustment displayed on the request | requirement side terminal device 10 in this embodiment. It is a figure which shows the other image for volume adjustments displayed on the request | requirement side terminal device 10 in this embodiment. It is a figure which shows the image which made the speaker icon P40 superimpose-display on the image P27 displayed on the terminal device 10 of the request side in this embodiment.

[Embodiment]
Hereinafter, an embodiment of the inter-terminal communication system 1 according to the present embodiment will be described with reference to the drawings.
The inter-terminal communication system 1 of the present embodiment provides an information sharing function among a plurality of terminal devices 10. The terminal device 10 here is a communication device that can communicate with each other by voice or text, and is, for example, a mobile phone or a smartphone. The terminal device 10 can be connected to a high-speed data communication line capable of transmitting and receiving voice information and image information through the same line. The high-speed data communication line is, for example, an LTE (Long Term Evolution) communication network. This high-speed data communication line provides a service (VoLTE; Voice over LTE) that enables voice calls in addition to transmission and reception of digital information such as images and characters.

  The information sharing function is a function for sharing image information or audio information among a plurality of terminal devices 10. More specifically, a certain terminal device 10 transmits image information or audio information to another terminal device 10. Accordingly, the inter-terminal communication system 1 displays or reproduces the image displayed on the transmission-side terminal device 10 or the sound being reproduced on the reception-side terminal device 10. The inter-terminal communication system 1 provides not only a one-way information sharing function from one terminal apparatus 10 to another terminal apparatus 10 but also a two-way information sharing function between the terminal apparatuses 10.

  The inter-terminal communication system 1 includes a plurality of terminal devices 10. In the following description, when distinguishing each of the plurality of terminal devices 10, for example, they are described as a terminal device 10-1 and a terminal device 10-2. Moreover, in the following description, when not distinguishing each of these several terminal devices 10, it describes as the terminal device 10 only. The configuration of the terminal device 10 will be described with reference to FIG.

[Configuration of terminal device]
FIG. 1 is a schematic diagram illustrating an example of a configuration of the terminal device 10 according to the present embodiment. The terminal device 10 includes a display unit 100, an operation detection unit 110, an imaging unit 120, a sound reproduction unit 130, a sound collection unit 140, a vibration unit 145, a communication unit 150, and a control unit 160. Yes.

  The display unit 100 includes a liquid crystal display and displays images such as characters and photographs. The image displayed by the display unit 100 is for a user to select a standby image displayed when waiting for an incoming call, an application image provided by an application operating on the terminal device, and various functions provided by the terminal device. Menu images and so on.

  This application image includes a dialer image provided by a telephone (voice communication) application, and an address book image provided by an address book application that manages names, telephone numbers, and mail addresses. In addition, the application image includes a moving image provided by the moving image reproduction application, a web image provided by the web browser application, a map image provided by the map application, and a photo image provided by the photo album application. The application image includes various images such as a mail text image provided by the e-mail application and a chat image provided by the text chat application.

  Of the menu images displayed on the display unit 100, the menu image for selecting the information sharing function is particularly referred to as an information sharing menu image. The information sharing menu image is also referred to as a shared menu ball image or a shared menu button image. Here, the menu includes various menus in addition to the information sharing menu. For example, the menu includes a voice recognition menu. The voice recognition menu is a menu for selecting a voice recognition function. The voice recognition function is a function in which the terminal device 10 recognizes a voice and extracts a keyword corresponding to the recognized voice. The terminal device 10 displays a web page related to the keyword by outputting the extracted keyword to a search engine or the like. Further, the terminal device 10 displays a search result such as store information obtained by outputting the extracted keyword to a search engine or the like. The menu shown here is an example and is not limited to this. The functions selected by the menu include all functions that can be used during a voice call. In the following description, the shared menu ball image is also simply referred to as “shared ball”, and the shared menu button image is also simply referred to as “shared button”. An example of this shared menu button is shown in FIG.

  FIG. 2 is a schematic diagram illustrating an example of a shared menu button displayed on the display unit 100 of the terminal device 10. In this example, the shared menu button image PMB is displayed at an arbitrary position on the display unit 100 so as to overlap the application image PAP.

  Returning to FIG. 1, the operation detection unit 110 includes a transparent touch panel sensor that covers the display unit 100, and detects a user operation. There are various types of user operations. Examples of operation types include a tap operation, a drag operation, and a pinch operation. The tap operation is an operation in which the user releases the finger immediately after touching the finger at a position where the touch panel is located. The drag operation is an operation in which the user moves a finger touching the touch panel in a certain direction while touching the touch panel. The pinch operation is an operation in which the user changes the interval between two fingers that touch the touch panel. The tap operation includes a single tap operation that is touched only once and a double tap operation that repeats the single tap operation twice quickly. Further, the drag operation includes a swipe operation in which the moving speed is faster than that of the drag operation and a faster flick operation. The pinch operation includes a pinch-out operation that increases the interval between two fingers that touch the touch panel, and a pinch-in operation that decreases the interval between two fingers that touch the touch panel.

  Here, the case where the user performs an operation by touching the touch panel sensor with a finger has been described as an example, but the present invention is not limited thereto. For example, the user may operate using an operation indicator bar instead of a finger. That is, the operation detection unit 110 detects an operation with a finger or an object other than the finger (for example, a pointing stick) as a user operation.

  The imaging unit 120 includes a camera that can capture still images and moving images. The camera includes an imaging element such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor and an optical member such as a lens, and captures an image around the terminal device 10. The imaging unit 120 includes an in-camera 121 having a lens on the front side of the terminal device 10 (display surface side of the display unit 100) and an out-camera 122 having a lens on the back side of the terminal device 10. The in-camera 121 is used when imaging a user who operates the terminal device 10. The out camera 122 is used when capturing an image of the scenery around the terminal device 10.

  The sound reproducing unit 130 includes a speaker and reproduces sound based on input sound information. The sound reproducing unit 130 includes an ear-facing call speaker (first speaker) 131 and a speaker mode (speaking call mode) speaker (second speaker) 132. The ear-talking speaker 131 is used when the terminal device 10 is placed on the user's ear to make a call. The speaker mode speaker 132 is used when sound is reproduced while the terminal device 10 is spoken from the ear. The case where the terminal device 10 is placed on the user's ear to make a call is also referred to as an earing mode or a normal call mode. A case where sound is reproduced while the terminal device 10 is spoken from the ear is also referred to as a loudspeaking call mode, a hands-free mode, a speakerphone mode, or a speaker mode.

  Note that the terminal device 10 may include a proximity sensor 133 for switching between the ear-touch mode and the speaker mode. The proximity sensor 133 detects whether any object (for example, a user's ear) is close to the terminal device 10. In the case where the proximity sensor 133 is provided, when the proximity sensor 133 detects that an object is in proximity to the terminal device 10, the terminal device 10 switches to the earpiece mode. When the proximity sensor 133 detects that an object is not in proximity to the terminal device 10, the terminal device 10 switches to the speaker mode.

  The sound collection unit 140 includes a microphone, collects sounds around the terminal device 10, and converts the collected sounds into sound information. The sounds around the terminal device 10 include the noise around the terminal device 10, the sound that the user is making to make a call, and the sound that is produced by the speaker of the sound reproducing unit 130. That is, the sound collection unit 140 collects various sounds such as noise around the terminal device 10, the voice of the user during a call, and the sound emitted from the speaker of the terminal device 10.

  The vibration unit 145 includes a vibration motor and a weight attached to the rotation shaft of the vibration motor. The vibration unit 145 rotates the rotation shaft when a voltage is applied to the vibration motor. Thereby, the vibration part 145 vibrates a weight by rotation of a rotating shaft, and transmits the said vibration to the housing | casing of an own apparatus.

  The communication unit 150 includes a communication control IC (Integrated Circuit), and performs communication with other devices. The terminal device 10 communicates with other devices by the communication operation by the communication unit 150. In the following description, description of the configuration and operation of the communication unit 150 is omitted.

  The control unit 160 includes a CPU (Central Processing Unit) and controls each unit included in the terminal device 10. The terminal device 10 performs operations such as display on the display unit 100, detection of operation by the operation detection unit 110, vibration of the vibration unit 145, communication with other devices by the communication unit 150, and the like by control of each unit by the control unit 160 I do. In the following description, description of the configuration and operation of the control unit 160 is omitted.

[Overview of information sharing function]
Next, an overview of the information sharing function provided by the inter-terminal communication system 1 will be described with reference to FIG.
FIG. 3 is a schematic diagram illustrating an example of an outline of an information sharing function provided by the inter-terminal communication system 1. Examples of the information sharing function provided by the inter-terminal communication system 1 include a function for sharing an image being displayed, a function for sharing a handwritten image, a function for sharing position information, and an image captured by a camera. There is a function.
An example of the display image by the function of sharing the image being displayed is shown in FIG. An example of a display image by a function for sharing a handwritten image is shown in FIG. An example of a display image by the function of sharing position information is shown in FIG. An example of a display image by a function of sharing an image captured by the camera is shown in FIG.

  These information sharing functions are provided by an information sharing function application that operates on the terminal device 10. In the following description, this information sharing function application is also referred to as an information sharing application AP1. The information sharing application AP1 provides an information sharing function by operating in cooperation with the basic software in the terminal device 10 and other applications.

  Here, the basic software in the terminal device 10 with which the information sharing application AP1 cooperates includes communication software for the terminal device 10 to communicate with other devices. Other applications in the terminal device 10 with which the information sharing application AP1 cooperates include a basic address book application AP2 and an image sharing address book application AP3.

  The basic address book application AP2 is an application that stores thumbnail images, names, telephone numbers, mail addresses, and the like of communication partners. That is, the basic address book application AP2 is an application provided as a basic function of the terminal device 10 regardless of whether or not the terminal device 10 supports the information sharing function. Here, the case where the terminal device 10 is compatible with the information sharing function refers to the case where the terminal device 10 has installed an application that provides the information sharing function. Further, the case where the terminal device 10 does not support the information sharing function refers to a case where the terminal device 10 does not install an application that provides the information sharing function.

  The image sharing address book application AP3 is an application that displays operation buttons and the like used for image sharing. The image sharing address book application AP3 displays various operation images so that a user operating the terminal device 10 can easily use the information sharing function. The image sharing address book application AP3 is an application that provides an information sharing function.

  Here, in describing the information sharing function, a description method of the terminal device 10 in the description of the present embodiment will be described. The operation state of the terminal device 10 has the following two distinctions.

  The first distinction is a distinction between the terminal device 10 that requests information sharing and the terminal device 10 that requests information sharing. In the following description, the terminal device 10 that requests information sharing among the plurality of terminal devices 10 is also referred to as the requesting terminal device 10, and the terminal device 10 that is requested to share information The terminal device 10 on the side is also described. The requesting terminal device 10 makes a request for information sharing to the requested terminal device 10. When the request-side terminal device 10 receives an information sharing request from the request-side terminal device 10, the requested-side terminal device 10 accepts the request to start information sharing with the request-side terminal device 10. In the following description, of the plurality of terminal devices 10, the requesting terminal device 10 is also referred to as a terminal device 10-1, and the requested terminal device 10 is also referred to as a terminal device 10-2.

The second distinction is a distinction between the terminal device 10 that transmits information to be shared and the terminal device 10 that receives information to be shared. In the following description, the terminal device 10 that transmits information to be shared among the plurality of terminal devices 10 is also referred to as an information transmitting terminal device 10 or a terminal device 10-S. Further, the terminal device 10 on the information receiving target side is also referred to as an information receiving terminal device 10 or a terminal device 10-R.
Note that which of the requesting terminal device 10 and the requested terminal device 10 transmits information to be shared and which receives this information depends on the function between the information sharing functions. It depends on the difference and the operating state of the terminal device 10. That is, in the inter-terminal communication system 1, there are a case where the requesting terminal device 10 transmits information to be shared information and a case where the requested terminal device 10 transmits information to be shared information.

  In the following description, the terminal device 10 may be described by combining the first distinction and the second distinction described above. In this case, the requesting and information transmitting terminal device 10 is also referred to as a terminal device 10-1S. The terminal device 10 on the requesting side and the information receiving side is also referred to as a terminal device 10-1R. The terminal device 10 on the requested side and the information transmission side is also referred to as a terminal device 10-2S. The terminal device 10 on the requested side and the information receiving side is also referred to as a terminal device 10-2R.

[Information sharing function 1: Sharing images being displayed (display screen sharing function)]
Next, an overview of each information sharing function will be described with reference to FIG. Among the information sharing functions provided by the inter-terminal communication system 1, first, sharing of an image being displayed (display screen sharing function) will be described.

  The inter-terminal communication system 1 provides a display screen sharing function. With this display screen sharing function, an image P1 displayed on the display unit 100 of the terminal device 10 is displayed as an image P2 on the display unit 100 of another terminal device 10, whereby an image is displayed between the terminal devices 10. It is a function to share. An example of a screen displayed on the terminal device 10 by this display screen sharing function is shown in FIG. The image displayed on the display unit 100 here includes the above-described standby image, application image, and menu image.

  When the display screen sharing function is selected in a certain terminal device 10-1, the inter-terminal communication system 1 uses the image P1 displayed on the display unit 100 of the terminal device 10-1 as a sharing destination. Is displayed on the display unit 100 of the terminal device 10-2 as an image P2.

  A specific example of this display screen sharing function will be described. For example, a case will be described in which a photo album application is activated in the terminal device 10-1, and this photo album application displays a photo image (image P1 shown in FIG. 3A) on the display unit 100. . In this case, the inter-terminal communication system 1 displays the image P1 of the photograph displayed on the display unit 100 of the terminal device 10-1 as the image P2 on the display unit 100 of the other terminal device 10-2 that is the sharing destination. Let In this way, the image P1 of the same photograph is displayed on the respective display units 100 of the two terminal devices 10, whereby the photograph image P1 is shared.

  Next, another specific example of the display screen sharing function will be described. For example, a case where a moving image application is activated in the terminal device 10-1 and the moving image application displays a moving image with sound on the display unit 100 will be described. In this case, the inter-terminal communication system 1 displays the moving image displayed on the display unit 100 of the terminal device 10-1 on the display unit 100 of the other terminal device 10-2 that is the sharing destination. In this way, the same moving image is displayed on the respective display units 100 of the two terminal devices 10, thereby sharing the moving image. Simultaneously with the image sharing, the inter-terminal communication system 1 reproduces the sound reproduced from the sound reproduction unit 130 of the terminal device 10-1 from the sound reproduction unit 130 of the other terminal device 10-2 that is the sharing destination. Let In this way, the sound attached to the moving image is shared by reproducing the same sound from the sound reproducing units 130 of the two terminal devices 10. In this way, a moving image with sound is shared between the two terminal devices 10. That is, according to the inter-terminal communication system 1, it is possible to share both information of image information and sound information.

[Information sharing function 2: Sharing handwritten images (handwriting sharing function)]
Next, among the information sharing functions provided by the inter-terminal communication system 1, handwritten image sharing (handwriting sharing function) will be described with reference to FIG.
The inter-terminal communication system 1 provides a handwriting sharing function for sharing an image of a handwritten graphic. Here, the handwritten graphic image is an image of a drag operation locus displayed on the display unit 100 by dragging the operation detection unit 110 of the terminal device 10 with a finger or the like. That is, the handwriting sharing function refers to an image P3 of a handwritten graphic displayed on the display unit 100 by dragging the operation detection unit 110 of the terminal device 10 with a finger or the like on the display unit 100 of another terminal device 10. This is a function of sharing an image between the terminal devices 10 by displaying as P4.

  A specific example of the handwriting sharing function will be described. For example, the case where the handwriting tool application is activated in the terminal device 10-1 and the handwriting tool application displays an image of a handwritten graphic (image P3 shown in FIG. 3B) on the display unit 100 will be described. To do. In this case, the inter-terminal communication system 1 uses the image P3 of the handwritten graphic displayed on the display unit 100 of the terminal device 10-1 as the image P4 on the display unit 100 of the other terminal device 10-2 that is the sharing destination. Display. In this way, by displaying the same handwritten graphic image on the display units 100 of the two terminal devices 10, the handwritten graphic image is shared.

[Image layer display]
Note that the display unit 100 of the terminal device 10 displays a photograph (for example, the image P5 in FIG. 3B) displayed by the photo album application and a handwritten graphic image (for example, FIG. 3B) displayed by the handwriting tool application. The image P3) can be displayed in an overlapping manner. That is, the display unit 100 of the terminal device 10 can display a plurality of images stacked. In this case, the display unit 100 displays an image using a layer set for each application. This layer is a logical hierarchy of images displayed in a stacked manner.
For example, the display unit 100 displays the image P5 on a first layer that is a lower layer, and the image P3 on a second layer that is a higher layer than the first layer. Here, the upper layer is a layer that displays an image on the near side (side closer to the user) among the plurality of stacked layers. The lower layer is a layer that displays an image on the back side (the side far from the user) among the plurality of stacked layers. For images displayed on these layers, the opacity can be set between 0% and 100%, for example. When the opacity of the image displayed on the upper layer is 100%, the upper image becomes opaque and the lower layer image is hidden, so the lower layer image is not displayed. When the opacity of the image displayed on the upper layer is 0%, the upper image becomes transparent, and the lower layer image is displayed transparently without being hidden.
When there are a plurality of regions in a certain layer, this opacity can be set for each of these regions. For example, if the layer on which the handwriting tool application displays an image includes an area where an image of a handwritten graphic is displayed and an area where nothing is displayed, the opacity of these areas can be set for each area. it can.

Further, the operation detection unit 110 detects an operation for the highest layer. In this example, a case will be described in which the handwriting tool application displays the handwritten graphic image P3 on the second highest layer. In this case, the operation detection unit 110 outputs an operation (for example, a drag operation) detected by the touch panel to a handwriting tool application displaying an image on the highest layer. Based on the drag operation output from the operation detection unit 110, the handwriting tool application displays an image indicating the locus of the drag operation on the display unit 100 as an image of a handwritten figure.
Here, as another example, a case will be described in which the menu control software displays a shared menu button image in a third layer higher than the second layer. In this case, the operation detection unit 110 detects an operation on the highest layer, that is, the third layer. The operation detection unit 110 detects an operation (for example, a tap operation) with respect to the position where the shared menu button image is displayed by using the touch panel. The operation detection unit 110 outputs the detected operation (for example, a tap operation) to menu control software displaying an image on the highest layer. The menu control software selects a function corresponding to the position of the tap operation based on the operation output from the operation detection unit 110, and provides the selected function.
That is, the operation detection unit 110 detects an operation on the highest layer among a plurality of layers of the image displayed on the display unit 100. Therefore, in the example shown in FIG. 3B, when the image P3 of the handwritten graphic is displayed on the lower layer than the image P5 of the photograph, even if the user operates the touch panel of the display unit 100, The image of the handwritten figure corresponding to this operation is not drawn.

In the handwriting sharing function, the inter-terminal communication system 1 can also share a handwritten graphic image bidirectionally between the plurality of terminal devices 10. Specifically, the inter-terminal communication system 1 displays an image P3 of a handwritten graphic displayed on the display unit 100 of the terminal device 10-1 on the display unit 100 of another terminal device 10-2 that is a sharing destination. Display as P4. In this case, the inter-terminal communication system 1 displays the image P6 of the handwritten graphic displayed on the display unit 100 of the terminal device 10-2 on the display unit 100 of the other terminal device 10-1 that is the sharing destination. Display. In this way, by displaying the image of the handwritten graphic on the respective display units 100 of the two terminal devices 10, the image of the handwritten graphic is shared bidirectionally.
In the handwriting sharing function, the mechanism in which the inter-terminal communication system 1 shares an image and the mechanism in which the inter-terminal communication system 1 shares an image in the display image sharing function described above are different in the following points. That is, in the display image sharing function, the inter-terminal communication system 1 shares an image by sharing data obtained by capturing an image displayed on the display unit 100 between the terminal devices 10. On the other hand, in the handwriting sharing function, the inter-terminal communication system 1 shares an image by sharing data for drawing an image, not data obtained by capturing an image. Specifically, in the handwriting sharing function, the terminal device 10-S on the information transmission side, such as the coordinate data of the handwritten graphic detected by the operation detection unit 110 of the terminal device 10-S, and the color and thickness of the line, etc. Data (that is, data for drawing an image) is transmitted to the terminal device 10-R on the information receiving side. The terminal device 10-R on the information receiving side displays an image on the display unit 100 of the terminal device 10-R based on the received coordinate data of the handwritten figure and data such as line color and thickness.

[Information Sharing Function 3: Location Information Sharing (Location Information Sharing Function)]
Next, among the information sharing functions provided by the inter-terminal communication system 1, sharing of location information (location information sharing function) will be described with reference to FIG.
The inter-terminal communication system 1 provides a position information sharing function for sharing position information among a plurality of terminal devices 10. Here, the position information is information indicating a certain position such as a current position of the terminal device 10 or a position designated by the user. Hereinafter, as an example, a case will be described in which this position information is information indicating the current position of the terminal device 10.
The position information includes information indicating the current position of the terminal device 10. The information indicating the current position includes, for example, longitude, latitude, altitude, and direction / direction of the terminal device 10. When the current position of the terminal device 10 is in a building, for example, the position information includes the name of the building, the number of floors (floor name) of the current position, the name of the room in the building, and the like. It may be. In addition, when the current position of the terminal device 10 is, for example, in a theater, a movie theater, or a sports game venue, the position information includes a seat position, a seat symbol number, and a seat type (for example, Reserved seats and non-reserved seats) may be included.

  A specific example of this position information sharing function will be described. For example, a map application is activated in the terminal device 10-1, and this map application displays a map image (map image P9 shown in FIG. 3C) around the current position of the terminal device 10-1. The case where it is displayed in will be described. In this case, the inter-terminal communication system 1 displays the map image P9 displayed on the display unit 100 of the terminal device 10-1 as the map image P10 on the display unit 100 of the other terminal device 10-2 that is the sharing destination. Let In this way, the same map image is displayed on each of the display units 100 of the two terminal devices 10 so that the map image is shared.

  Here, the terminal device 10-1 displays position information indicating the current position of the terminal device 10-1 together with the map image P9 on the terminal device 10-2. Thus, the position information indicating the current position of the terminal device 10-1 is displayed together with the map image on the display units 100 of the two terminal devices 10, so that the map image and the terminal device 10-1 The current position is shared.

  In the location information sharing function, the inter-terminal communication system 1 can also share location information bidirectionally between a plurality of terminal devices 10. Specifically, the inter-terminal communication system 1 displays the terminal device 10 when the position information indicating the current position of the terminal device 10-1 is displayed together with the map image P9 on the display unit 100 of the terminal device 10-2. -2 is displayed together with the map image P10 on the display unit 100 of the terminal device 10-1. Thus, the position information indicating the current position of the terminal device 10-1 and the position information indicating the current position of the terminal device 10-2 are displayed together with the map image on each display unit 100 of the two terminal devices 10. Is done. Thereby, the current positions of the plurality of terminal devices 10 are shared bidirectionally.

[Information sharing function 4: Camera image sharing (camera image sharing function)]
Next, among the information sharing functions provided by the inter-terminal communication system 1, camera image sharing (camera image sharing function) will be described with reference to FIG.
The inter-terminal communication system 1 provides a camera image sharing function for sharing camera images among a plurality of terminal devices 10. Here, the camera image is an image captured by the imaging unit 120 of the terminal device 10. This camera image includes an image of a landscape around the terminal device 10 and an image of a user who operates the terminal device 10. In addition, the camera image includes a real-time image currently captured by the imaging unit 120 of the terminal device 10. That is, the camera image includes a real-time image indicating the current scenery around the terminal device 10 and a real-time image indicating the current state of the user who operates the terminal device 10.

  A specific example of this camera image sharing function will be described. For example, a camera application is activated in the terminal device 10-1, and the camera application displays a real-time image (the real-time image P11 shown in FIG. 3D) that shows the current scenery around the terminal device 10-1. The case where it is displaying on the part 100 is demonstrated. In this case, the inter-terminal communication system 1 displays the real-time image P11 displayed on the display unit 100 of the terminal device 10-1 as the real-time image P12 on the display unit 100 of the other terminal device 10-2 that is the sharing destination. Let In this way, the same real-time image is displayed on the respective display units 100 of the two terminal devices 10, whereby the real-time image is shared.

[Overview of selecting information sharing function]
Next, an outline of an operation for the user to activate the information sharing function will be described with reference to FIG.
FIG. 4 is a schematic diagram illustrating an example of the selection operation of the information sharing function provided by the inter-terminal communication system 1 of the present embodiment.
Here, a case where user A (Mr. A) and user B (Mr. B) use display screen sharing using the terminal device 10 will be described as an example. In the description of this example, the terminal device 10 used by the user A is the requesting and information transmitting terminal device 10, that is, the terminal device 10-1S. In the description of this example, the terminal device 10-2 is the terminal device 10 on the requested side and the information receiving side, that is, the terminal device 10-2R. Some terminal devices 10 may not support the display screen sharing function. In the description of this example, a case where each terminal device 10 is compatible with the display screen sharing function will be described.

  The user A and the user B use the display screen sharing function by sequentially performing the operations shown in FIGS. 4A to 4E. The user A and the user B make a voice call using a voice call session via the LTE communication network. FIG. 4A shows an example of an image displayed on each terminal device 10 when the user A and the user B are making a voice call via the LTE communication network.

  At this time, the image PMB of the shared menu button is displayed on both the display unit 100 of the terminal device 10-1S of the user A and the display unit 100 of the user B. The display of the image PMB of the sharing menu button indicates that the terminal device 10 on the communication partner side supports one of the information sharing functions described above. In this example, the display of the shared menu button image PMB indicates that both the terminal device 10-1S and the terminal device 10-2R support the display screen sharing function.

  User A taps the image PMB of the shared menu button displayed on the display unit 100 of the terminal device 10-1S. When the operation detection unit 110 of the terminal device 10-1S detects this tap operation, a launcher image P20 is displayed (see FIG. 4B). The launcher image P20 is an example of the menu image described above. In this example, the launcher image P20 shows icon images of a display screen sharing function, a handwriting sharing function, a position information sharing function, and a camera sharing function. When the user taps the position on the display unit 100 where the icon image is displayed, the function associated with the icon image is activated. Here, a case where the user A taps the icon image P21 of the display screen sharing function among the information sharing functions displayed on the launcher image P20 will be described.

  When the operation detection unit 110 of the terminal device 10-1S detects this tap operation, a function explanation image P22 of the display screen sharing function is displayed (see FIG. 4C). When the operation detection unit 110 of the terminal device 10-1S detects this tap operation, the screen sharing start consent image P23 is displayed on the display unit 100 of the sharing destination terminal device 10-2R. The consent image P23 is an image displayed on the requesting terminal device 10 for information sharing, and prompts the user to perform an operation indicating whether or not to accept the information sharing request by the requesting terminal device 10. It is an image. In this example, the consent image P23 is an image prompting the user B to perform a tap operation indicating whether or not to accept the screen sharing request from the user A.

  Next, the user B taps the image P24 of the consent button of the consent image P23. When the operation detection unit 110 of the terminal device 10-2R detects this tap operation, an image P25 indicating that it is connected is displayed on each display unit 100 of the terminal device 10-2R and the terminal device 10-1S (FIG. 4 ( See D)). At this time, an inter-terminal communication session for display screen sharing is established between the terminal device 10-1S and the terminal device 10-2R.

Next, when a session for inter-terminal communication for display screen sharing is established between the terminal device 10-1S and the terminal device 10-2R, an image P26 displayed on the display unit 100 of the terminal device 10-1S. Is displayed as an image P27 on the display unit 100 of the terminal device 10-2R (see FIG. 4E). That is, display screen sharing is started between the terminal device 10-1S and the terminal device 10-2R.
When a call end operation is received by either the terminal device 10-1S or the terminal device 10-2R, the terminal device 10 that has received the call end operation ends the voice call session and displays a screen. The communication session between terminals for sharing is also terminated. At this time, the terminal device 10 transmits a session end request to the server device in the LTE communication network in which the voice call session is established, and transmits a session end request to the signaling server 20.

[Configuration of inter-terminal communication system]
Next, the configuration of the inter-terminal communication system 1 that provides the above-described information sharing function will be described with reference to FIG.
FIG. 5 is a schematic diagram illustrating an example of the configuration of the inter-terminal communication system 1 of the present embodiment. The inter-terminal communication system 1 includes the following devices in addition to the plurality of terminal devices 10 (in this example, the terminal device 10-1S and the terminal device 10-2R). That is, the inter-terminal communication system 1 includes an RMS (Rights Management Server) server 20, an IPDB (Internet Protocol Database) server 30, a push server 40, and a relay server 50.

  The signaling server 20 manages a session of information sharing communication performed between the terminal devices 10. Specifically, the signaling server 20 can communicate with a plurality of terminal devices 10, the authentication server 30, and the push server 40. When the terminal device 10 supports the information sharing function, the signaling server 20 performs registration (initial registration) of information of the terminal device 10. Further, when there is a request for information sharing from the terminal device 10, the signaling server 20 performs signaling processing for establishing a communication session with the sharing destination terminal device 10. In addition, when there is a request for information sharing from the terminal device 10, the signaling server 20 returns a response as to whether or not the terminal device 10 that is the sharing request destination is compatible with the information sharing function. To do.

  Further, the signaling server 20 determines whether or not the terminal device 10 is a legitimate device at the start of session construction. Whether or not the terminal device 10 is a legitimate device can be determined by various means. In this example, the signaling server 20 determines whether or not the terminal device 10 is a regular device based on the telephone number and the IP address transmitted from the terminal device 10. More specifically, the signaling server 20 is based on whether or not the telephone number and IP address transmitted from the terminal apparatus 10 match the telephone number and IP address previously assigned to the terminal apparatus 10. Then, it is determined whether or not the terminal device 10 is a regular device. In the following description, the determination performed by the signaling server 20 as to whether or not the terminal device 10 is a regular device is also referred to as secure determination.

  In addition, the signaling server 20 stores registration information. The signaling server 20 uses this registration information to perform session construction and secure determination. Registration information stored in the signaling server 20 will be described with reference to FIG.

  FIG. 6 is a table showing an example of registration information stored in the signaling server 20 of the present embodiment. The signaling server 20 stores the telephone number of the terminal device 10, the terminal model name, the push ID, and the application version as registration information. Here, the push ID is identification information for identifying an application for the information sharing function that operates on the terminal device 10.

  Returning to FIG. 5, after the session is established, the signaling server 20 determines whether or not the terminal device 10 is a regular device at a predetermined frequency. The signaling server 20 establishes a session or maintains an established session only when it is determined that the terminal device 10 is a legitimate device. Specifically, when the signaling server 20 determines that the terminal device 10 is a legitimate device, the signaling server 20 issues a token to the terminal device 10. This token is permission information for the terminal devices 10 to communicate with each other via a high-speed communication line. This token has an expiration date from the time of issue. The terminal device 10 can share screen information with other terminal devices 10 when this valid token is issued. Further, the terminal device 1 cannot share screen information with other terminal devices 10 when a token has not been issued and when the validity period of the issued token has expired. In this way, the signaling server 20 manages a session between the terminal devices 10.

  The authentication server 30 stores the telephone number of the terminal device 10 and the Internet protocol address (IP address) in association with each other. When there is a request for an IP address corresponding to the telephone number from the signaling server 20, the authentication server 30 returns an IP address corresponding to the telephone number of the terminal device 10 to the signaling server 20.

  If the combination of the telephone number and the IP address for a certain terminal device 10 does not match the combination of the telephone number and the IP address stored in the authentication server 30, this terminal device 10 is not a legitimate device. there is a possibility. The signaling server 20 acquires a telephone number and an IP address from a certain terminal device 10, and compares the telephone number and the IP address stored in the authentication server to see if the combination of the telephone number and the IP address matches. Determine. As a result of this comparison, when it is determined that the combination of the telephone number and the IP address matches, the signaling server 20 issues a token to the terminal device 10. Further, as a result of this comparison, when it is determined that the combination of the telephone number and the IP address does not match, the signaling server 20 does not issue a token to the terminal device 10.

  The push server 40 stores a plurality of push IDs. The push ID stored in the push server 40 is associated with the application operating on the terminal device 10 by the signaling server 20. Specifically, when a new downloaded application is activated for the first time in the terminal device 10, the terminal device 10 requests the push server 40 to issue a push ID. The push server 40 issues a push ID for the new application of the terminal device 10 based on this request, and transmits the issued push ID to the terminal device 10. The terminal device 10 transmits information associating the new application and the issued push ID as registration information to the signaling server 20. The signaling server 20 stores the registration information transmitted from the terminal device 10. Thereby, the push ID stored in the push server 40 and the application operating on the terminal device 10 are associated by the signaling server 20.

  Further, when the application is upgraded in the terminal device 10, the push server 40 issues a new push ID along with the upgrade. That is, the push server 40 is updated with the update of the application version of the application running on the terminal device 10.

  Further, when the signaling server 20 requests the requesting terminal device 10 to start the session construction, the push server 40 uses the push technology to obtain the telephone number of the requesting terminal device 10 by the push technique. 10 is transmitted. This push technology is a technology for delivering a communication request to the terminal device 10 from the server device side (in this example, the signaling server 20 and the push server 40 side) using an IP address. In the following description, transmitting information by the push technique is also referred to as simply pushing information.

  The relay server 50 is a relay server that performs information sharing communication between a plurality of terminal devices 10 when peer-to-peer connection is not possible in information sharing communication between the plurality of terminal devices 10. A peer-to-peer connection is also referred to as a P2P connection. When a P2P session of information sharing communication between a plurality of terminal devices 10 cannot be established even after a predetermined time, a session via the relay server 50 is established.

[Operation procedure of inter-terminal communication system]
Next, an operation procedure of the inter-terminal communication system 1 will be described with reference to FIGS. First, an operation procedure of registration (initial setting) of the terminal device 10 will be described with reference to FIG. Next, with reference to FIG. 8 and FIG. 9, the operation procedure of session construction by the inter-terminal communication system 1 will be described. The number of step S shown in FIG. 5 corresponds to the number of step S in FIGS.

[Operation procedure of inter-terminal communication system 1: initial setting]
With reference to FIG. 7, an operation procedure of registration (initial setting) of the terminal device 10 will be described. Here, a case where the terminal device 10-1 and the terminal device 10-2 perform registration in order will be described.

FIG. 7 is a flowchart illustrating an example of an operation procedure of registration according to the present embodiment.
When the application corresponding to the information sharing function is activated in the terminal device 10-1, the terminal device 10-1 transmits a push ID issuance request corresponding to the activated application to the push server 40 ( Step S10).
Upon receiving the push ID issuance request, the push server 40 transmits a push ID corresponding to the activated application to the terminal device 10-1 (step S20). The terminal device 10-1 acquires a push ID from the push server 40 (step S30).

  Next, the terminal device 10 transmits registration information to the signaling server 20 (step S40). Here, the registration information includes the push ID acquired in step S10, the application version information of the application corresponding to the push ID, the phone number of the own device, and the terminal model name of the own device.

  The signaling server 20 stores the registration information transmitted in step S20 (step S50), and ends the registration process for the terminal device 10-1.

  Next, the signaling server 20 transmits a telephone number to the authentication server 30 (step S60). This telephone number is a telephone number included in the registration information transmitted in step S20.

  The authentication server 30 receives the telephone number transmitted in step S60, determines whether this telephone number is a telephone number registered in advance, and transmits this determination result to the signaling server 20 (step). S70). Here, the telephone number registered in advance in the authentication server 30 is information indicating that the terminal device 10 is a legitimate terminal device 10. That is, the authentication server 30 determines whether or not the telephone number included in the registration information transmitted from the terminal device 10 is the regular telephone number of the terminal device 10. Moreover, the regular terminal device 10 here is the terminal device 10 used by a regular user registered in advance. That is, the signaling server 20 determines whether or not the terminal device 10 that has transmitted the registration information is a terminal device 10 that is used by a registered regular user. That is, the signaling server 20 authenticates the user of the terminal device 10 in step S60 and step S70.

  Further, the terminal device 10-2 performs registration through steps S10-2 to S50-2 shown in FIG. Here, Steps S10-2 to S50-2 correspond to Steps S10 to S50 described above. A description of steps S10-2 to S50-2 will be omitted.

[Operation of inter-terminal communication system 2: session construction]
Next, with reference to FIG. 8 and FIG. 9, the operation procedure of session construction of the terminal device 10 will be described.
FIG. 8 is a flowchart illustrating an example of an operation procedure for session construction according to this embodiment. Here, the case where a session is constructed | assembled between the terminal device 10-1 and the terminal device 10-2 is demonstrated. FIG. 9 is a flowchart showing a sharing consent process in the session construction operation procedure of the present embodiment. Here, a case will be described in which the terminal device 10-1 is the information sharing requesting side and the terminal device 10-2 is the information sharing requested side.

The requesting terminal device 10-1 transmits an information sharing request to the signaling server 20 (step S100).
The signaling server 20 receives an information sharing request from the terminal device 10 (step S110). This request for information sharing includes the telephone number of the requested terminal device 10.

Next, the signaling server 20 compares the telephone number received in step S110 with the registration information stored in advance, and determines whether or not the received telephone number is available (step S120). .
The signaling server 20 can also determine the push ID instead of or in addition to the determination of the telephone number. Specifically, the signaling server 20 determines whether or not the push ID is registered in the registration information corresponding to the telephone number received in step S110.

Next, when the signaling server 20 determines that the telephone number received in step S110 is available (step S120; YES), the information sharing function can be used for the requesting terminal device 10. A notice to that effect is sent (step S130). On the other hand, if the signaling server 20 determines that the telephone number received in step S110 is not usable (step S120; NO), the information sharing function cannot be used for the requesting terminal device 10. This is notified and the process is terminated.
In the above description, the signaling server 20 indicates that the information sharing function can be used for the requesting terminal device 10 in the process of establishing a session after the signaling server 20 requests information sharing. Although described as notification, it is not limited to this. As an example, the signaling server 20 may notify the requesting terminal device 10 that the information sharing function is available before the information sharing request is made. In this case, the signaling server 20 sends a request to the requesting terminal device 10 before the information sharing request is made by the background processing of the voice call at the start of the voice call between the terminal devices 10 or during the voice call. That the information sharing function is available.

  As another modification, the signaling server 20 may notify that the information sharing function can be used according to the conditions exemplified below instead of or in addition to the determination of the telephone number. Good. As an example, the signaling server 20 indicates that the information sharing function can be used when the information sharing application is installed in both the requesting terminal device 10 and the requested terminal device 10. Notice. As an example, in the signaling server 20, the application version of the information sharing application installed in the requesting terminal device 10 and the application version of the information sharing application installed in the requested terminal device 10 are the same. If it is, it is notified that the information sharing function is available. Further, as an example, the signaling server 20 is compatible with the information sharing application installed in the requesting terminal device 10 and the information sharing application installed in the requested terminal device 10. Notify that the information sharing function is available.

  As another example, the signaling server 20 is a terminal device in which both the requesting terminal device 10 and the requested terminal device 10 support communication via a high-speed data communication line (for example, VoLTE). In such a case, a notification that the information sharing function is available is sent.

Next, when receiving the notification that the information sharing function is available, the requesting terminal device 10 transmits a session construction request to the signaling server 20 (step S140). This request for session construction includes the telephone number and IP address of the terminal device 10 on the transmission request side and the telephone number of the terminal device 10 on the request side.
Note that there is no limitation on the order of processing timing between step S100 and step S140 described above. For example, step S100 may be performed first, step S140 may be performed later, and step S100 and step S140 may be performed simultaneously.

  Next, when the signaling server 20 receives the session construction request transmitted in step S130, the signaling server 20 transmits the session construction information to the push server 40 (step S150). This session construction information includes the telephone number of the requesting terminal device 10 and the push ID. At this time, the signaling server 20 acquires the push ID by searching the registration information using the telephone number of the requested terminal device 10 as a search key. This push ID corresponds to an information sharing application that operates in the requested terminal device 10.

  Upon receiving the session construction information transmitted in step S150, the push server 40 pushes an information sharing application activation request to the requested terminal device 10 based on the push ID included in the session construction information. (Step S160). The information pushed by the push server 40 at this time includes the telephone number of the requesting terminal device 10. The information pushed by the push server 40 is information resulting from the requesting terminal device 10 sending an information sharing request (step S100). The information is sent to the information sharing request sent from the requesting terminal device 10. Equivalent to.

  The requested terminal device 10 proceeds to the sharing consent process (step S300) when the push in step S160 is performed. In this sharing consent process, as shown in FIG. 9, the requested terminal device 10 determines whether or not the received telephone number of the requesting terminal device 10 matches the telephone number of the other party currently on the call. Determination is made (step S170). If the requested terminal device 10 determines that the received telephone number of the requesting terminal device 10 matches the telephone number of the other party who is currently talking (step S170; YES), the process proceeds to step S180. Proceed. On the other hand, if the requested terminal device 10 determines that the received telephone number of the requesting terminal device 10 does not match the telephone number of the other party who is currently calling (step S170; NO), information sharing is performed. Is transmitted to the requesting terminal device 10 via the signaling server 20 and the processing is terminated.

  Next, the requested terminal device 10 displays the information sharing approval image P23 and waits for the user's approval operation (step S180). The requested terminal device 10 transmits the session construction approval information to the signaling server 20 when the user performs an operation of approval (step S180; YES) (step S190). This session construction approval information includes the telephone number, IP address, and port number of the requested terminal device 10.

  When the requested terminal device 10 performs an operation that the user accepts (step S180; YES), the terminal device 10 switches from the earpiece mode to the speaker mode (speaking call mode). In other words, the requested terminal device 10 stops the ear-talking speaker 131 and activates the speaker-mode speaker 132. At this time, the requested terminal device 10 switches the output destination of the audio data transmitted from the requesting terminal device 10 from the speaker 131 for calling to the speaker 132 for speaker mode. As a result, the requested-side terminal device 10 causes the speaker mode speaker 132 to output sound instead of the ear-talking speaker 131. Further, the requested terminal device 10 may stop the detection of the object by the proximity sensor 133 until the information sharing session is ended when the user performs an operation to accept (step S180; YES). Thus, the requested terminal device 10 can avoid erroneous detection of the proximity sensor 133 during the period in which the information sharing session is established, and can avoid switching from the speaker mode to the earpiece mode.

The requested terminal device 10 automatically switches from the earpiece mode to the speaker mode when the operation detection unit 110 does not accept an operation after a predetermined period of time after displaying the approval image P23, Audio may be output from the speaker 132 for speaker mode. Thereby, for example, the ear-addressing mode is continued in a period in which the user removes his / her ear from the ear-calling speaker 131 and the user accepts the consent image, and the conversation in the ear-addressing mode can be prevented from being interrupted. Even if the user is not aware of the notification of information sharing in step S172, the user can be prompted to view the approval image P23 on the display unit 100 by switching to the speaker mode.
In addition, it is desirable that the operation of automatically switching from the earpiece mode to the speaker mode can be changed by setting change processing in the information sharing application. When the information sharing application set to automatically prohibit the switching operation from the earpiece mode to the speaker mode receives the information sharing request transmitted from the requesting terminal device 10, the requesting terminal device 10 Send setting information that cannot be shared. When receiving the setting information that cannot share information, the requesting terminal device 10 displays a message “cannot share information because switching to the speaker mode is prohibited” on the display unit 100. As a result, the requested terminal device 10 can avoid the automatic switching from the earpiece mode to the speaker mode in response to receiving the information sharing request. On the other hand, the requesting terminal device 10 can notify the user of the reason why the information sharing request is not accepted even though the information sharing request is transmitted, and can prompt the user to start information sharing through conversation.

  The requested-side terminal device 10 described above displays the information sharing approval image P23, shares information with the requesting terminal device 10 and switches to the speaker mode only by performing an operation approved by the user. It is possible to improve operability when sharing information with the terminal device 10 on the request side. That is, an operation in which a request for information sharing is accepted during a call in the earphone mode is a trigger for the user to remove the ear from the earphone speaker 131 and display information shared on the display unit 100. Become. Therefore, the requested terminal device 10 can automatically amplify the sound from the speaker mode speaker 132 in response to an operation that the user accepts. As a result, the terminal device 10 on the requested side does not have to perform an operation of switching the speaker from which the user outputs sound from the speaker 131 for talking to the speaker 132 for speaker mode, and can improve operability. .

  On the other hand, if the requested terminal device 10 performs an operation not accepted by the user (step S180; NO), the requesting terminal device 10 notifies the requesting terminal device 10 via the signaling server 20 that the information sharing is not accepted. To finish the process.

  Next, when the signaling server 20 receives the session construction approval information transmitted in step S190, the signaling server 20 constructs a session between the requesting terminal device 10 and the requested terminal device 10 (FIG. 8, Step S200), the session construction process is terminated. Specifically, in step S200, the signaling server 20 transmits the IP address and port number of the requested terminal device 10 transmitted in step S190 to the requesting terminal device 10. Next, the requesting terminal device 10 transmits the IP address and port number of the requesting terminal device 10 to the signaling server 20. When the signaling server 20 receives the IP address and port number of the requesting terminal device 10, the signaling server 20 transmits the received IP address and port number to the requesting terminal device 10. As a result, the requesting terminal device 10 acquires the IP address and port number of the requested terminal device 10. Further, the requested terminal device 10 acquires the IP address and port number of the requesting terminal device 10. That is, the plurality of terminal devices 10 acquire the IP address and port number of the information sharing partner terminal device 10 from each other.

  When the session construction process ends, the terminal device 10-1 and the terminal device 10-2 share information with each other (step S210).

When the requesting terminal device 10 and the requested terminal device 10-2 are switched from the speaker mode to the earpiece mode based on the detection result of the object by the proximity sensor 133, the information sharing session is terminated. Information notifying the end of information sharing may be transmitted to the partner terminal device 10. At this time, the terminal device 10 causes the signaling server 20 to transmit an information sharing session termination request. Also, the terminal device 10 that has received the information notifying that the information sharing is to be ended is displayed on the display unit 100 to notify the user that the information sharing has been ended.
Further, the requested terminal device 10 may display the consent image P23 and may transmit the consent information to the signaling server 20 before accepting the consent operation. Thereby, the terminal device 10 causes the signaling server 20 to construct an information sharing session. Thereafter, the requested terminal device 10 receives the information transmitted from the requesting terminal device 10. As described above, the inter-terminal communication system 1 can start sharing information in the background after displaying the consent image P23 on the requested terminal device 10, and is shared in a short period after the acceptance operation is accepted. Information can be displayed. When the requested terminal device 10 accepts an operation for rejecting the information sharing request for the consent image P23, the requesting terminal device 10 transmits a request to end the information sharing session to the signaling server 20, and the signaling server 20 Disconnect the information sharing session.

[Operation of communication system between terminals 2: Modification 1 of session construction]
Next, with reference to FIG. 10, the modification 1 of the session construction in the operation | movement of the communication system between terminals mentioned above is demonstrated. FIG. 10 is a flowchart showing another example of the operation procedure of the share acceptance process in the session construction of this embodiment.

  In the second modification of session construction, when the requested terminal device 10 determines that the received telephone number of the requesting terminal device 10 matches the telephone number of the other party who is currently talking (step S170; YES) In this case, the information sharing request is notified to the user. At this time, the requested-side terminal device 10 causes the control unit 160 to vibrate the vibration unit 145 or causes the control unit 160 to output sound from the call speaker 131. The requested terminal device 10 transmits vibrations to the user's ears, cheeks, and the like by generating vibrations. Further, the requested terminal device 10 supplies a predetermined notification sound as audio data to the speaker 131 for calling and makes the user hear the notification sound together with the audio data during the call.

  Thereafter, the requested terminal device 10 displays the information sharing approval image P23 and waits for the user's approval operation (step S180). It should be noted that the information sharing approval image P23 is a predetermined number of times until the user removes his ear from the call speaker 131 and visually recognizes the approval image P23 when notifying the information sharing request or after notifying the information sharing request. Displayed in time. Note that the requested terminal device 10 may display the approval image P23 before notifying the information sharing after determining that the requesting terminal device 10 matches the telephone number of the other party who is currently talking.

  The requested terminal device 10 described above notifies the user of the request in response to receiving the information sharing request transmitted from the requesting terminal device 10, so that the approval image of the display unit 100 is displayed to the user. It can be urged to see P23. As a result, according to the requested terminal device 10, the user of the requesting terminal device 10 verbally communicated to the user of the requested terminal device 10 that the user of the requesting terminal device 10 has transmitted a request for information sharing during a call. However, it is possible to prevent the display unit 100 from being confirmed in a state where the approval image P23 is not yet displayed due to a session construction time lag or the like, and to suppress the troublesomeness given to the user of the requested terminal device 10 can do. Further, according to the requested terminal device 10, it is possible to make the user recognize in advance that there is an information sharing request notification. Therefore, if the display unit 100 is viewed after the information sharing notification is received, the approval image is displayed. A sense of security that P23 is displayed can be given, and the operability can be further improved.

[Operation of communication system between terminals 2: Modification 2 of session construction]
Next, with reference to FIG. 10, the modification 2 of the session construction in operation | movement of the communication system between terminals mentioned above is demonstrated. FIG. 10 is a flowchart showing another example of the operation procedure of session construction according to this embodiment.

  In the second modification of session construction, the requested-side terminal device 10 waits until the user's ear leaves the own device after notifying the information sharing (step S172) (step S174). The requested terminal device 10 indicates that the object detection result of the proximity sensor 133 is “there is an object” before notification of information sharing, and after the notification of information sharing, the object detection result of the proximity sensor 133 is “ When the detection result of “no object” continues for a predetermined time or more, it is determined that the user's ear has left the own device, and then the requested terminal device 10 The approval image P23 is displayed, and the user's approval operation is waited (step S180).

  The information sharing approval image P23 is displayed at the same time when it is determined that the user's ear is away from the user's ear, or within a predetermined time until the user removes the ear from the call speaker 131 and views the acceptance image P23. Is done. Note that the terminal device 10 on the requested side may display the information sharing approval image P23 before determining that the user's ear is away from the own device after determining that it matches the telephone number of the other party who is currently talking. Good.

  The requested terminal device 10 described above notifies the user of the request in response to receiving the information sharing request transmitted from the requesting terminal device 10, and after the notification, the information sharing approval image P23. Can be displayed. Thus, according to the requested terminal device 10, after the information sharing notification is made, the information sharing approval image P23 can be displayed when the user actually takes action to view the information sharing approval image P23. In addition, the display can be controlled with the user's action as a trigger, and the operability can be improved. Further, according to the requested terminal device 10, it is possible to suppress the display of the approval image P <b> 23 when the user does not want to view the information sharing approval image P <b> 23. It is possible to suppress the troublesomeness of the user caused by switching between the two.

[Operation of communication system between terminals 2: Modification 3 of session construction]
Next, with reference to FIG. 11, the modification 3 of the session construction in the operation | movement of the communication system between terminals mentioned above is demonstrated. FIG. 11 is a flowchart showing another example of the operation procedure of session construction according to this embodiment.

  In the third modification of session construction, when the requested terminal device 10 performs an operation that the user accepts (step S180; YES), the earphone mode is switched to the speaker mode after a predetermined period, and during this predetermined period, The volume adjustment image is displayed and the volume adjustment in the speaker mode is performed (step S190). That is, the requested terminal device 10 prompts the user of the requested terminal device 10 to adjust the volume.

  Specifically, the requested terminal device 10 displays the volume adjustment image P30 when receiving an operation approved by the user. FIG. 12 is a diagram showing a volume adjustment image P30 displayed on the requested terminal device 10. As shown in FIG. As shown in FIG. 12A, the volume adjustment image P30 is displayed on the consent image P23, which is a requested launcher image that is activated when the requested terminal device 10 receives an information sharing request. included. The volume adjustment image P30 includes a volume adjustment slider P31. The volume adjustment slider P31 is slid by a drag operation between volume: low and volume: high. Volume: The lower end is a position corresponding to the minimum volume or mute in the speaker mode. On the other hand, the volume: high side end is a position corresponding to the maximum volume in the speaker mode. Note that the volume adjustment image P30 may include a message that prompts the user to adjust the volume in the speaker mode. This message is "Automatically switch to speaker call after XX seconds. Please adjust the volume."

The requested terminal device 10 waits until a predetermined period elapses from the consent operation (step S196). When a predetermined period has elapsed since the authentication operation (step S196; YES), the requested terminal device 10 outputs a sound from the speaker mode speaker 132 instead of the ear-talking speaker 131. At this time, the requested terminal device 10 determines the position of the volume adjustment slider P31 when a predetermined period has elapsed from the operation of accepting the information sharing, and determines the volume in the speaker mode. It is desirable that the requested terminal device 10 increase the volume of the sound output from the speaker mode speaker 132 gradually from the low volume to the volume in the determined speaker mode. As a result, even if the requested terminal device 10 is automatically switched to the speaker mode, it is possible to suppress sudden output of a loud sound.
Further, the requested-side terminal device 10 displays the consent image P23 in FIG. 12A for a predetermined time, and when a period during which no operation for the consent image P23 is accepted has passed a predetermined period, FIG. As shown in FIG. 5, an image including the image PMB of the shared menu button may be displayed as the consent image P23. When the requested terminal device 10 accepts an operation on the image PMB of the share menu button, the requester-side terminal device 10 can display the launcher image P20 in the same manner as the requesting terminal device 10, and can activate the information sharing application. it can.

Furthermore, the requested terminal device 10 may change the consent image P23 as a launcher image based on the type of information to be shared. FIG. 13 is a diagram illustrating another example of the consent image P23 as a launcher image to be displayed on the requested terminal device 10.
The requested terminal device 10 determines whether the audio data is included in the information shared by the information sharing session. At this time, if the information sharing request is a request to share information on the screen or camera image of the requesting terminal device 10, the requested terminal device 10 includes audio data in the information shared by the information sharing session. Is determined. If the information sharing request is a request for sharing position information or a handwritten image, the requested terminal device 10 determines that the information shared by the information sharing session does not include audio data.

  When the information to be shared includes voice data, the requested terminal device 10 is communicated by the information sharing session and the image P31b that adjusts the volume of the voice communicated by the voice call session to the volume adjustment image P30. The image P31a for adjusting the volume of the sound to be displayed is included and displayed (FIG. 13A). These images P31a and P31b may be the same as the above-described volume adjusting slider. The requested terminal device 10 adjusts the volume of the voice communicated by the voice call session and the volume of the voice communicated by the information sharing session based on the operation received for the volume adjustment image P30. On the other hand, if the information shared by the information sharing session does not include audio data, the requested terminal device 10 includes the volume adjustment image P30 including the image P31b for adjusting the volume of the voice communicated by the voice call session. Display (FIG. 13B). Then, the requested terminal device 10 adjusts the volume of the voice communicated by the voice call session based on the operation received for the volume adjustment image P30.

  As a result, when the requested terminal device 10 outputs both the call voice and the information-sharing voice, the volume of both voices can be adjusted, further improving the operability of the call and information sharing. Can be improved. In addition, when the information to be shared does not include voice, the requested terminal device 10 does not use the image P31a for adjusting the volume of the voice communicated by the information sharing session but communicates by the voice call session. Since only the image P31 for adjusting the sound volume is displayed, the operability can be further improved.

  Further, when a predetermined period elapses from the authentication operation, the requested terminal device 10 switches the display image from the volume adjustment image P30 to the image P27 to which information is shared. FIG. 14 is a diagram showing an image to be displayed on the requested terminal device 10 in the inter-terminal communication system of the present embodiment. The requested terminal device 10 superimposes and displays a speaker icon P40 indicating that sound is output in the speaker mode on the image P27. As a result, the requested terminal device 10 presents that it is currently in the speaker mode and that the speaker mode speaker 132 is outputting sound.

  Further, the inter-terminal communication system 1 may display on the requesting terminal device 10 as the speaker icon P40 that the requested terminal device 10 is outputting sound in the speaker mode. Further, the inter-terminal communication system 1 may display on the requested terminal device 10 as the speaker icon P40 that the requesting terminal device 10 is outputting sound in the speaker mode.

  For this purpose, the inter-terminal communication system 1 sends status information indicating whether the requesting terminal device 10 is in the earpiece mode or the speaker mode from the requesting terminal device 10 to the requested terminal device 10. Send it. Further, the inter-terminal communication system 1 transmits status information indicating whether the requested terminal device 10 is in the earpiece mode or the speaker mode from the requested terminal device 10 to the requesting terminal device 10. Let As a result, the requesting terminal device 10 and the requested terminal device 10 exchange status information with each other, and display the status information of the other party during the call as the speaker icon P40 on the own device.

  The requested terminal device 10 and the requesting terminal device 10 mutually exchange a speaker icon P40 indicating that the own device is in speaker mode and a speaker icon indicating that the other party is on speaker. Display as a distinct icon. As a result, the inter-terminal communication system 1 can inform the user whether or not the terminal device 10 of the other party is sharing information while making a call in the speaker mode. Encourage the use of shared applications.

  The requested-side terminal device 10 described above displays the volume adjustment image P30 during a predetermined period after the acceptance operation is accepted, so that the volume in the speaker mode can be adjusted before switching to the speaker mode. Therefore, according to the requested terminal device 10, even if the mode is automatically switched to the speaker mode, it is possible to prevent the voice call from being continued at a volume not intended by the user, and the operability can be further improved.

  Further, according to the requested terminal device 10, when switching to the speaker mode, the speaker icon P40 indicating that the speaker mode speaker 132 of the own device is operating is displayed, so that the own device is in the speaker mode. It can be visually recognized that the switch has been made. Thereby, according to the requested terminal device 10, even if the mode is automatically switched to the speaker mode, the user can be recognized, for example, prompting the user to adjust the volume in the speaker mode or to stop the speaker mode. And the operability can be further improved.

  Further, according to the requested terminal device 10, the speaker icon for the speaker mode speaker 132 of the other party in a call can be displayed on the own apparatus, and the speaker mode speaker 132 of the other party in a call can be displayed. Can recognize that sound is output from Thereby, according to the requested terminal device 10, it is possible to suppress an unintended call from being output from the speaker mode speaker 132 of the other party during the call.

[Modification: Tablet terminal]
In addition, although the case where the terminal device 10 was a mobile phone, a smart phone, etc. was demonstrated as an example until now, it is not restricted to this. For example, the terminal device 10 may be a device such as a tablet. Here, the mobile phone and the smartphone are provided with an ear-facing call speaker 131, and can be talked to the ear. On the other hand, the tablet does not include the earphone speaker 131. That is, the terminal device 10 may be a device that does not include the earphone call speaker 131. When the terminal device 10 is a tablet, a call may be performed in the speaker mode.

[Variation: Information sharing during text communication]
In the above description, the inter-terminal communication system 1 has been described as providing an information sharing function during a voice call. However, the present invention is not limited to this. The inter-terminal communication system 1 may provide an information sharing function during text communication. For example, the inter-terminal communication system 1 may provide an information sharing function during chat provided by a text chat application.

  As mentioned above, although embodiment of this invention and its deformation | transformation were demonstrated, these embodiment and its deformation | transformation were shown as an example and are not intending limiting the range of invention. These embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and equivalents thereof.

  Each of the above devices has a computer inside. The process of each device described above is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer reading and executing the program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Communication system between terminals, 10 ... Terminal device, 20 ... Signaling server, 30 ... Authentication server, 40 ... Push server, 50 ... Relay server

Claims (9)

A first terminal device that executes a first information sharing function application;
A second terminal device for executing a second information sharing function application;
In response to an information sharing request transmitted from the first information sharing function application while a voice call session is established between the first terminal device and the second terminal device, the second information sharing function application And a server device that constructs an information sharing session in response to the acceptance response transmitted from the second information sharing function application,
The second information sharing function application displays a consent image for selecting whether to accept the information sharing request based on the information sharing request transmitted from the server device on the display unit, and accepts the information sharing request. In response to the operation being received by the operation unit, switching from the earpiece communication mode for driving the first speaker for ear pads to the voice communication mode for driving the second loudspeaker,
A communication system between terminals. The second information sharing function application causes the first speaker or the second speaker to generate sound in response to receiving the information sharing request, or causes the vibration unit to generate vibration.
The inter-terminal communication system according to claim 1. In the second information sharing function application, after the sound is generated in the first speaker or the second speaker or the vibration unit is vibrated, the object is separated from the first speaker by the proximity sensor. In response to the detection, the consent image is displayed.
The inter-terminal communication system according to claim 2. The second information sharing function application switches to the voice communication mode after a predetermined time when an operation for accepting an information sharing request is received by the operation unit, and adjusts the volume of the second speaker during the predetermined time. An adjustment image is displayed on the display unit, and the volume of the second speaker in the loudspeaking communication mode is adjusted based on an operation received with respect to the volume adjustment image.
The inter-terminal communication system according to any one of claims 1 to 3. The second information sharing application is
When audio data is included in the information shared by the information sharing session, an image for adjusting the volume of the voice communicated by the voice call session to the volume adjustment image and the volume of the voice communicated by the information sharing session are set. The volume of the voice communicated by the voice call session and the volume of the voice communicated by the information sharing session based on an operation received with respect to the volume adjustment image, including the image to be adjusted and displayed on the display unit Adjust
When audio data is not included in the information shared by the information sharing session, the volume adjustment image includes an image for adjusting the volume of the voice communicated by the voice call session and is displayed on the display unit. Adjusting the volume of the voice communicated by the voice call session based on the operation received for the adjustment image;
The inter-terminal communication system according to claim 4. After the second information sharing function application is switched to the voice communication mode, the second information sharing function application receives the voice communication mode from the voice communication mode in response to the proximity sensor detecting that the object is close to the first speaker. Switch to the contact communication mode,
The inter-terminal communication system according to any one of claims 1 to 5. The second information sharing function application indicates an icon indicating that the second speaker of the own device is operating in the voice communication mode or indicates that the second speaker of the first terminal device is operating. Displaying an icon on the display unit,
The inter-terminal communication system according to any one of claims 1 to 6. While the voice communication session is established between the first terminal device that executes the first information sharing function application and the second terminal device that executes the second information sharing function application, the server device In response to the information sharing request transmitted from the information sharing function application, an information sharing request is transmitted to the second information sharing function application,
The server device constructs an information sharing session in response to the acceptance response transmitted from the second information sharing function application,
The second information sharing function application causes the display unit to display a consent image for selecting whether to accept the information sharing request based on the information sharing request transmitted from the server device;
When the second information sharing function application accepts an operation for accepting an information sharing request by the operation unit, the second speaker for loudspeaking from the earpiece communication mode for driving the first speaker for earpiece. To switch to the voice communication mode that drives
Terminal-to-terminal communication method. A display for displaying information;
An operation unit for receiving an operation;
A communication unit for transmitting and receiving data;
A first speaker for ear pads that outputs audio based on audio data received by the communication unit;
A second loudspeaker for outputting voice based on the voice data received by the communication unit;
An application execution unit for executing an information sharing function application;
Select whether to accept the information sharing request based on the information sharing request transmitted from the other terminal device and received by the communication unit while a voice call session is established with the other terminal device An acceptance image is displayed on the display unit, and the second speaker is driven from an earpiece communication mode for driving the first speaker in response to an operation of accepting an information sharing request being accepted by the operation unit. A control unit for switching to the loudspeaker communication mode;
A terminal device.

Images