JP5537641B2 - Terminal device, cooperative display system, and cooperative display method - Google Patents

Description

  The present invention relates to a terminal device, a cooperative display system, and a cooperative display method.

  Conventionally, there is a method in which a plurality of terminal devices operate in cooperation to display more information. For example, Patent Document 1 discloses a method of dividing and displaying on a plurality of mobile terminals when display on one screen is not possible, and a screen for displaying a list of a plurality of items and a screen for displaying details of the items, respectively. A method of split display on different portable terminals is disclosed.

JP 2003-61140 A

  However, in the cooperative display method described in Patent Document 1, display content is transmitted from one mobile terminal to another mobile terminal. That is, in one portable terminal, data to be displayed in cooperation is acquired from the Internet or the like, and image information of a screen displayed on the other portable terminal is transmitted to the other portable terminal. For this reason, in the above-described cooperative display method, the amount of data acquired by one portable terminal and the amount of data transmitted / received between portable terminals increase, and screen display cannot be performed quickly.

  Accordingly, in order to solve such problems, the present invention provides a terminal device, a cooperative display system, and a cooperative display that can shorten the time required for cooperative display when information is displayed in cooperation between a plurality of terminal devices. It aims to provide a method.

In order to solve the above-described problem, the terminal device of the present invention includes a display unit that displays information in cooperation with another terminal device, and a linkage that determines whether or not a cooperation application is installed in the other terminal device. When it is determined by the application presence / absence determination unit and the cooperative application presence / absence determination unit that the application is not installed on the other terminal device, link information including image information of a screen to be displayed in cooperation with the other terminal device is generated , Cooperation that generates cooperation information including a command for generating a screen to be displayed in cooperation with another terminal device when it is determined by the cooperation application presence / absence determination means that the application for cooperation is installed on the other terminal device An information generation unit; and a transmission unit that transmits the cooperation information to another terminal device.

The cooperative display system of the present invention is a cooperative display system including one terminal device that displays information in cooperation with each other and another terminal device. The one terminal device inquires of the other terminal device whether or not the first display means for displaying information in cooperation with the other terminal device and the application for cooperation is installed in the other terminal device. Based on the response from the terminal device, the application is installed in the other terminal device by the cooperation application presence / absence determination unit that determines whether or not the application for cooperation is installed in the other terminal device. When it is determined that the application information is not included, the link information including the image information of the screen to be displayed in cooperation with another terminal device is generated , and it is determined that the application is installed on the other terminal device by the cooperation application presence / absence determination unit. Command to generate a screen to be displayed in cooperation with other terminal devices. It comprises a linkage information generating means for generating link information, and transmission means for transmitting link information to another terminal device. The other terminal device manages the application installed in the other terminal device, and responds whether or not the application is installed in the other terminal device in response to an inquiry from the cooperative application presence / absence determining unit. Management means and second display means for displaying a screen based on the linkage information.

The cooperative display method of the present invention is a cooperative display method in which a plurality of terminal devices display information in cooperation. This linked display method is a linked application presence / absence determining step for determining whether or not a linked application is installed in another terminal device of the plurality of terminal devices in one terminal device of the plurality of terminal devices. If the application is determined not to be installed on the other terminal device in the cooperative application presence / absence determination step, cooperation information including image information of a screen to be displayed in cooperation with another terminal device is generated, and the cooperative application presence / absence determination is performed. A link information generation step for generating link information including a command for generating a screen to be displayed in link with another terminal device when it is determined in the step that an application is installed in the other terminal device; A transmission step for transmitting the message to another terminal device, and .

  According to the present invention, the terminal device transmits cooperation information including image information of a screen to be displayed in cooperation with another terminal device when the application for cooperation is not installed in the other terminal device. For this reason, when the application for cooperation is installed in the other terminal device, it is not necessary to transmit the image information of the screen to be displayed in cooperation to the other terminal device, so the amount of data transmitted / received between the terminal devices Can be reduced. In addition, since the terminal device does not need to generate image information of a screen to be displayed in cooperation, the processing of the terminal device can be reduced. As a result, it is possible to reduce the time required for cooperative display.

In addition, when an application for cooperation is installed in another terminal device, the cooperation information including a command for generating a screen to be displayed in cooperation with the other terminal device is transmitted to the other terminal device. Screen image information can be generated. For this reason, it is not necessary for the terminal device to generate image information of a screen to be displayed in cooperation, and the processing of the terminal device can be reduced. As a result, it is possible to reduce the time required for cooperative display.

  The terminal device further includes an input unit that receives a user operation, and a cooperative operation determination unit that determines whether the operation received by the input unit is an operation that requires cooperative display with another terminal device. May be. In this case, the cooperative application presence / absence determining unit may determine whether or not a cooperative application is installed in another terminal device when the cooperative operation determining unit determines that cooperative display is necessary. According to this configuration, when an operation that requires cooperation display is performed, it is determined whether or not a cooperation application is installed in another terminal device, and cooperation information is generated and transmitted. For this reason, when cooperation display is unnecessary, the terminal device does not need to generate cooperation information, and the processing of the terminal device can be further reduced.

  The cooperation operation determination unit may determine that the operation received by the input unit is an operation that requires a cooperative display with another terminal device when the operation is a screen transition operation. In this case, the cooperation information generation unit includes a command for causing the other terminal device to generate a transition destination screen when the cooperation application presence / absence determination unit determines that the application is installed in the other terminal device. Cooperation information may be generated, and cooperation information including image information of a transition destination screen may be generated when it is determined that the application is not installed in another terminal device by the cooperation application presence / absence determination unit. According to this configuration, in the terminal device and other terminal devices, the transition source screen and the transition destination screen can be displayed in a coordinated manner, and the time required for the cooperative display of the transition source screen and the transition destination screen can be reduced. It can be shortened.

  The cooperation operation determination unit may determine that the operation received by the input unit during the activation of the application is an operation that activates another application and requires an operation to display in cooperation with another terminal device. Good. In this case, the cooperative application presence / absence determining unit may determine whether another application is installed in another terminal device. Furthermore, the cooperation information generation means includes a command for starting another application on the other terminal apparatus when it is determined by the cooperation application presence / absence determination means that the other application is installed on the other terminal apparatus. When the link information is generated and the link application presence / absence determining unit determines that no other application is installed on the other terminal device, the link information including the image information of the screen on which the other application is started is generated. Also good. According to this configuration, it is possible to display the screen of the active application and the screen of the other application in cooperation with each other in the terminal device and the other terminal device, and it is possible to reduce the time required for this cooperation display. .

  The cooperative application presence / absence determining means may determine whether or not a cooperative application is installed in another terminal device by transmitting a command for generating a screen to be displayed in cooperation to the other terminal device. . In this case, it is determined whether or not a cooperation application is installed in another terminal device by transmitting a command for generating a screen to be displayed in cooperation to the other terminal device. There is no need to make a separate inquiry as to whether or not a linkage application is installed. For this reason, the data amount transmitted / received between terminal devices can be reduced. In addition, when a cooperation application is installed in another terminal device, the other terminal device can display a screen to be displayed in cooperation without receiving the cooperation information. As a result, it is possible to further reduce the time required for cooperative display.

  ADVANTAGE OF THE INVENTION According to this invention, when displaying information in cooperation between several terminal devices, the time which a cooperation display requires can be shortened.

It is a figure which shows the function structure of the cooperation display system which concerns on embodiment of this invention. It is a figure which shows the hardware constitutions of the 1st terminal device of FIG. It is a figure which shows an example of a cooperation display necessity table. It is a figure which shows the example of a cooperation display. It is a figure which shows the example of cooperation information. It is a sequence diagram which shows an example of the cooperation operation | movement of the cooperation display system of FIG. It is a sequence diagram which shows the cooperation display operation | movement of FIG. It is a sequence diagram which shows the other example of cooperation operation | movement of the cooperation display system of FIG. It is a sequence diagram which shows the cooperation display operation | movement of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numerals, and redundant description will be omitted.

  FIG. 1 is a diagram schematically illustrating a functional configuration of the cooperative display system according to the present embodiment. As shown in FIG. 1, the cooperative display system 10 includes a first terminal device 1 (terminal device, one terminal device) and a second terminal device 2 (other terminal device). This is a system in which the first terminal device 1 and the second terminal device 2 display information in cooperation.

  The first terminal device 1 is a terminal device that performs cooperative display together with the second terminal device 2, and is a main terminal that leads cooperative display processing. The first terminal device 1 is, for example, a device such as a mobile phone, a smartphone, a PDA (Personal Digital Assistants), and a notebook PC. The first terminal device 1 is configured to be communicable with the second terminal device 2 and is connected to the network NW. Note that communication with the second terminal device 2 may be performed by either wired or wireless communication, for example, by a wireless local area network (LAN), Bluetooth, a dedicated cable, or the like.

  FIG. 2 is a diagram illustrating a hardware configuration of the first terminal device 1. As shown in FIG. 2, the first terminal device 1 physically includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102 as a main storage device, a ROM (Read Only Memory) 103, and a hard disk. Hardware such as an auxiliary storage device 104 such as a communication module 105 that is a data transmission / reception device such as a network card, an input device 106 that is an input device such as a touch panel and a keyboard, and a display device 107 that is an output device such as a liquid crystal display and a touch panel It is comprised by. Each function of the first terminal device 1 to be described later operates the input device 106 and the display device 107 under the control of the CPU 101 by reading predetermined computer software on the hardware such as the CPU 101 and the RAM 102. This is realized by reading and writing data in the RAM 102 and the auxiliary storage device 104.

  The second terminal device 2 is a terminal device that performs cooperative display together with the first terminal device 1, and is a sub-terminal that performs cooperative display processing based on an instruction from the first terminal device 1. This 2nd terminal device 2 is apparatuses, such as a mobile phone, a smart phone, PDA, and a notebook PC, for example. The second terminal device 2 is configured to be communicable with the first terminal device 1 and is connected to the network NW. The second terminal device 2 has the same hardware configuration as that of the first terminal device 1, and includes hardware such as a CPU, RAM, ROM, communication module, auxiliary storage device, input device, and display device. Each function of the second terminal device 2 to be described later operates an input device, a display device, and the like under the control of the CPU by reading predetermined computer software on hardware such as the CPU, RAM, and the like. Or by reading and writing data in an auxiliary storage device.

  The network NW may be either wired or wireless, and is a network such as a mobile communication network, the Internet, a wired LAN, or a wireless LAN. Devices such as a web server that holds information to be displayed on the display devices of the first terminal device 1 and the second terminal device 2 are connected to the network NW.

  Next, the functional configuration of the cooperative display system 10 will be described. As shown in FIG. 1, the first terminal device 1 functionally includes an input unit 11 (input unit), an inter-terminal information transmission unit 12 (transmission unit), an inter-terminal information reception unit 13, and an NW. The transmission / reception part 14, the cooperation display control part 15, the application execution part 16, and the display part 17 (a display means, a 1st display means) are provided.

  The input unit 11 functions as an input unit that receives a user operation via the input device 106. The input unit 11 transmits operation information related to the received operation to the cooperative display control unit 15 and the application execution unit 16. The inter-terminal information transmission unit 12 functions as an inter-terminal information transmission unit that transmits various data to the second terminal device 2. The inter-terminal information receiving unit 13 functions as an inter-terminal information receiving unit that receives various data from the second terminal device 2. The NW transmission / reception unit 14 functions as an NW transmission / reception unit that transmits various data to the network NW and receives various data from the network NW.

  The cooperative display control unit 15 functions as a cooperative display control unit that controls cooperative display between the first terminal device 1 and the second terminal device 2. When the cooperation display control unit 15 receives operation information indicating the start of the cooperation mode from the input unit 11, the cooperation display control unit 15 notifies the application execution unit 16 that the operation mode is the cooperation mode. Moreover, the cooperation display control part 15 notifies that the cooperation mode was complete | finished to the application execution part 16, when the operation information which shows completion | finish of cooperation mode is received from the input part 11. FIG. And the cooperation display control part 15 performs control regarding a cooperation display in cooperation mode. The link display control unit 15 includes a link operation determination unit 51 (link operation determination unit), a link application presence / absence determination unit 52 (link application presence / absence determination unit), and a link information generation unit 53 (link information generation unit). I have.

  The cooperative operation determination unit 51 functions as a cooperative operation determination unit that determines whether the operation received by the input unit 11 is an operation that requires cooperative display with the second terminal device 2. The cooperative operation determination unit 51 receives operation information from the input unit 11 via the application execution unit 16 and determines whether or not cooperative display is necessary according to the operation content indicated by the operation information. Specifically, the cooperative operation determination unit 51 determines that cooperative display is necessary when an operation including screen transition is performed in all applications. For example, when one screen of the application is configured with an object called one activity (Activity), the cooperative operation determination unit 51 detects whether or not the operation includes a screen transition by detecting the switching of the activity. judge. When the cooperative operation determination unit 51 determines that the operation includes a screen transition, the cooperative operation determination unit 51 determines that cooperative display is necessary. Moreover, the cooperation operation determination part 51 may be provided with a cooperation display necessity table, for example, and may determine whether it is operation which requires cooperation display based on this cooperation display necessity table.

  FIG. 3 is a diagram illustrating an example of the cooperation display necessity table. As shown in FIG. 3, information indicating the application type, information indicating an application operation, and information indicating whether or not a cooperation display is necessary are stored in association with each other in the cooperation display necessity table. The information indicating the application type is information indicating the type of application being executed in the first terminal device 1. The information indicating the application operation is information indicating an operation in the application being executed. The information indicating whether or not cooperation display is necessary is information indicating whether or not cooperation display with the second terminal device 2 is necessary.

In this example, when an operation including screen transition is performed in all applications, the cooperative operation determination unit 51 determines that cooperative display is necessary. When an operation for selecting one image is performed in the thumbnail list of images in the gallery (image viewer), the cooperative operation determination unit 51 determines that cooperative display is necessary. In addition, when a scroll operation, an enlargement operation, and a reduction operation are performed during browser browsing, the cooperative operation determination unit 51 determines that cooperative display is unnecessary. On the other hand, when an operation for selecting URL information is performed during browser browsing, the cooperative operation determination unit 51 determines that cooperative display is necessary. Also, on the received mail list screen of the mail application (mailer), an operation to select a mail to open the mail text detail screen, and an operation to start the browser by selecting URL information in the mail text detail screen of the mail application Is performed, the cooperative operation determination unit 51 determines that cooperative display is necessary.
In this cooperation display necessity table, necessity of cooperation display is registered in advance for all operations for all applications installed in the first terminal device 1.

  Here, a specific example of cooperative display will be described. FIG. 4 is a diagram illustrating an example of cooperative display. As shown in (a) of FIG. 4, when a mail is selected on the received mail list screen displayed on the first terminal device 1, a mail detail screen of the selected mail is displayed on the second terminal device 2. Is displayed. Further, as shown in FIG. 4B, when URL information in the mail text is selected on the mail detail screen displayed on the first terminal device 1, the link destination web of the selected URL information is selected. A screen of the page is displayed on the second terminal device 2.

  As described above, the cooperative operation determination unit 51 determines that the cooperative display is unnecessary when the operation information indicating the operation in the same screen is received, such as the scroll operation during browsing the browser. Also, the cooperative operation determination unit 51 determines that cooperative display is necessary when it receives operation information indicating an operation that causes a transition to a different screen, such as an operation of opening a detailed screen of the mail text in the mail application. Further, the cooperative operation determination unit 51 determines that the cooperative display is necessary when it receives operation information indicating an operation for starting another application during the activation of the application, such as a URL information selection operation in the mail application. Also, when the cooperative operation determination unit 51 determines that cooperative display is not necessary, the cooperative operation determination unit 51 transmits an operation execution instruction to the application execution unit 16.

  The cooperative application presence / absence determining unit 52 functions as a cooperative application presence / absence determining unit that determines whether or not a cooperative application that is a cooperative application is installed in the second terminal device 2. The cooperative application presence / absence determining unit 52 transmits a cooperative mode start notification to the second terminal device 2 via the inter-terminal information transmitting unit 12 when the cooperative operation determining unit 51 determines that cooperative display is necessary. Then, the cooperative application presence / absence determination unit 52 transmits a cooperative application presence / absence confirmation request to the second terminal device 2 via the inter-terminal information transmission unit 12. This cooperation application presence / absence confirmation request is a request for confirming whether or not the cooperation application is installed.

  In addition, the cooperative application presence / absence determining unit 52 receives a cooperative application presence / absence response from the second terminal device 2 via the inter-terminal information receiving unit 13. This cooperation application presence / absence response is a response indicating whether or not the cooperation application is installed. And the cooperation application presence determination part 52 determines whether the cooperation application is installed in the 2nd terminal device 2 based on a cooperation application presence / absence response.

  Note that the cooperative application is an application for performing cooperative display. For example, when the detailed screen of the mail text is displayed on the second terminal device 2, the cooperation application is a mail application. Moreover, when displaying the screen of the link destination of URL information on the 2nd terminal device 2, a cooperation application is a browser.

  The cooperation information generation unit 53 functions as a cooperation information generation unit that generates cooperation information including a cooperation application command when the cooperation application presence / absence determination unit 52 determines that the cooperation application is installed in the second terminal device 2. To do. Here, the cooperative application command is a command for causing the cooperative application to generate a screen to be cooperatively displayed on the second terminal device 2. In addition, the cooperation information generation unit 53 displays image information of a screen to be displayed in cooperation with the second terminal device 2 when the cooperation application presence / absence determination unit 52 determines that the cooperation application is not installed in the second terminal device 2. Generate linkage information including.

  When it is determined that the cooperative application is not installed in the second terminal device 2, the cooperative information generation unit 53 transmits an operation execution instruction to the application execution unit 16 and causes the application execution unit 16 to execute the operation. . Then, image information on the screen to be displayed in cooperation is acquired from the application execution unit 16. Then, the cooperation information generation unit 53 generates cooperation information including image information. In addition, the cooperation display control unit 15 transmits the cooperation information generated by the cooperation information generation unit 53 to the second terminal device 2 via the inter-terminal information transmission unit 12.

  FIG. 5 is a diagram illustrating an example of cooperation information. As shown in FIG. 5, the cooperation information is information including a command to be displayed in cooperation with the second terminal apparatus 2, information on the cooperation application type, and parameters of the command. This cooperation information is determined based on whether or not the cooperation application is installed in the second terminal device 2 and the operation content.

  The cooperation information generation unit 53 determines a command based on the cooperation application presence / absence response received from the second terminal device 2 and the operation information received from the application execution unit 16. And the cooperation information production | generation part 53 produces | generates the cooperation information containing a command, cooperation application classification information, and a parameter. For example, when the cooperation application presence / absence response indicates that the mail application has been installed, and mail is selected from the mail list, the cooperation information generation unit 53 displays the cooperation application type information indicating the mail application and details of the selected mail. The linkage information including the information and the linkage application command for displaying the detailed information of the mail is generated.

  When the cooperation application presence / absence response indicates that the browser has been installed and the URL information is selected, the cooperation information generation unit 53 is indicated by the cooperation application type information indicating the browser, the URL information, and the URL information. Cooperation information including a cooperation application command for causing a web page to be acquired from the web server is generated. On the other hand, when the cooperation application presence / absence response indicates that the cooperation application has not been installed, the cooperation information generation unit 53 generates cooperation information including image information and a command for displaying a screen.

  The application execution unit 16 functions as an application execution unit that executes an application. When the operation information received from the input unit 11 indicates the activation of the application, the application execution unit 16 activates the application. Then, the application execution unit 16 generates image information for displaying the activated application, and transmits the image information to the display unit 17. Further, when not in the cooperation mode, the application execution unit 16 executes the operation indicated by the operation information received from the input unit 11 using the running application.

  In addition, the application execution unit 16 transmits the operation information received from the input unit 11 to the cooperation display control unit 15 during the cooperation mode. When the application execution unit 16 receives an operation execution instruction from the cooperation operation determination unit 51 and the cooperation information generation unit 53, the application execution unit 16 executes the operation. When receiving an operation execution instruction from the cooperative operation determination unit 51, the application execution unit 16 generates image information of a screen to be displayed as a result of executing an operation based on the operation execution instruction. Then, the application execution unit 16 transmits the generated image information to the display unit 17.

  On the other hand, when the application execution unit 16 receives an operation execution instruction from the cooperation information generation unit 53, the application execution unit 16 executes an operation based on the operation execution instruction and generates a screen to be displayed in cooperation with the second terminal device 2. Then, the application execution unit 16 captures the screen to acquire image information, and transmits the acquired image information to the cooperation information generation unit 53 of the cooperation display control unit 15.

  For example, in the received mail list screen of the mail application, when receiving an operation execution instruction of an operation for selecting a mail and opening a detailed screen of the mail text, the application execution unit 16 executes the operation using the mail application, The mail text data is acquired from the storage means in the first terminal device 1, and a screen for displaying the mail text is generated. Then, the application execution unit 16 captures the screen and acquires image information.

  When receiving an operation execution instruction for selecting the URL information in the mail text and starting the browser, the application execution unit 16 starts the browser and based on the selected URL information, the NW transmission / reception unit 14 to access a web server on the network NW. And the application execution part 16 acquires the data regarding the link destination web page of URL information from a web server, and produces | generates the screen which displays a link destination web page using a browser. Then, the application execution unit 16 captures the screen and acquires image information.

  The display unit 17 functions as a display unit that displays information. The display unit 17 displays a screen on the display device 107 based on the image information received from the application execution unit 16.

  Functionally, the 2nd terminal device 2 is the input part 21, the inter-terminal information transmission part 22, the inter-terminal information reception part 23, the NW transmission / reception part 24, the cooperation display control part 25, and the application execution part 26. And a display unit 27 (second display unit) and an application management unit 28 (application management unit).

  The input unit 21 functions as an input unit that receives a user operation via the input device. The input unit 21 transmits operation information regarding the accepted operation to the cooperative display control unit 25 and the application execution unit 26. The inter-terminal information transmission unit 22 functions as an inter-terminal information transmission unit that transmits various data to the first terminal device 1. The inter-terminal information receiving unit 23 functions as an inter-terminal information receiving unit that receives various data from the first terminal device 1. The NW transmission / reception unit 24 functions as NW transmission / reception means for transmitting various data to the network NW and receiving various data from the network NW.

  The cooperative display control unit 25 functions as a cooperative display control unit that controls cooperative display between the first terminal device 1 and the second terminal device 2. The cooperation display control unit 25 receives the cooperation application presence / absence confirmation request from the first terminal device 1 via the inter-terminal information reception unit 23. And the cooperation display control part 25 inquires whether the cooperation application is installed in the application management part 28, and transmits the result to the 1st terminal device 1 via the inter-terminal information transmission part 22 as a cooperation application presence / absence response. To do. In addition, the cooperation display control unit 25 receives cooperation information from the first terminal device 1 via the inter-terminal information reception unit 23. Then, the cooperation display control unit 25 transmits the cooperation information to the application execution unit 26.

  The application execution unit 26 functions as an application execution unit that executes an application. The application execution unit 26 receives cooperation information from the cooperation display control unit 25. In addition, when a command for displaying a screen is included in the cooperation information, the application execution unit 26 transmits image information that is a parameter to the display unit 27.

  On the other hand, when the cooperation application command is included in the cooperation information, the application execution unit 26 activates the cooperation application indicated by the cooperation application type information included in the cooperation information. Then, the application execution unit 26 executes the cooperative application command included in the cooperative information using the parameters included in the cooperative information in the cooperative application, and generates screen image information. Then, the application execution unit 26 transmits the generated image information to the display unit 27.

  For example, when receiving cooperation information including cooperation application type information indicating a mail application, mail detailed information, and a cooperation application command for displaying mail detailed information, the application execution unit 26 activates the mail application. Then, image information for displaying the detailed information of the mail is generated using the mail application.

  In addition, when receiving cooperative information including cooperative application type information indicating a browser, URL information, and a cooperative application command for acquiring a web page indicated by the URL information, the application execution unit 26 activates the browser. To do. And the application execution part 26 accesses the web server on the network NW via the NW transmission / reception part 24 based on URL information. And the application execution part 26 acquires the data regarding the link destination web page of URL information from a web server, and produces | generates the image information for displaying a link destination web page by a browser.

  The display unit 27 functions as a display unit that displays information. The display unit 27 displays a screen on the display device based on the image information received from the application execution unit 26.

  The application management unit 28 functions as an application management unit that manages applications installed in the second terminal device 2. When the application management unit 28 receives an inquiry from the cooperation display control unit 25 as to whether or not the cooperation application is installed, it makes a response as to whether or not the cooperation application is installed.

  Then, the cooperation display method in the cooperation display system 10 which has the structure mentioned above is demonstrated. 6 and 7 are sequence diagrams illustrating an example of the cooperative display operation of the cooperative display system 10. As shown in FIG. 6, first, in the first terminal device 1, when the input unit 11 receives an operation for starting the cooperation mode, the cooperation display operation is started (step S <b> 01). At this time, the input unit 11 transmits operation information indicating the start of the cooperation mode to the cooperation display control unit 15. Then, the cooperation display control unit 15 notifies the application execution unit 16 that it is in the cooperation mode.

  Next, when receiving an operation for starting the application AP1, the input unit 11 transmits the operation information to the application execution unit 16. Then, the application execution unit 16 activates the application AP1 (step S02). Subsequently, the application execution unit 16 generates image information for displaying the application AP 1 and transmits the image information to the display unit 17. And the display part 17 displays the screen of application AP1 on the display apparatus 107 based on the image information received from the application execution part 16 (step S03).

  Thereafter, when receiving an operation of the application AP1, the input unit 11 transmits the operation information to the application execution unit 16 (step S04). Then, the application execution unit 16 transmits the operation information to the cooperative display control unit 15 without executing the operation indicated by the operation information. Then, the cooperative operation determination unit 51 of the cooperative display control unit 15 receives the operation information from the application execution unit 16, and determines whether or not cooperative display is necessary according to the operation content indicated by the received operation information (step S05). ).

  In the determination of step S05, when the operation information indicates an operation within the same screen, such as a scroll operation during browser browsing, the cooperative operation determination unit 51 determines that the cooperative display is unnecessary (step S05; No). Then, the cooperative operation determination unit 51 transmits an operation execution instruction to the application execution unit 16. Upon receiving the operation execution instruction from the cooperative operation determination unit 51, the application execution unit 16 executes the operation indicated by the operation information (Step S06). Then, the application execution unit 16 generates image information of the screen of the application AP1 after the operation execution, and transmits the image information to the display unit 17. Then, returning to step S03, the display unit 17 displays the screen of the application AP1 on the display device 107.

  On the other hand, in the determination of step S05, when the operation information indicates an operation that causes a transition to a different screen, such as an operation for opening a detailed screen of the mail text in the mail application, the cooperative operation determination unit 51 needs to display the cooperation. (Step S05; Yes). In the determination of step S05, when the operation information indicates an operation for starting another application such as an operation for selecting URL information in the mail application, the cooperative operation determination unit 51 determines that cooperative display is necessary ( Step S05; Yes). At this time, the cooperation application presence / absence determination unit 52 transmits a cooperation mode start notification to the second terminal device 2 via the inter-terminal information transmission unit 12. Then, the cooperative application presence / absence determining unit 52 transmits a cooperative application presence / absence confirmation request to the second terminal device 2 via the inter-terminal information transmitting unit 12 (step S07).

  The cooperation display control unit 25 of the second terminal device 2 receives the cooperation application presence / absence confirmation request via the inter-terminal information reception unit 23. And the cooperation display control part 25 inquires of the application management part 28 whether the cooperation application is installed in the 2nd terminal device 2 (step S08). And the cooperation display control part 25 transmits the result to the 1st terminal device 1 via the inter-terminal information transmission part 22 as a cooperation application presence / absence response (step S09). And the cooperation application presence determination part 52 receives a cooperation application presence / absence response from the 2nd terminal device 2 via the inter-terminal information reception part 13, and performs cooperation display operation | movement (step S10).

  FIG. 7 is a sequence diagram showing details of the cooperative display operation. As illustrated in FIG. 7, the cooperative application presence / absence determination unit 52 determines whether or not a cooperative application is installed in the second terminal device 2 based on the cooperative application presence / absence response received from the second terminal device 2. (Step S11, cooperative application presence / absence determination step). And when it determines with the cooperation application being installed in the 2nd terminal device 2 (step S11; Yes), the cooperation information generation part 53 produces | generates a cooperation application command, and produces | generates the cooperation information containing a cooperation application command. (Step S12, linkage information generation step).

  On the other hand, when it is determined in step S11 that the cooperative application is not installed in the second terminal device 2 (step S11; No), the cooperative information generation unit 53 transmits an operation execution instruction to the application execution unit 16. To do. And the application execution part 16 performs the said operation, and acquires data required for the display of the screen displayed on the 2nd terminal device 2 in cooperation (step S13). Next, the application execution part 16 produces | generates the screen displayed on the 2nd terminal device 2 in cooperation, captures the screen, and acquires the image information of a screen. Then, the application execution unit 16 transmits the acquired image information to the cooperation information generation unit 53 of the cooperation display control unit 15. The cooperation information generation unit 53 generates cooperation information including a command for displaying a screen and the image information received from the application execution unit 16 (step S14, cooperation information generation step).

  And the cooperation display control part 15 transmits the cooperation information produced | generated by the cooperation information production | generation part 53 in step S12 or step S14 to the 2nd terminal device 2 via the inter-terminal information transmission part 22 (step S15, transmission). Step). Moreover, the display part 17 displays the screen of application AP1 before operation as it is (step S16).

  Next, the cooperation display control unit 25 of the second terminal device 2 receives the cooperation information from the first terminal device 1 via the inter-terminal information reception unit 23 and transmits the cooperation information to the application execution unit 26. And the application execution part 26 determines whether a cooperation application command is contained in cooperation information (step S17). When it is determined that the cooperation application command is included in the cooperation information (step S17; Yes), the application execution unit 26 activates the cooperation application indicated by the cooperation application type included in the cooperation information.

  Then, the application execution unit 26 executes the cooperation application command included in the cooperation information by using the parameter included in the cooperation information in the cooperation application, and acquires data necessary for displaying the screen to be displayed in cooperation (Step S18). ). Then, the application execution unit 26 generates image information of a screen to be displayed in cooperation (step S19) and transmits it to the display unit 27. Then, the display unit 27 displays a screen on the display device based on the image information received from the application execution unit 26 (step S20). Then, the process returns to step S04.

  On the other hand, when it is determined in step S17 that the cooperative application command is not included in the cooperative information, that is, when it is determined that the command for displaying the screen is included (step S17; No). The application execution unit 26 transmits the image information included in the cooperation information to the display unit 27. Then, the display unit 27 displays a screen on the display device based on the image information received from the application execution unit 26 (step S20). Then, the process returns to step S04.

  8 and 9 are sequence diagrams illustrating another example of the cooperative display operation of the cooperative display system 10. As shown in FIG. 8, first, in the first terminal device 1, when the input unit 11 receives an operation for starting the cooperation mode, the cooperation display operation is started (step S <b> 31). Steps S31 to S36 are the same as steps S01 to S06 in FIG.

  If it is determined in step S35 that cooperative display is necessary (step S35; Yes), the cooperative application presence / absence determining unit 52 notifies the second terminal device 2 of the cooperative mode start via the inter-terminal information transmitting unit 12. Send. Further, the cooperation information generation unit 53 generates a cooperation application command. And the cooperation application presence determination part 52 transmits a cooperation application command with a cooperation application classification and various parameters to the 2nd terminal device 2 via the inter-terminal information transmission part 12 (step S37).

  The cooperative display control unit 25 of the second terminal device 2 receives the cooperative application command, the cooperative application type, and various parameters via the inter-terminal information receiving unit 23. Then, the cooperative display control unit 25 causes the application execution unit 26 to execute the received cooperative application command based on the cooperative application type and various parameters, and determines whether or not the cooperative application command can be executed (step S38). And the cooperation display control part 25 transmits the determination result to the 1st terminal device 1 via the inter-terminal information transmission part 22 as an execution availability response (step S39). Then, the cooperative application presence / absence determining unit 52 receives an execution availability response from the second terminal device 2 via the inter-terminal information receiving unit 13 and performs a cooperative display operation (step S40).

  FIG. 9 is a sequence diagram showing details of the cooperative display operation. As illustrated in FIG. 9, the cooperative application presence / absence determining unit 52 determines whether or not the cooperative application is installed in the second terminal apparatus 2 based on the execution availability response received from the second terminal apparatus 2 ( Step S41, cooperative application presence / absence determination step). If it is determined in step S41 that the cooperative application is not installed in the second terminal device 2 (step S41; No), the cooperative information generation unit 53 transmits an operation execution instruction to the application execution unit 16. To do. And the application execution part 16 performs the said operation, and acquires data required for the display of the screen displayed on the 2nd terminal device 2 in cooperation (step S42).

  Next, the application execution part 16 produces | generates the screen displayed on the 2nd terminal device 2 in cooperation, captures the screen, and acquires the image information of a screen. Then, the application execution unit 16 transmits the acquired image information to the cooperation information generation unit 53 of the cooperation display control unit 15. The cooperation information generation unit 53 generates cooperation information including a command for displaying a screen and the image information received from the application execution unit 16 (step S43, cooperation information generation step).

  And the cooperation display control part 15 transmits the cooperation information produced | generated by the cooperation information production | generation part 53 in step S43 to the 2nd terminal device 2 via the terminal information transmission part 22 (step S44, transmission step). Moreover, the display part 17 displays the screen of application AP1 before operation as it is (step S45).

  On the other hand, when it is determined that the cooperation application is installed in the second terminal device 2 (step S41; Yes), the generation and transmission of the cooperation information are not performed. And the display part 17 displays the screen of application AP1 before operation as it is (step S45).

  Next, in the second terminal device 2, when it is determined in step S38 that the cooperation application command can be executed (step S46; Yes), the application execution unit 26 is based on the execution result of the cooperation application command in step S38. Thus, data necessary for displaying the screen to be displayed in cooperation is acquired (step S47). Then, the application execution unit 26 generates image information of a screen to be displayed in cooperation (Step S48), and transmits the image information to the display unit 27. And the display part 27 displays a screen on a display apparatus based on the image information received from the application execution part 26 (step S49). Then, the process returns to step S34.

  On the other hand, when it is determined in step S38 that the cooperation application command cannot be executed (step S46; No), the cooperation display control unit 25 receives the cooperation information from the first terminal device 1 via the inter-terminal information reception unit 23. The link information is received and transmitted to the application execution unit 26. Since the link information includes a command for displaying a screen and image information, the application execution unit 26 transmits the image information included in the link information to the display unit 27. And the display part 27 displays a screen on a display apparatus based on the image information received from the application execution part 26 (step S49). Then, the process returns to step S34.

  Next, the effect of the 1st terminal device 1 is demonstrated. The first terminal device 1 includes a display unit 17 that displays information in cooperation with the second terminal device 2, a cooperative application presence / absence determining unit 52 that determines whether or not a cooperative application is installed in the second terminal device 2, and When it is determined that the cooperative application is not installed in the second terminal device 2, a cooperative information generating unit 53 that generates cooperative information including image information of a screen to be displayed in cooperation with the second terminal device 2, and the cooperative information Is transmitted to the second terminal device 2.

  Thereby, the 1st terminal device 1 transmits the cooperation information containing the image information of the screen displayed on the 2nd terminal device 2, when the cooperation application is not installed in the 2nd terminal device 2. FIG. For this reason, when the cooperative application is installed in the second terminal device 2, it is not necessary to transmit the image information of the screen to be displayed in cooperation to the other terminal devices, so the first terminal device 1 and the second terminal device 2 can be reduced. In addition, since the first terminal device 1 does not need to generate image information of a screen to be displayed in cooperation, the processing of the first terminal device 1 can be reduced. As a result, it is possible to reduce the time required for cooperative display.

  The cooperation information generation unit 53 is a command for generating a screen to be displayed in cooperation with the second terminal device 2 when the cooperation application presence / absence determination unit 52 determines that the cooperation application is installed in the second terminal device 2. Linkage information including may be generated. When a cooperative application is installed in the second terminal device 2, the screen is displayed on the second terminal device 2 by transmitting cooperative information including a command for generating a screen to be displayed in cooperation to the second terminal device 2. Image information can be generated. For this reason, the 1st terminal device 1 does not need to produce | generate the image information of the screen displayed in cooperation, and can reduce the process of the 1st terminal device 1. FIG. As a result, it is possible to reduce the time required for cooperative display.

  In addition, the first terminal device 1 has an input unit 11 that receives a user operation, and a cooperative operation determination that determines whether the operation received by the input unit 11 is an operation that requires cooperative display with the second terminal device 2. The unit 51 may be further provided. In this case, the cooperative application presence / absence determining unit 52 may determine whether the cooperative application is installed in the second terminal device 2 when the cooperative operation determining unit 51 determines that cooperative display is necessary. According to this configuration, when the cooperative display is unnecessary, the first terminal device 1 does not need to generate the cooperative information, and the processing of the first terminal device 1 can be further reduced.

  The cooperation operation determination unit 51 may determine that the operation received by the input unit 11 is an operation that requires the cooperation display with the second terminal device 2 when the operation is a screen transition operation. In this case, the cooperation information generation unit 53 includes a cooperation application command for causing the second terminal apparatus 2 to generate a transition destination screen when it is determined that the cooperation application is installed in the second terminal apparatus 2. When the cooperation information is generated and it is determined that the cooperation application is not installed in the second terminal device 2, the cooperation information including the image information of the transition destination screen may be generated. According to this configuration, the first terminal device 1 and the second terminal device 2 can display the transition source screen and the transition destination screen in a coordinated manner, and the cooperation between the transition source screen and the transition destination screen. The time required for display can be shortened.

  The cooperation operation determination unit 51 determines that the operation received by the input unit 11 during the activation of the application is an operation that activates another application and requires an operation to display in cooperation with the second terminal device 2. May be. In this case, the cooperative application presence / absence determination unit 52 may determine whether another application is installed in the second terminal device 2. Furthermore, when it is determined that the other application is installed in the second terminal device 2, the cooperation information generation unit 53 sends a cooperation application command for causing the second terminal device 2 to start the other application. The link information including the image information of the screen on which the other application is started may be generated when it is determined that the other application is not installed in the second terminal device 2 . According to this configuration, the first terminal device 1 and the second terminal device 2 can display the screen of the active application and the screen of another application in a coordinated manner, thereby reducing the time required for the coordinated display. It becomes possible.

  Even if the cooperation application presence determination part 52 determines whether the cooperation application is installed in the 2nd terminal device 2 by transmitting the command for generating the screen to display in cooperation to the 2nd terminal device 2. Good. In this case, it is determined whether or not a cooperative application is installed in the second terminal device 2 by transmitting a command for generating a screen to be displayed in cooperation to the second terminal device 2. There is no need to make a separate inquiry as to whether a cooperative application is installed in For this reason, the data amount transmitted / received between the 1st terminal device 1 and the 2nd terminal device 2 can be reduced. Moreover, when the cooperation application is installed in the 2nd terminal device 2, the 2nd terminal device 2 can display the screen displayed on a cooperation display, without receiving cooperation information. As a result, it is possible to further reduce the time required for cooperative display.

  The terminal device, the information display system, and the information display method according to the present invention are not limited to those described in this embodiment. For example, the first terminal device 1 may further include an application management unit that manages applications installed in the first terminal device 1. The cooperative display control unit 25 may include functional units similar to the cooperative operation determination unit 51, the cooperative application presence / absence determination unit 52, and the cooperation information generation unit 53. In this case, either the first terminal device 1 or the second terminal device 2 can be a main terminal, and the other terminal device can be a sub-terminal.

  Further, when the first terminal device 1 and the second terminal device 2 are performing the cooperative display, it is determined whether the operation received by the input unit 21 is an operation that requires the cooperative display, and the cooperative display is performed. When the operation is necessary, it may be displayed in cooperation with another terminal device. Thus, it is also possible to perform cooperative display using a plurality of terminal devices of three or more.

  DESCRIPTION OF SYMBOLS 1 ... 1st terminal device (terminal device, one terminal device), 2 ... 2nd terminal device (other terminal devices), 10 ... Cooperation display system, 11 ... Input part (input means), 12 ... Information transmission between terminals Part (transmission means), 15 ... cooperation display control part, 16 ... application execution part, 17 ... display part (display means, first display means), 21 ... input part, 22 ... information transmission part between terminals, 23 ... between terminals Information receiving unit 25 ... Cooperation display control unit 26 ... Application execution unit 27 ... Display unit (second display unit) 28 ... Application management unit (application management unit) 51 ... Cooperation operation determination unit (cooperation operation determination unit) ), 52... Linked application presence / absence determination unit (linkage application presence / absence determination unit), 53... Linkage information generation unit (linkage information generation unit).

Claims (10)

Display means for displaying information in cooperation with other terminal devices;
Cooperation application presence / absence determining means for determining whether or not an application for cooperation is installed in the other terminal device;
If the application to the other terminal device is determined to be not installed by the cooperative application presence determining means, generates link information including the image information on the screen to link displayed on the other terminal devices, said linkage Cooperation information for generating cooperation information including a command for generating a screen to be displayed in cooperation with the other terminal device when the application presence / absence determination unit determines that the application is installed on the other terminal device. Generating means;
Transmitting means for transmitting the cooperation information to the other terminal device;
A terminal device comprising: Input means for accepting user operations;
Cooperative operation determination means for determining whether the operation received by the input means is an operation that requires cooperative display with other terminal devices;
Further comprising
The cooperative application presence / absence determining means determines whether or not a cooperative application is installed in the other terminal device when the cooperative operation determining means determines that cooperative display is necessary. The terminal device according to claim 1 . The cooperation operation determination unit determines that the operation received by the input unit is an operation that requires a cooperative display with the other terminal device when the operation is a screen transition operation. Item 3. The terminal device according to Item 2 . The cooperation information generation means is a command for causing the other terminal apparatus to generate a transition destination screen when the cooperation application presence / absence determination means determines that the application is installed in the other terminal apparatus. Is generated, and when it is determined that the application is not installed in the other terminal device by the cooperative application presence / absence determining unit, the cooperative information including the image information of the transition destination screen is generated. The terminal device according to claim 3 . The cooperative operation determination unit determines that the operation received by the input unit during the activation of the application is an operation that activates another application and requires an operation that requires cooperative display with the other terminal device. The terminal device according to any one of claims 2 to 4 , wherein: The terminal device according to claim 5 , wherein the cooperation application presence / absence determination unit determines whether or not the other application is installed in the other terminal device. The cooperation information generation unit causes the other terminal device to start the other application when the cooperation application presence / absence determination unit determines that the other application is installed in the other terminal device. Image information of a screen on which the other application is activated when it is determined that the other application is not installed in the other terminal device by the cooperation application existence determination unit. The terminal device according to claim 6 , wherein the cooperation information including the information is generated. The linked application presence / absence determining means transmits whether or not the command for generating the screen to be displayed in cooperation can be executed by transmitting a command for generating the screen to be displayed in cooperation to the other terminal device. The terminal device according to claim 1, wherein a permission response is received from the other terminal device, and it is determined based on the execution permission response whether the application is installed in the other terminal device. . A cooperative display system including one terminal device and other terminal devices that display information in cooperation with each other,
The one terminal device is:
First display means for displaying information in cooperation with the other terminal device;
Inquires of the other terminal device whether or not a cooperation application is installed in the other terminal device, and installs the cooperation application in the other terminal device based on a response from the other terminal device. Linked application presence / absence determining means for determining whether or not
If the application to the other terminal device is determined to be not installed by the cooperative application presence determining means, generates link information including the image information on the screen to link displayed on the other terminal devices, said linkage Cooperation information for generating cooperation information including a command for generating a screen to be displayed in cooperation with the other terminal device when the application presence / absence determination unit determines that the application is installed on the other terminal device. Generating means;
Transmitting means for transmitting the cooperation information to the other terminal device;
With
The other terminal device is:
Application management means for managing an application installed in the other terminal apparatus and responding whether or not the application is installed in the other terminal apparatus in response to an inquiry from the cooperation application presence / absence determination means When,
Second display means for displaying the screen based on the linkage information;
A coordinated display system comprising: A cooperative display method in which a plurality of terminal devices display information in cooperation with each other,
In one terminal device of the plurality of terminal devices,
A cooperative application presence / absence determining step for determining whether a cooperative application is installed in another terminal device among the plurality of terminal devices; and
When it is determined that the application is not installed in the other terminal device in the cooperation application presence / absence determination step, the cooperation information including image information of a screen to be displayed in cooperation with the other terminal device is generated, and the cooperation is generated. Cooperation information for generating cooperation information including a command for generating a screen to be displayed in cooperation with the other terminal device when it is determined in the application presence / absence determination step that the application is installed in the other terminal device Generation step;
A transmission step of transmitting the cooperation information to the other terminal device;
A linked display method characterized by comprising: