JP2016076056A - Information processing system, information processing device, and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make display referable among a plurality of terminal devices by performing synchronization.SOLUTION: An information processing device includes: a reference management part for associating, with each of a plurality of terminal devices, reference information representing displayed reference contents for either first display operable on each terminal device or second display displayed on another terminal device except the terminal device, to store the reference information; a display control part for specifying a second terminal device displaying the operated first display from the reference management part on the basis of operation information representing operated contents that is to be transmitted when an operation is performed to the first display displayed on the first terminal device of the plurality of terminal devices, and preparing second operation information of which destination is the second terminal device; and a transmission part for transmitting the second operation information.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing system.

  A screen that can share a display screen that reflects the input operation with a small amount of data by sharing display content between information processing devices in advance and exchanging only the operation information of the input operation accepted by each information processing device A shared service providing system is known (see, for example, Patent Document 1).

  When a relatively large number of documents are shared by each information processing device by the screen sharing service providing system, and presentations are given by multiple presenters using these multiple documents, followed by a question-and-answer session think of.

  In the screen sharing service providing system of the prior art, it is possible to synchronize the viewing pages of the presenter and other participants or switch the presenter in accordance with an operation such as page turning by the presenter.

  However, if the browsing page is synchronized in the case of a question-and-answer session in which multiple participants speak continuously, the presenter or questioner must display in advance the location to be referred to when speaking. is there. That is, it is not possible to view a part other than the part to be referred to when speaking.

  In addition, when one questioner asks questions in a plurality of places, the proceeding of the proceedings is stagnant while the screen is changed to the question place by the questioner. Further, since the presenter or the questioner refers to the question part, the operation of referring to the question part is complicated.

  Also, for example, if there are multiple questioners for a presenter, even if each questioner points out the question location, it may be difficult to determine which part of which page of which material the question location is. is there.

  Therefore, an object of the present invention is to make it possible to refer to a display in synchronization between a plurality of terminal devices.

An information processing system according to an embodiment of the disclosure includes:
An information processing system having an information processing device and a plurality of terminal devices connected to the information processing device,
The first terminal device of the plurality of terminal devices is
A first display processing unit that performs a process of displaying a first display operable on the first terminal device and a second display displayed on a terminal device other than the first terminal device;
An operation information creation unit that creates first operation information representing the operation content when the first display that has been processed by the first display processing unit is operated;
A first transmission unit for transmitting the first operation information to the information processing apparatus,
The information processing apparatus includes:
For each of the plurality of terminal devices, a reference management unit that associates and stores reference information representing the reference content of the one displayed in the first display or the second display;
Based on the first operation information transmitted by the first terminal device, the reference management unit identifies the second terminal device displaying the operated first display, and A display control unit that creates second operation information destined for the second terminal device, and a second transmission unit that transmits the second operation information,
The second terminal device is:
An operation type determination unit for determining an operation type based on the second operation information;
And a second display processing unit that updates the displayed second display based on the operation type determined by the operation type determination unit.

  According to the disclosed embodiment, it is possible to refer to the display in synchronization between a plurality of terminal devices.

It is a figure which shows one Example of a conference system. It is a hardware block diagram of an example of the computer which concerns on a present Example. It is a functional block diagram which shows one Example of a conference server apparatus. It is a figure which shows an example of a display management table. It is a figure which shows an example of a reference management table. It is a functional block diagram which shows one Example of a participating terminal. It is a sequence chart which shows one Example of operation | movement of a conference system. It is a figure which shows an example of a display of each participating terminal. It is a figure which shows an example of a reference management table. It is a figure which shows an example of a display of each participating terminal. It is a sequence chart which shows one Example of operation | movement of a conference system. It is a figure which shows an example of the operation part of each participating terminal. It is an example image figure showing a mode that a XMPP message is distributed. It is explanatory drawing of an example of a XMPP message. It is a functional block diagram which shows one Example of a conference server apparatus. It is a figure which shows an example of a question list | wrist. It is a figure which shows an example of an answer list. It is a functional block diagram which shows one Example of a participating terminal. It is a sequence chart which shows one Example of operation | movement of a conference system. It is a figure which shows an example of a display of each participating terminal. It is a figure which shows an example of a reference management table.

Next, the form for implementing this invention is demonstrated, referring drawings based on the following Examples. Examples described below are merely examples, and embodiments to which the present invention is applied are not limited to the following examples.
In all the drawings for explaining the embodiments, the same reference numerals are used for those having the same function, and repeated explanation is omitted.

<Example>
In the present embodiment, a conference system as an example of an information processing system will be described. In the conference system according to the present embodiment, the participating terminals acquire conference materials from the conference server device, and share the display screen using the conference materials.
[First Embodiment]
<System configuration>
FIG. 1 is a configuration diagram of an example of a conference system according to the present embodiment. The conference system 1 in FIG. 1 shows an example of a configuration in which a conference server device 10, a participating terminal A11, a participating terminal B12, a participating terminal C13, and a participating terminal D14 are connected to a network N1 such as a LAN. Although four participating terminals are shown in FIG. 1, if the number of participating terminals is two or more, it can be two or three or five or more.

  The conference server device 10 manages conference information and conference materials. In addition, the conference server device 10 holds the image displayed on the participation terminal of a questioner among the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 during the holding and participation control of the conference. Control to share with participating terminals other than the questioner's participating terminal.

  A conference participant starts software (hereinafter referred to as a “screen sharing application”) that realizes a screen sharing function for realizing the provision of the service in each participating terminal A11-participating terminal D14, and displays contents such as presentation materials on the screen. To display. The display content is shared in advance between the participating terminal A11 and the participating terminal D14.

  The participation terminal A11 has a function of registering conference information and conference materials in the conference server device 10. A conference organizer or presenter can register conference information and conference materials in the conference server device 10 from the participation terminal A11. The participation terminal A11 is a device operated by the conference organizer or presenter. The participating terminal A11 can be a conference dedicated terminal in addition to a personal computer (PC), a tablet terminal, a smart phone, a mobile phone, a portable information terminal such as a PDA, a display device such as an electronic whiteboard, a projection device such as a projector. . Further, the participating terminal A11 may be an image forming apparatus such as a printer or a multifunction peripheral, or a projection apparatus such as a projector.

  Upon receiving an input operation such as a page turning operation or a pointer operation by a presenter who makes a presentation, the participating terminal A11 transmits the operation information to the conference server device 10. The conference server device 10 transmits operation information to the participation terminal B12, the participation terminal C13, and the participation terminal D14. As a result, in the participant terminals B12, C13, and D14 of the conference participants B, C, and D, the presenter's input operation is reproduced on the screen based on the received operation information. Thereby, in the conference system 1 according to the present embodiment, the display screen reflecting the input operation is shared among the participating terminals. The participation terminals B12, C13, and D14 are devices that can be operated by the conference organizer and the presenter, similarly to the participation terminal A11. Participating terminals B12, C13, and D14 are PCs (personal computers), tablet terminals, smart phones and mobile phones, PDAs and other portable information terminals, electronic whiteboards and other display devices, projectors and other projection devices, and conference-only terminals It can be. In addition, the participating terminals B12, C13, and D14 can be image forming apparatuses such as printers and multifunction peripherals, and projection apparatuses such as projectors.

  Further, the conference server device 10 may be software or a service that operates on a plurality of computers. Moreover, the conference server apparatus 10 can also take the form of what is called a cloud service, for example. The conference server device 10 may have a conference server function, and the hardware may be a terminal.

  The participating terminals A11-D14 have a function of acquiring conference information and conference materials from the conference server device 10 and participating in the conference. Presenters and participants of the conference can participate in the conference from the participation terminals A11-D14. As described above, the participation terminals A11 to D14 are devices operated by the conference presenter and participants.

  While participating in the conference, the participating terminals A11 to D14 receive input operation information transmitted from the conference server apparatus 10 by the presenting participant's participating terminals in real time, and display screens with the participating terminals of other participants. Share.

<Hardware configuration>
The conference server device 10, the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 are realized by, for example, the computer 100 having a hardware configuration illustrated in FIG. FIG. 2 is a hardware configuration diagram of an example of a computer according to the present embodiment.

  As shown in FIG. 2, the computer 100 includes an input device 21, a display device 22, an external I / F 23, a RAM 24, a ROM 25, a CPU 26, a communication I / F 27, an HDD 28, etc., which are connected to each other via a bus B. Has been.

  The input device 21 includes a keyboard, a mouse, and the like, and is used to input each operation signal to the computer 100. The display device 22 includes a display and the like, and displays a processing result by the computer 100.

  The communication I / F 27 is an interface that connects the computer 100 to the network N1. Accordingly, the computer 100 can perform data communication with another computer 100 via the communication I / F 27.

  The HDD 28 is a nonvolatile storage device that stores programs and data. The stored programs and data include an OS that is basic software for controlling the entire computer 100, and application software that provides various functions on the OS.

  The external I / F 23 is an interface with an external device. The external device includes a recording medium 23a. Thereby, the computer 100 can read and / or write the recording medium 23a via the external I / F 23. Examples of the recording medium 23a include a flexible disk, a CD, a DVD, an SD memory card, and a USB memory.

  The ROM 25 is a nonvolatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 25 stores programs and data such as BIOS, OS settings, and network settings that are executed when the computer 100 is started up. The RAM 24 is a volatile semiconductor memory that temporarily stores programs and data.

  The CPU 26 is an arithmetic device that realizes control and functions of the entire computer 100 by reading a program and data from a storage device such as the ROM 25 and the HDD 28 onto the RAM 24 and executing processing. Note that the computer 100 may have a configuration other than the hardware configuration illustrated in FIG. 2, and may include a camera, a microphone, and a speaker, for example.

  The conference server device 10, the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14 can implement various processes as described later by executing a program on the computer 100 having the above hardware configuration, for example.

<Software configuration>
The conference server device 10, the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 of the conference system 1 according to the present embodiment are realized by the following functional blocks, for example.

<Conference server device>
The conference server device 10 is realized by a functional block diagram as shown in FIG. 3, for example. FIG. 3 is a functional block diagram of an example of the conference server device 10. The conference server device 10 executes the program, thereby transmitting and receiving unit 102, reference location registration unit 104, storage / reading processing unit 106, storage unit 108, operation type determination unit 110, display control unit 112, and display sharing processing unit 114. To realize.

  The transmission / reception unit 102 of the conference server apparatus 10 is realized by the communication I / F 27 illustrated in FIG. 2, and between the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14 via the network N1. Sends and receives various data (information).

  The reference location registration unit 104 of the conference server device 10 is realized by a command from the CPU 26 shown in FIG. 2, and the participant A who uses the participation terminal A11, the participant B who uses the participation terminal B12, and the participation terminal C13. Are registered in the storage unit 108 in association with each other by associating reference information with each participant terminal in accordance with the registration operation of the participant C who uses and the conference participant D who uses the participant terminal D14.

  The storage / reading processing unit 106 of the conference server apparatus 10 is realized by the command from the CPU 26 shown in FIG. 2 and the HDD 28 shown in FIG. 2 as an example. The storage / reading processing unit 106 performs processing of storing various data in the storage unit 108 and reading out various data stored in the storage unit 108.

  The storage unit 108 of the conference server apparatus 10 is realized by the HDD 28 illustrated in FIG. 2 and stores a display management table 1082, a reference management table 1084, a material DB 1086, and the like.

  FIG. 4 shows an example of the display management table 1082. In the display management table 1082, for each participating terminal, display information such as a display name, a material name, a page number, a reference portion, and the like regarding a display that can be operated by the participant (hereinafter referred to as “first display”). Are associated with each other and stored.

  FIG. 5 shows an example of the reference management table 1084. In the reference management table 1084, reference information such as a reference display (hereinafter referred to as “second display”) is associated and stored for each participating terminal. In FIG. 5, reference terminal A11 refers to view A, which is the first display of participating terminal A11, reference terminal B12 refers to view B, which is the first display of participating terminal B12, and reference terminal C13 participates. An example of the reference management table 1084 in the case where the view C that is the first display of the terminal C13 is referenced and the reference terminal D14 is referencing the view D that is the first display of the participating terminal D14 is shown.

  The material DB 1086 stores presentation materials and the like.

  The operation type discriminating unit 110 of the conference server apparatus 10 is realized by an instruction from the CPU 106 shown in FIG. 2, and includes operation information transmitted by the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14. Based on this, in each participating terminal, the type of operation such as page turning or pointer executed for the first display is determined. The operation type determination unit 110 inputs the participating terminal information on which the operation determined based on the operation information is performed and the operation type information to the display control unit 112.

  The display control unit 112 of the conference server apparatus 10 is realized by a command from the CPU 26 shown in FIG. 2, and for the first display of each participating terminal, its display name, material name, page number, and reference part Are stored in the display management table 1082 of the storage unit 108 by associating them with each other.

  Furthermore, the display control unit 112 updates the display management table 1082 of the storage unit 108 according to the operation type information input by the operation type determination unit 110. Then, the display control unit 112 refers to the reference information in the reference management table 1084, determines whether or not the first display view of the reference terminal on which the operation has been performed is stored, and if it is stored. In order to notify the reference terminal referring to the first display view of the reference terminal on which the operation has been performed, the operation destined to the reference terminal referring to the first display view Information is input to the transceiver 102.

  The display sharing processing unit 114 of the conference server apparatus 10 displays the first display or the second display of a participating terminal among the participating terminals A11, B12, C13, and D14 as other participating terminals. In other words, a process of displaying a first display or a second display of a certain participant on another participating terminal is performed.

<Participating terminal>
Participating terminal A11, participating terminal B12, participating terminal C13, and participating terminal D14 are described as “participating terminal 20” unless they need to be distinguished because they have the same configuration.

  The participating terminal 20 is realized by a functional block diagram as shown in FIG. 6, for example. FIG. 6 is a functional block diagram of an example of the participating terminal 20. The participating terminal 20 executes the program, thereby transmitting / receiving unit 202, reference location notification processing unit 204, storage / reading processing unit 206, storage unit 208, operation information creation unit 210, display sharing processing unit 212, and operation type determination unit 214. And the display processing unit 216 are realized.

  The transmission / reception unit 202 of the participating terminal 20 is realized by the communication I / F 27 illustrated in FIG. 2, and transmits / receives various data (information) to / from the conference server device 10 via the network N1.

  The reference location notification processing unit 204 of the participating terminal 20 is realized by a command from the CPU 26 shown in FIG. 2 and is displayed in accordance with the reference location notification operation of the participant who uses the participating terminal 20. A process of notifying the conference server apparatus 10 of reference information such as display or second display is performed.

  The storage / reading processing unit 206 of the participating terminal 20 is realized by the command from the CPU 26 shown in FIG. 2 and the HDD 28 shown in FIG. 2 as an example. The storage / reading processing unit 206 stores various data in the storage unit 208 and reads out various data stored in the storage unit 208.

  The storage unit 208 of the participating terminal 20 is realized by the HDD 28 illustrated in FIG. 2 and stores the material DB 2082 and the like. The material DB 2082 stores presentation materials and the like.

  The operation information creation unit 210 of the participating terminal 20 is realized by a command from the CPU 26 shown in FIG. 2, and the participating terminal 20 performs operations such as page feed and pointer operations executed for the first display. Operation information to be expressed is created and transmitted from the transmission / reception unit 202 to the conference server apparatus 10.

  The display sharing processing unit 212 of the participating terminal 20 is realized by a command from the CPU 26 illustrated in FIG. 2, and the first display or the second display displayed on the participating terminal 20 is transmitted to other participating terminals. Processing is performed by transmitting a display sharing command that is a command to be displayed.

  The operation type discriminating unit 214 of the participating terminal 20 is realized by a command from the CPU 26 shown in FIG. 2, and based on operation information transmitted by the conference server device 10, a type of operation such as a page feed or a pointer. Is determined. The operation type determination unit 214 inputs operation type information determined based on the operation information to the display processing unit 216.

  The display processing unit 216 of the participating terminal 20 is realized by an instruction from the CPU 26 shown in FIG. 2, displays the material stored in the material DB 2082 of the storage unit 208 on the display device 22, and operates the operation type determination unit. The display is updated according to the operation type information input from 214.

<Details of processing>
Below, the detail of the process of the conference system 1 which concerns on this embodiment is demonstrated.
<Operation of the conference system (part 1)>
FIG. 7 shows processing for causing the participating terminal B12 to the participating terminal D14 to display the first display displayed on the participating terminal A11.

  The conference server device 10 and the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 that participate in the conference participate from the presenter's participation terminal A11 via the conference server device 10 through the established communication path. The user's input operations to the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 are distributed in real time to realize screen sharing. An example of such a communication path is a protocol called XMPP.

  XMPP is standardized as RFC 3920 (Extensible Messaging and Presence Protocol: Core) and RFC 3921 (Extensible Messaging and Presence Protocol: Instant Messaging and Presence) in IETF, and the extended specification is documented as XEP.

  XMPP can provide a service called group chat (Multi-User Chat [XEP-0045]). In the group chat, when a participating terminal A11, a participating terminal B12, a participating terminal C13, and a participating terminal D14 that participate in a conference room (room) provided by XMPP send a message to the conference server device 10, the message is a conference. This function is distributed from the server device 10 to all participating terminals participating in the conference room. In addition to XMPP, for example, protocols such as BOSH, WebSocket, and Comet, which are extensions of XMPP, can be used as such communication paths.

  BOSH (Bidirectional-streams Over Synchronous HTTP) [XEP-0206] is a technique for transmitting and receiving messages using HTTP in XMPP. These protocols can realize the Push function from the conference server device 10. The Push function is a function of receiving information from the conference server device 10 asynchronously to the participating terminal and receiving the information at the participating terminal, rather than receiving a request from the participating terminal and sending information. In general HTTP communication, a similar function can also be realized by periodically obtaining information from participating terminals.

  In step S702, connection processing according to a protocol such as XMPP is performed among the conference server device 10, the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  FIG. 8 shows an example of the display device 22 of each participating terminal. The display device 22 includes a display portion 1022 for displaying the first display or the second display being referred to, and a first selection portion for selecting the first display or the second display to be displayed on the display portion 1022. 1024, second selection portion 1026, third selection portion 1028, and fourth selection portion 1030 are formed.

  When the first selection portion 1024 is pushed down, the first selection portion 1024 is highlighted, and the display of the participating terminal A11 is displayed on the display portion 1022. When the second selection portion 1026 is pushed down, the second selection portion 1026 is highlighted, and the display of the participating terminal B12 is displayed on the display portion 1022. When the third selection portion 1028 is pushed down, the third selection portion 1028 is highlighted, and the display of the participating terminal C13 is displayed on the display portion 1022. When the fourth selection portion 1030 is pushed down, the fourth selection portion 1030 is highlighted, and the display of the participating terminal D14 is displayed on the display portion 1022.

  Immediately after the connection process is performed, as shown in FIG. 8, the first display of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal A11, and the display portion 1022 of the participating terminal B12 displays the first display of the participating terminal B12. The first display is displayed, the first display of the participating terminal C13 is displayed on the display portion 1022 of the participating terminal C13, and the first display of the participating terminal D14 is displayed on the display portion 1022 of the participating terminal D14. .

  Returning to FIG.

  In step S704, the conference participant B of the participation terminal B12 performs an operation of switching the display to the view A that is the first display displayed on the participation terminal A11 by operating the input device 21. By this operation, the reference location notification processing unit 204 of the participating terminal B 12 creates reference location information accompanying information for switching to the view A and inputs the reference location information to the transmission / reception unit 202.

  In step S <b> 706, the transmission / reception unit 202 of the participating terminal B <b> 12 transmits reference location information to the conference server device 10.

  In step S708, the conference participant C of the participation terminal C13 performs an operation of switching the display to the view A that is the first display displayed on the participation terminal A11 by operating the input device 21. By this operation, the reference location notification processing unit 204 of the participating terminal C13 creates reference location information attached with information for switching to the view A and inputs the reference location information to the transmission / reception unit 202.

  In step S <b> 710, the transmitting / receiving unit 202 of the participating terminal C <b> 13 transmits reference location information to the conference server device 10.

  In step S712, the conference participant D of the participation terminal D14 performs an operation of switching the display to the view A that is the first display displayed on the participation terminal A11 by operating the input device 21. By this operation, the reference location notification processing unit 204 of the participating terminal D14 creates reference location information accompanied by information for switching to the view A and inputs the reference location information to the transmission / reception unit 202.

  In step S <b> 714, the transmission / reception unit 202 of the participating terminal D <b> 14 transmits reference location information to the conference server device 10.

  In step S716, the reference location registration unit 104 of the conference server apparatus 10 updates the reference management table 1084 of the storage unit 108 based on the reference location information transmitted by the participating terminal B12, the participating terminal C13, and the participating terminal D14. .

  FIG. 9 shows the reference management table 1084 after the update. Reference information of all participating terminals is view A.

  In step S718, the display control unit 112 of the conference server apparatus 10 associates with the changed view to be transmitted to the reference terminal whose reference information has been changed, based on the updated reference management table 1084 and the display management table 1082. Operation information accompanying the displayed display information is created and input to the transmission / reception unit 102.

  In steps S720 to S724, the transmission / reception unit 102 transmits operation information to each of the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  In step S726, the operation type determination unit 214 of the participating terminal B12 determines the type of operation based on the operation information transmitted by the conference server device 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal B12 executes the display according to the operation type information input from the operation type determination unit 214.

  In step S728, the operation type determination unit 214 of the participating terminal C13 determines the type of operation based on the operation information transmitted by the conference server apparatus 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal C13 updates the display according to the operation type information input from the operation type determination unit 214.

  In step S730, the operation type determination unit 214 of the participating terminal D14 determines the operation type based on the operation information transmitted by the conference server apparatus 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal D14 updates the display according to the operation type information input from the operation type determination unit 214.

  Immediately after step S730, as shown in FIG. 10, the first display of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal A11, and the view A of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal B12. The view A of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal C13, and the view A of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal D14.

  In step S732, an operation is performed on the first display displayed on the participating terminal A11.

  In step S734, the operation information creation unit 210 of the participating terminal A11 creates operation information indicating the operation content executed on the first display in step S732, and transmits the operation information from the transmission / reception unit 202.

  In step S736, the operation type determination unit 110 of the conference server apparatus 10 determines the operation type represented by the operation information based on the operation information transmitted by the participating terminal A11 and inputs the operation type to the display control unit 112. The display control unit 112 updates the display management table 1082 of the storage unit 108 based on the operation content determined by the operation type determination unit 110.

  In step S738, the display control unit 112 of the conference server apparatus 10 refers to the reference management table 1084 in the storage unit 108, and identifies the participating terminal whose reference information is the view A of the participating terminal A11. Here, as shown in FIG. 9, since the reference information of all participating terminals is view A, participating terminal B12, participating terminal C13, and participating terminal D14 are selected. Then, the display control unit 112 creates operation information representing the operation content destined for the selected participating terminal, and inputs the operation information to the transmission / reception unit 102.

  In steps S740 to S744, the transmission / reception unit 102 transmits operation information to the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  In step S746, the operation type determination unit 214 of the participating terminal B12 determines the type of operation based on the operation information transmitted by the conference server apparatus 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal B12 updates the display according to the operation type information input from the operation type determination unit 214.

  In step S748, the operation type determination unit 214 of the participating terminal C13 determines the type of operation based on the operation information transmitted by the conference server device 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal C13 updates the display according to the operation type information input from the operation type determination unit 214.

  In step S750, the operation type determination unit 214 of the participating terminal D14 determines the type of operation based on the operation information transmitted by the conference server apparatus 10, and inputs the operation type information to the display processing unit 216. The display processing unit 216 of the participating terminal D14 updates the display according to the operation type information input from the operation type determination unit 214. That is, the part referred to by the questioner is displayed on the participation terminals of all the participants.

<Operation of the conference system (part 2)>
FIG. 11 shows a process of sharing the display between the participating terminal A11 and the participating terminal D14. This process can be applied to a case where the part referred to by the questioner is shared by all the participants during the question-and-answer session.

  In step S1102, connection processing is performed among the conference server device 10, the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  In step S <b> 1104, the display sharing processing unit 212 creates a display sharing command and inputs the display sharing command to the transmission / reception unit 202 by performing a display sharing operation on the participating terminal A <b> 11.

  In step S <b> 1106, the transmission / reception unit 202 of the participating terminal A <b> 11 transmits a display sharing command to the conference server device 10.

  In step S1108, the display sharing processing unit 114 of the conference server apparatus 10 displays the reference information of each participating terminal stored in the reference management table 1084 of the storage unit 108 based on the display sharing command transmitted by the participating terminal A11. Change to view A. Further, the display sharing processing unit 114 creates display sharing information associated with display information associated with the view A from the display management table 1082 and inputs the display sharing information to the transmission / reception unit 102.

  In steps S1110 to S1114, the transmission / reception unit 102 of the conference server apparatus 10 transmits the display sharing information to the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  In step S <b> 1116, the display sharing processing unit 212 of the participating terminal B <b> 12 switches the display to the view A based on the display sharing information transmitted by the conference server device 10.

  In step S1118, the display sharing processing unit 212 of the participating terminal C13 switches the display to the view A based on the display sharing information transmitted by the conference server device 10.

  In step S1120, the display sharing processing unit 212 of the participating terminal D14 switches the display to the view A based on the display sharing information transmitted by the conference server device 10.

<Operation part of participating terminals>
FIG. 12 shows an example of the operation part of each participating terminal, in which an operation part is added to the example of the display device 22 shown in FIG.

  The operation part includes an exit button 501, a display page selection field 502, a figure selection field 503, a handwritten memo selection field 504, a question button 506, and an answer button 507.

  The exit button 501 is pushed down when the conference is finished and the conference room is exited. The user can exit the conference room by pressing the exit button 501 while participating in the conference room. When the exit button 501 is pressed, the operation information creation unit 210 transmits an XMPP message for exiting the conference room in which the user is participating to the conference server device 10. The XMPP message leaving the conference room is a standardized specification.

  The display page selection field 502 is pushed down when selecting a page to be displayed for the conference material. For example, when the right arrow button in the display page selection field 502 is pressed by the user, the display processing unit 216 displays the next page of the conference material. When the left arrow button in the display page selection field 502 is pressed by the user, the display processing unit 216 displays the previous page of the conference material. Then, the operation information creation unit 210 generates an XMPP message that designates a display page, and transmits it to the conference server device 10.

  The figure selection field 503 is pushed down when a figure to be added is selected. For example, the user can add a figure to the display part 1022 by pressing the button in the figure selection field 503 indicating the figure to be added and instructing the start point, end point, etc. by clicking the mouse on the display part 1022 or touching the screen. . The graphic selection field 503 in FIG. 12 is provided with buttons indicating a straight line, a rectangle, an ellipse, and text from the left as an example of a figure to be added. Note that the method for instructing the start point, end point, etc. on the display portion 1022 differs depending on the device used by the user. For example, a mouse is used in a PC. Screen touch is used in a tablet PC. When a button indicating text is pressed as an example of a figure to be added, the display processing unit 216 allows the user to input text after selecting an area. The operation information creation unit 210 generates an XMPP message for adding a graphic from the user's operation for adding a graphic, and transmits it to the conference server apparatus 10.

  When deleting a graphic added to the display portion 1022 and displaying it, the user displays a menu by long-pressing or right-clicking the graphic. The user can delete a figure from the display portion 1022 by selecting delete from the menu. When the figure of the display portion 1022 is long pressed or right-clicked, the display processing unit 216 displays a menu.

  When the user selects deletion from the menu, the display processing unit 216 deletes the figure that has been long-pressed or right-clicked from the display portion 1022. Then, the operation information creation unit 210 generates an XMPP message for deleting the graphic and transmits it to the conference server device 10.

  The handwritten memo selection field 504 is a place for selecting addition or deletion of a handwritten memo. For example, the user can select a brush, eraser, brush size, brush color, or the like by pressing a button in a handwritten memo selection field 504 indicating a brush, eraser, brush size, brush color, or the like. As an example, the handwritten memo selection field 504 in FIG. 12 is provided with two buttons indicating a brush and an eraser in the upper row, three buttons indicating the brush size in the middle row, and six buttons indicating the color of the brush in the lower row.

  After selecting the brush, brush size, brush color, etc., the user presses the mouse button or touches the screen at a certain position on the display portion 1022, and moves the position while pressing or touching the screen. Thus, a handwritten memo can be drawn on the display portion 1022.

  The display processing unit 216 records the coordinates of the locus by the user at a certain sampling interval, and draws one trace of the handwritten memo by connecting the coordinates with a straight line. In addition, the operation information creation unit 210 generates an XMPP message for updating the handwritten memo and transmits it to the conference server device 10.

  Further, after the user presses a button indicating an eraser from the handwritten memo selection field 504, the trace can be deleted by tracing across one trace of the handwritten memo on the display portion 1022.

  The display processing unit 216 deletes the trace from the display portion 1022 by the user's operation to trace across one trace of the handwritten memo on the display portion 1022. The operation information creation unit 210 generates an XMPP message for deleting one trace of the handwritten memo and transmits it to the conference server device 10.

  FIG. 13 is an image diagram showing an example of how an XMPP message is distributed.

  As shown in FIG. 13, the XMPP message transmitted from the operation information creation unit 210 of the presenter's participating terminal A11 to the conference server device 10 via the transmitting / receiving unit 202 includes the participant's participating terminal B12, the participating terminal C13, and It is distributed to the participation terminal D14 and input to the operation type determination unit 110.

  As shown in FIG. 13, the XMPP message corresponding to the presenter's input operation transmitted from the presenter's participation terminal A11 to the conference server apparatus 10 is sent by the conference server apparatus 10 to the participation terminals of the participants participating in the conference. Delivered to B12, participating terminal C13, and participating terminal D14. The operation type determination unit 214 of the participant's participation terminal B12, participation terminal C13, and participation terminal D14 determines the XMPP message corresponding to the presenter's input operation distributed from the conference server device 10, and the display processing unit 216 performs the conference. By updating the screen, the display screen can be shared with the presenter's participating terminal A11.

  FIG. 14 is an explanatory diagram of an example of an XMPP message. FIG. 14 shows a data format of a display page designation message, a graphic addition message, a graphic deletion message, and a handwritten memo update message as an example of the XMPP message.

  Group chat in XMPP is performed in units called conference rooms. When transmitting an XMPP message in a group chat, as shown below, groupchat is specified in the type attribute of the message tag. In addition, the sender is specified in the from attribute. In the to attribute, a conference room is designated. In the “body” attribute, the content of the message is designated.

  Since the message tag is a standardized specification, detailed description is omitted. The conference system 1 according to the present embodiment is defined by the conference system 1 in the body tag of the XMPP message in order to synchronize the display of conference material display pages, graphics, handwritten notes, and the like between the participating terminal A11 and the participating terminal D14. Embed XML data.

  In this XML data, a conference tag is used for the root element, and an operation attribute is used to add additional information to the root element, and values such as update (update), add (add), and delete (delete) are specified. To do.

  The display page designation message designates a display page with a page tag in order to share the display page between the participating terminal A11 and the participating terminal D14 when the material (conference material) used in the conference is composed of a plurality of pages.

  In the graphic addition message, graphic data is specified by an autoshape tag in order to share a graphic added by a certain participating terminal between the participating terminal A11 and the participating terminal D14. The id attribute of the autoshape tag indicates a graphic identifier. The x and y attributes indicate the upper left position (for example, coordinates) of the figure. The contents of the autoshape tag define a graphic using SVG (Scalable Vector Graphics) data.

  The graphic deletion message is for deleting a graphic. The value of the operation attribute is specified by delete, and the autoshape tag to be deleted is specified. At this time, the deletion target is determined based on the id attribute of the autoshape tag.

  The handwritten memo update message specifies handwritten memo data with an ink tag in order to share a handwritten memo added at a participating terminal between participating terminals. For the ink tag, the format defined by InkML is used. InkML describes handwritten data as XML data, and recommended specifications are recommended by the World Wide Web Consortium (W3C). A handwritten memo for a conference material is expressed by one InkML data. When updating a handwritten memo, the value of the operation attribute is designated by update.

  According to one embodiment of the conference system, the presenter and the questioner can register the part (which part of which page of which material) is referred to as each view. In addition, the contents of each view are updated in synchronization with operations such as page turning and enlargement / reduction of display contents by the presenter or questioner who has registered the view.

  In addition, presenters and interrogators can refer to views of other presenters or interrogators. The presenter can also refer to the view of the presenter or questioner who is speaking. Other participants can refer to the moderator's view by following the part that is the subject of the question and answer session even if the presenter or the questioner changes.

  Each participant can refer to a plurality of views. For example, while confirming the progress of proceedings through the moderator's view, it is possible to pre-read the material being explained or refer to the attached material in his / her view.

<Second embodiment>
In the second embodiment of the conference system, the presentation is finished and a process for answering questions is added to the first embodiment of the conference system described above. FIG. 1 can be applied to the configuration of the conference system.

  For example, the hardware configuration shown in FIG. 2 can be applied to the conference server device 10, the participating terminal A11, the participating terminal B12, the participating terminal C13, and the participating terminal D14.

<Software configuration>
The conference server device 10, the participation terminal A11, the participation terminal B12, the participation terminal C13, and the participation terminal D14 of the conference system 1 according to the present embodiment are realized by the following functional blocks, for example.

<Conference server device>
The conference server apparatus 10 is realized by a functional block diagram as shown in FIG. FIG. 15 is a functional block diagram of an example of the conference server device 10. The conference server device 10 executes the program, thereby transmitting / receiving unit 102, reference location registration unit 104, storage / reading processing unit 106, storage unit 108, operation type determination unit 110, display control unit 112, display sharing processing unit 114, And the speech list processing unit 116 is realized. That is, the statement list processing unit 116 is added to the functional block diagram of the conference server apparatus 10 shown in FIG.

  The speech list processing unit 116 of the conference server device 10 is realized by a command from the CPU 26 shown in FIG. 2 and performs processing related to speech transmitted by the participating terminals. Specifically, the utterance list processing unit 116 stores utterances such as questions and answers transmitted by the participating terminals in the storage unit 108. For example, the utterance list processing unit 116 associates and stores the questions by associating the utterance date and time, the terminal to which the question is transmitted, the display when the question is made, and the question as the type of utterance. Further, for example, the speech list processing unit 116 stores an answer to a question in association with the question by associating it. Then, when the answer is stored, it is associated with the terminal to which the answer is transmitted, the display when the answer is made, and the answer type associating with the answer.

  The storage unit 108 of the conference server apparatus 10 is realized by the HDD 28 illustrated in FIG. 2, and stores a message list 1088 in addition to the display management table 1082, the reference management table 1084, the material DB 1086, and the like.

  FIG. 16 shows an example of the question list of the statement list 1088. The question list of the comment list 1088 is stored in association with each other by identifying the comment date and time, the participating terminal that transmitted the comment, the display at the time of the comment, and the comment type for each identification information for identifying the comment. Is done.

  FIG. 17 shows an example of a reply list of the speech list 1088. Corresponding to the reply list of the speech list 1088 by associating with the identification information for identifying the question to be answered, the participating terminal that sent the reply, the display at the time of speech, and the speech type Attached and memorized. In addition, the question list and the answer list can be integrated into a remark list.

<Participating terminal>
Participating terminal A11, participating terminal B12, participating terminal C13, and participating terminal D14 are described as “participating terminal 20” unless they need to be distinguished because they have the same configuration.

  The participating terminal 20 is realized by a functional block diagram as shown in FIG. FIG. 18 is a functional block diagram of an example of the participating terminal 20. The participating terminal 20 executes the program, thereby transmitting / receiving unit 202, reference location notification processing unit 204, storage / reading processing unit 206, storage unit 208, operation information creation unit 210, display sharing processing unit 212, and operation type determination unit 214. The display processing unit 216 and the message list processing unit 218 are realized. That is, the statement list processing unit 218 is added to the functional block diagram of the conference server device 10 shown in FIG.

  The speech list processing unit 218 of the participating terminal 20 is realized by a command from the CPU 26 shown in FIG. 2, and when a speech such as a question is input by the input device 21 in response to a presentation by the presenter, The utterance information is sent to the conference server device 10 with the utterance date and time, the participating terminal that transmitted the utterance, the display of the participating terminal at the time of the utterance, the utterance type, and the like.

  In addition, the utterance list processing unit 218 of the participating terminal 20 is configured such that, when a utterance such as an answer to a question transmitted by a certain participating terminal 20 is input by the input device 21, the participating terminal that has transmitted the utterance to the utterance, The information of the participating terminal at the time of speaking and the speech information such as the speech type are attached and transmitted to the conference server device 10.

<Operation of the conference system (part 3)>
FIG. 19 shows a process when a question and answer is performed between the participating terminal A11 and the participating terminal D14. In this process, when a question and answer session is performed, the part referred to by the questioner and the respondent is shared by all the participants. Specifically, the process when the participating terminal C13 and the participating terminal D14 ask questions about the announcement of the participating terminal A11 is shown.

  In step S1902, when a question is input to the participating terminal C13, the utterance list processing unit 218 includes the utterance date, the participating terminal that transmitted the utterance, the display of the participating terminal at the time of the utterance, the utterance type, and the like. Is input to the transmission / reception unit 202.

  In step S1904, the transmission / reception unit 202 of the participating terminal C13 transmits the message information input by the message list processing unit 218 to the conference server apparatus 10. For example, in the example of the participating terminal shown in FIG. 12, the speech information can be transmitted by depressing the question button 506 of the participating terminal C13.

  In step S1906, when a question is input to the participating terminal D14, the utterance list processing unit 218 includes the utterance date and time, the participating terminal that transmitted the utterance, the display of the participating terminal at the time of the utterance, the utterance type, and the like. Is input to the transmission / reception unit 202.

  In step S1908, the transmission / reception unit 202 of the participating terminal D14 transmits the message information input by the message list processing unit 218 to the conference server apparatus 10. For example, in the example of the participating terminal shown in FIG. 12, a comment can be input by depressing the question button 506 of the participating terminal D14.

  In step S1910, the message list processing unit 116 of the conference server apparatus 10 stores the message information transmitted by the participating terminal C13 and the participating terminal D14 in the message list 1088 of the storage unit 108.

  In step S1912, the speech list processing unit 218 of the participating terminal A11 creates a speech list request and transmits it from the transmission / reception unit 202 by an operation of referring to the speech list by the conference participant A of the participating terminal A11.

  In step S1914, the message list processing unit 116 of the conference server apparatus 10 transmits a message list to the participating terminal A11 in response to the message list request transmitted from the participating terminal A11.

  In step S1916, the utterance list processing unit 218 of the participation terminal A11 accepts the selection of the utterance by the conference participant A of the participation terminal A11, and the utterance selection information accompanied by the identification information of the utterance selected by the conference participant A is obtained. Input to the transmission / reception unit 202.

  In step S <b> 1918, the transmission / reception unit 202 of the participating terminal A <b> 11 inputs the speech selection information input by the speech list processing unit 218 to the conference server device 10.

  In step S1920, the utterance list processing unit 116 of the conference server device 10 refers to the utterance list 1088 of the storage unit 108, and at the time of the utterance associated with the utterance specified by the utterance selection information transmitted by the participating terminal A11. The display sharing processing unit 114 is instructed to share the utterance view, which is a display of, between the participating terminal A11 and the participating terminal D14. The display sharing processing unit 114 changes the reference information of each participating terminal stored in the reference management table 1084 of the storage unit 108 to the utterance time view in accordance with an instruction from the utterance list processing unit 116. Further, the display sharing processing unit 114 creates display sharing information accompanied by a view at speech, and inputs the display sharing information to the transmitting / receiving unit 102.

  In steps S1922 to S1926, the transmission / reception unit 102 of the conference server apparatus 10 transmits the display sharing information to the participating terminal B12, the participating terminal C13, and the participating terminal D14.

  In step S1928, the display sharing processing unit 212 of the participating terminal B12 switches the display to the utterance view based on the display sharing information transmitted by the conference server device 10.

  In step S1930, the display sharing processing unit 212 of the participating terminal C13 switches the display to the speech view based on the display sharing information transmitted by the conference server device 10.

  In step S1932, the display sharing processing unit 212 of the participating terminal D14 switches the display to the speech view based on the display sharing information transmitted by the conference server device 10.

  As shown in FIG. 10, the display of each reference terminal immediately after step S1932 displays the first display of the participating terminal A11 on the display portion 1022 of the participating terminal A11 and the display portion 1022 of the participating terminal B12. The view A of the terminal A11 is displayed, the view A of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal C13, and the view A of the participating terminal A11 is displayed on the display portion 1022 of the participating terminal D14.

  FIG. 20 shows an example of the display of each reference terminal during a question-and-answer session. In the example shown in FIG. 20, the conference participant B of the participation terminal B12 is a questioner, and the conference participant C of the participation terminal C13 is an answerer.

  The presenter or the moderator uses the conference terminal A11 to perform an operation of switching the display between the participating terminal B12 and the participating terminal C13, so that the operation information is transmitted to the conference server device 10. The conference server device 10 switches the display of the reference terminal A11 according to the operation information, and switches the display of the participating terminal B12 and the participating terminal C13 with reference to the reference management table 1084.

  FIG. 21 shows an example of the reference management table 1084 during the question-and-answer session. As shown in FIG. 21, the display of participating terminal A11 is changed between view B and view C, the display of participating terminal B12 is view B, the display of participating terminal C13 is the view C, and the display of participating terminal D14. Is view A. That is, the conference participant D of the participation terminal D can refer to the display of the participation terminal B of the questioner and the display of the participation terminal C of the respondent alternately by referring to the display of the participation terminal A11.

  According to one embodiment of the conference system, a speaker (presenter or questioner) can register a snapshot of his / her view according to a location referred to in connection with the speech in the speech list. . Furthermore, when speaking a plurality of times, it can be realized by repeating the registration operation a plurality of times.

  Here, as shown in FIG. 16 and FIG. 17, the remark list has a hierarchical structure. When a certain question is answered, the question is registered in the higher remark list, and the answer is It is registered in the lower-level speech list according to the question.

  The presenter or presenter can select a view registered in the speech list. Thus, in addition to being able to select in the registered order, for example, when a specific speaker is biased, it is possible to skip or change the selection order.

  Further, if there is a remaining message in the comment list at the end of the conference, it can be reused in the next conference.

  In addition, by leaving the moderator's view in the log, it is possible to confirm which presenter or questioner is responsible for the presentation and question-and-answer session, which is helpful when preparing the minutes.

<Summary>
According to this embodiment, the location referred to by the presenter and the questioner can be registered as each view, and each view is further operated by the presenter or the questioner who registered the view. Can be updated synchronously.

  In addition, the presenter and the questioner can refer to the views of other presenters or questioners, and the presenter can also refer to the view of the presenter or questioner who is speaking. Other participants can refer to the moderator's view by following the part that is the subject of the question and answer session even if the presenter or the questioner changes.

  The present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, although the conference system is shown as an example of the information processing system in the present embodiment, the present invention can be applied to a system for sharing and displaying information such as a seminar system and an education system.

  For example, the conference server device is an example of an information processing device, and the participating terminal is an example of a terminal device. The reference management table is an example of a reference management unit.

  The conference system 1 having the conference server device 10 and the participation terminal A11-participation terminal D14 is merely an example, and it goes without saying that there are various system configuration examples depending on applications and purposes.

  Further, for example, in the present embodiment, the conference system is configured by the conference server device and the participating terminals, but the function of the conference server device can also be applied to each participating terminal. Thereby, an information processing system can be constituted without providing a conference server device.

  Although the present invention has been described with reference to specific embodiments and modifications, each embodiment and modification is merely an example, and those skilled in the art will recognize various modifications, modifications, alternatives, substitutions, and the like. Will understand. For convenience of explanation, an apparatus according to an embodiment of the present invention has been described using a functional block diagram, but such an apparatus may be implemented in hardware, software, or a combination thereof. The present invention is not limited to the above-described embodiments, and various variations, modifications, alternatives, substitutions, and the like are included without departing from the spirit of the present invention.

10 Conference server device 11 Participating terminal A
12 Participating terminal B
13 Participating terminal C
14 Participating terminal D
21 Input device 22 Display device 23 External I / F
24 RAM
25 ROM
26 CPU
27 Communication I / F
28 HDD
DESCRIPTION OF SYMBOLS 102 Transmission / reception part 104 Reference location registration part 106 Storage / reading process part 108 Storage part 110 Operation type discrimination | determination part 112 Display control part 114 Display share processing part 116 Statement list processing part 1082 Display management table 1084 Reference management table 1086 Reference DB
1088 Statement list 202 Transmission / reception unit 204 Reference location notification processing unit 206 Storage / reading processing unit 208 Storage unit 210 Operation information creation unit 212 Display sharing processing unit 214 Operation type determination unit 216 Display processing unit 218 Statement list processing unit 2082 Data DB

JP 2012-108872 A

Claims (6)

An information processing system having an information processing device and a plurality of terminal devices connected to the information processing device,
The first terminal device of the plurality of terminal devices is
A first display processing unit that performs a process of displaying a first display operable on the first terminal device or a second display displayed on another terminal device other than the first terminal device;
An operation information creation unit that creates first operation information representing the operation content when the first display that has been processed by the first display processing unit is operated;
A first transmission unit for transmitting the first operation information to the information processing apparatus,
The information processing apparatus includes:
For each of the plurality of terminal devices, a reference management unit that associates and stores reference information representing the reference content of the one displayed in the first display or the second display;
Based on the first operation information transmitted by the first terminal device, the reference management unit identifies the second terminal device displaying the operated first display, and A display control unit for creating second operation information destined for the second terminal device;
A second transmitter for transmitting the second operation information,
The second terminal device is:
An operation type determination unit for determining an operation type based on the second operation information;
An information processing system comprising: a second display processing unit that updates the displayed second display based on the operation type determined by the operation type determination unit. Each of the plurality of terminal devices is
A reference location notification processing unit for creating reference location information representing reference information representing the reference content of the first display or the second display being displayed,
The first transmission unit transmits the reference location information to the information processing apparatus,
The information processing apparatus includes:
The information processing system according to claim 1, further comprising: a reference location registration unit that registers reference content of the reference management unit based on the reference location information transmitted from each of the plurality of terminal devices. The first terminal device is:
A first display sharing processing unit for creating a display sharing command for performing processing for sharing display with other terminal devices;
The first transmission unit transmits the display sharing command to the information processing apparatus,
The information processing apparatus includes:
The information processing system according to claim 1, further comprising: a second display sharing processing unit configured to share a display with the other terminal device in accordance with the display sharing command transmitted by the first terminal device. Each of the plurality of terminal devices is
A first utterance list processing unit for creating utterance data associated with data input in connection with the first display or the second display displayed by the first terminal device;
The first transmission unit transmits the message data to the information processing device,
The information processing apparatus includes:
A second utterance list processing unit for registering the utterance data transmitted by each of the plurality of terminal devices;
Based on the utterance data registered by the second utterance list processing unit, the second display for sharing the first display or the second display related to the utterance data among the plurality of terminal devices. The information processing system according to any one of claims 1 to 3, further comprising: a display sharing processing unit. An information processing apparatus connected to a plurality of terminal devices,
For each of the plurality of terminal devices, the reference content of the first display that can be operated on each terminal device and the second display that is displayed on other terminal devices other than the terminal device A reference management unit that associates and stores reference information representing
The reference management based on operation information representing the operated content transmitted when operated on the first display displayed on the first terminal device of the plurality of terminal devices. A display control unit that identifies a second terminal device that displays the operated first display and creates second operation information that is destined for the second terminal device; An information processing apparatus comprising: a transmission unit that transmits the operation information of 2. An information processing method in an information processing apparatus connected to a plurality of terminal devices,
The information processing apparatus includes:
For each of the plurality of terminal devices, the reference content of the first display that can be operated on each terminal device and the second display that is displayed on other terminal devices other than the terminal device A reference management unit for storing reference information representing
The information processing apparatus includes:
The reference management based on operation information representing the operated content transmitted when operated on the first display displayed on the first terminal device of the plurality of terminal devices. A second terminal device displaying the operated first display from the section,
Creating second operation information destined for the second terminal device;
An information processing method for transmitting the second operation information.