JP2015153016A - Display system - Google Patents

Abstract

【Task】
It is an object of the present invention to provide a display system in which a plurality of portable communication terminals are arranged in the vertical and / or horizontal direction and each of the display devices is displayed as one display device.
[Solution]
A portable communication terminal that is a communication partner of the portable communication terminal is identified, a combination position determination processing unit that determines a combination position with the portable communication terminal, and an operation log that receives an operation input in the portable communication terminal An operation input reception processing unit that generates information, an operation log information transmission processing unit that directly transmits the generated operation log information to a communication partner portable communication terminal, and a portable communication terminal that is a communication partner. Operation log information received based on operation input on portable communication terminals, operation log information reception processing unit that receives directly, and operation execution processing unit that executes the operation corresponding to the generated operation log information or received operation log information And a display processing unit that executes display processing of an area corresponding to the combination position.
[Selection] Figure 1

Description

  The present invention relates to a display system in which a plurality of portable communication terminals are combined in a vertical and / or horizontal direction to perform display using each portable communication terminal as one display device.

  In recent years, portable communication terminals such as mobile phones, PHS, and tablet computers have become widespread and are used in various situations such as school education and company training. Then, portable communication terminals are handed to students, students, students (hereinafter collectively referred to as “learners”), and between teachers and lecturers (hereinafter collectively referred to as “teachers”). In addition, classes and lectures using two-way communication (hereinafter collectively referred to as “classes”) are beginning to be implemented.

  In the class, a group may be formed by multiple learners, such as group learning. For example, an operation of creating one picture among a plurality of learners is an example. In such a case, work is performed using one portable communication terminal in the group.

  However, when a plurality of people form a group, the portable communication terminal is small because of its characteristics, and the display device is also small. Therefore, it is easy for a plurality of people to work. Therefore, a system that is used as one display device by combining a plurality of portable communication terminals is disclosed in Patent Document 1 and Non-Patent Document 1 below.

JP 2009-94775 A

Eiyuki Suzuki, Hideo Funyu, Yoshihiko Kubota, “Support for Collective Collaborative Learning Using Tablet Devices-XingBoard: Concept and Prototype of Crossing Board”, IEICE Tech. 113, no. 166, The Institute of Electronics, Information and Communication Engineers, July 2013, p. 41-p. 46

  In the case of the above-described system disclosed in Patent Document 1, it is possible to form a single display device and display a screen by combining a plurality of portable terminals vertically and horizontally. However, a process is performed in which the central mobile terminal that displayed the original screen divides the original screen into display areas to be displayed on each combined mobile terminal, and sends it to each mobile terminal. ing.

  For this reason, a process of dividing the original screen in the central mobile terminal and a process of sending the screen to each mobile terminal are required. Therefore, the load on the central mobile terminal is large. Since the amount of communication increases due to the necessity of sending screen data and communication time is required, if sufficient high speed cannot be ensured for communication by a non-contact IC, processing is delayed.

  In addition, since the processing is performed in the central mobile terminal, the screen can only be divided and displayed, and the combined mobile terminals cannot perform editing such as writing on the screen. Therefore, work restrictions occur when working in groups.

  The system of Non-Patent Document 1 is the same as Patent Document 1 in that a plurality of tablet terminals are combined vertically and horizontally to form one display device and display a screen. However, the system of Non-Patent Document 1 has a client-server configuration. Therefore, all operations on each tablet terminal are once sent to the server, and after the processing for the operation is executed on the server, the screen is sent to each tablet terminal. Therefore, transmission of data from the tablet terminal to the server, processing at the server, and transmission of data from the server to the tablet terminal are required, each of which requires processing time. Therefore, if an operation is performed on any one of the combined tablet terminals, a delay occurs before the operation is reflected on the other tablet terminals. In addition, it cannot be used unless the environment can ensure communication with the server. Since processing is performed on the server, processing is concentrated and processing delays are likely to occur.

  In particular, in school education and corporate training where multiple groups are created, the server handles all of the processing performed independently by each group, increasing the server load. End up. Furthermore, depending on the progress of the lesson, it is assumed that many portable communication terminals used by the learners can connect to the network all at once. In that case, in the case of a system that goes through a server, the load on the server increases, and there is a high possibility that processing delay will occur.

  Due to such problems, in the case of Non-Patent Document 1, even if a plurality of tablet terminals are combined, the operation is not immediately reflected on other terminals due to the processing time delay, and it is difficult to feel unity. It has become. Especially in the field of school education, it is required to bring out the effects of collaborative learning such as “sharing awareness” through group learning, “invigoration of discussions”, and “exchange among learners”. For this purpose, smooth reflection (visualization) of operation contents is necessary.

  In view of the above problems, the present inventor has no load on a specific portable communication terminal, and when one screen is displayed on a plurality of portable communication terminals, the screen of which portable communication terminal is displayed. We have invented a display system that can reflect the operation even if the operation is performed.

  A first invention is a display system that displays a single display device by combining a plurality of portable communication terminals, and an operation input reception processing unit that receives an operation input in the portable communication terminal and generates operation log information. And an operation log information transmission processing unit for directly transmitting the generated operation log information to another portable communication terminal, and an operation based on an operation input at the portable communication terminal from the other portable communication terminal. The operation log information reception processing unit that directly receives the log information and the portable communication terminal of the portable communication terminal when the one display device displays the log information based on the generated operation log information or the received operation log information. And a display processing unit that executes display processing of information in the display area.

  With the configuration of the display system according to the present invention, operations performed on the portable communication terminal are directly transmitted to other portable communication terminals as operation log information. An operation execution process and a display process based on the operation log information are performed in each portable communication terminal. As a result, since the processing is not concentrated on one portable communication terminal and the operation is not executed, it is possible to realize a configuration in which no load is applied to a specific portable communication terminal. In addition, according to the configuration of the present invention, operations at each portable communication terminal are sent to each other, so that any operation performed at any portable communication terminal can be reflected to other portable communication terminals. It becomes possible. Furthermore, since processing can be executed without going through a server, communication time can be saved, and processing can be executed even when a network connection such as the Internet cannot be secured.

  For example, when the present invention is used in a school lesson, depending on the progress of the lesson, it can be assumed that a large number of portable communication terminals used by the learner perform network connection all at once. At that time, in the case of a system via a server, the load on the server increases, and there is a high possibility that processing delay will occur.

  In addition, the display system of the present invention is configured such that each portable communication terminal having a connection relationship uses the same operation log information. Therefore, each portable communication terminal has an operation process execution result based on the operation log information. On the other hand, the display displays the position of its own portable communication terminal when the portable communication terminals are combined. For example, when four portable communication terminals are connected in a combination of two vertically and two horizontally, if the combined position is the upper left, the portable communication terminal can be another portable type. The operation is executed using the same operation log information as that of the communication terminal, but the display area of the display device is the upper left display area of the four display areas. With this configuration, when a plurality of portable communication terminals are arranged side by side, it is possible to realize a display as if it were used as one display device.

  Further, in the display system of the present invention, each portable communication terminal having a connection relationship uses the same operation log information. Even when shifting to the above work, it is possible to cancel the connection relationship at an arbitrary timing and smoothly shift to individual work. Especially in the field of school education, this smooth realization from collaborative operation to individual work by such groups has a great effect in terms of maintaining the continuity of work without stopping the learner's thinking. Play.

  On the other hand, in the case of a system using a server as in Non-Patent Document 1, operations by each terminal connected to the server are executed on the server and all are managed by the server. Then, the server performs distribution processing of contents to be displayed on each terminal, thereby storing information on contents to be displayed on each terminal. For this reason, when the connection relation is canceled from the cooperative operation state by the group and the process shifts to individual work, it is necessary to wait for execution of processing for distributing data from the server.

  In addition, only a specific terminal among the portable communication terminals having a connection relationship holds all data, and functions as a virtual “parent terminal” that handles operation processing and distribution processing at once. It is also possible to realize this. However, when this method is used, high processing performance is required to perform both the operation processing and the distribution processing in the portable communication terminal that is the “parent terminal”.

  As described above, by using the display system of the present invention, a load is not applied to a specific portable communication terminal, and when one screen is displayed on a plurality of portable communication terminals, which portable communication terminal is displayed. Even if an operation is performed on the terminal screen, a display system that can reflect the operation is possible.

  In the above-described invention, the display system receives a start of connection establishment processing for performing direct communication between portable communication terminals, and receives a reference information distribution processing unit that distributes predetermined inquiry information; A response determination unit that receives response information sent from a portable communication terminal of a communication partner corresponding to the information, determines whether the inquiry information matches or corresponds to the response information, and based on the determination result Thus, it can be configured as a display system having a connection establishment processing unit for establishing a connection with the portable communication terminal of the communication partner.

  In order to establish a direct communication connection with another portable communication terminal, it is preferable to execute as in the present invention. For example, in an elementary school class of 20 students, assuming a scene in which each portable communication terminal is used, a large number of learners start using the portable communication terminal at the same time. In particular, when learning is started by combining each portable communication terminal in a group of four people, the above-described process for establishing a direct communication connection is executed simultaneously in five groups. In that case, it is necessary to establish a connection only with the learner's portable communication terminal belonging to a specific group, but this can be easily realized by using transmission information and response information as in the present invention. .

  In the above-described invention, the combination position determination processing unit specifies the communication partner portable communication terminal based on the determination result by the correspondence determination processing unit, and determines a combination position with the portable communication terminal. It can be configured like a display system to determine.

  There are various methods for combining the portable communication terminals. By executing the processing as in the present invention, the operability is good and can be easily realized. That is, when information indicating the combination position is included as inquiry information and response information in the correspondence determination processing unit, the portable communication terminal of the communication partner can be specified and the combination position can be determined based on the determination result. For example, when “right” is included as the inquiry information and “left” is included as the response information, those portable communication terminals can be specified and determined as the combination position.

  In the above-described invention, the display system is a display system having a timing synchronization processing unit that transmits timing information for synchronizing timing between portable communication terminals to the portable communication terminal of the communication partner. Can be configured.

  In the present invention, since display processing is executed in each portable communication terminal using operation log information, it is necessary to share information indicating timing such as date information. In other words, even if each portable communication terminal performs its own processing, combining multiple portable communication terminals side by side makes it possible to realize a display that makes it easy to feel unity as a single display device. In order to achieve this, display must be performed simultaneously or nearly simultaneously. Therefore, it is required to synchronize the timing by executing processing as in the present invention.

  In addition, when the same operation log information is used in each portable communication terminal by synchronizing the timing of the date and time information of the present invention, for example, an operation executed by one portable communication terminal, for example, “Undo operation (one Can be immediately reflected on other portable communication terminals. This is because the timing is synchronized at each portable communication terminal, and the operation log information performed at one portable communication terminal is also sent to other portable communication terminals, and processing is executed at each portable communication terminal. Because.

  In this way, the fact that operations performed on other portable communication terminals can be immediately reflected is also important in terms of realizing a display that makes it easy to feel unity as a single display device. .

  In the above-described invention, the display system includes an operation log information storage unit that stores operation log information generated by the operation input reception processing unit and / or operation log information received by the operation log information reception processing unit. When the connection establishment processing unit establishes a connection with a new communication partner portable communication terminal, the operation log information transmission processing unit stores the operation log information in the operation log information storage unit before the connection is established. The operation log information to be transmitted can be configured as a display system that transmits to the new communication partner portable communication terminal.

  As in the present invention, operation log information is stored in each portable communication terminal, and when a new portable communication terminal is added to the connection while connected to another portable communication terminal (for example, four units) When two groups are newly added in the case where one group is formed), it is necessary to send the operation log information so far to the two. As a result, the same operation log information can be used in all connected portable communication terminals, and it is possible to perform a unified display as one display device.

  The above display system can be realized by using the display program of the present invention. That is, a display program in a display system that displays a single display device by combining a plurality of portable communication terminals, and the display program makes the portable communication terminal a communication partner of the portable communication terminal. A combination position determination processing unit that identifies a portable communication terminal and determines a combination position with the portable communication terminal; an operation input reception processing unit that receives operation input in the portable communication terminal and generates operation log information; An operation log information transmission processing unit for directly transmitting the generated operation log information to the communication partner portable communication terminal, an operation based on an operation input at the portable communication terminal from the communication partner portable communication terminal Log information is directly received from the operation log information reception processing unit, the generated operation log information, or the received operation log information. Operation execution unit for executing an operation that, the display processing unit for executing display processing of the region corresponding to the combination positions the determined, a display program to function as a.

  By loading the display program of the present invention into a portable communication terminal and executing it, the same technical effect as that of the first invention can be obtained.

  With the display system of the present invention, when no load is applied to a specific portable communication terminal and one screen is displayed on a plurality of portable communication terminals, editing is performed on the screen of any portable communication terminal. Even if is done, it can be reflected.

It is a figure which shows typically an example of the conceptual diagram of the whole display system. It is a figure which shows typically the conceptual diagram of an example of the system configuration | structure of a display system. It is a figure which shows an example of the hardware constitutions of a portable communication terminal. It is a flowchart which shows an example of the process of a display system typically. It is a sequence diagram which shows typically an example of the connection establishment process in the case of a combination of portable communication terminals. It is a figure which shows an example of an operation log information storage part typically. It is a figure which shows an example of a timing synchronous process typically. It is a figure which shows typically the relationship between a physical coordinate (absolute coordinate) and a client coordinate (relative coordinate) at the time of combining a portable communication terminal. It is a figure which shows typically the relationship of the physical coordinate at the time of combining the portable communication terminal A thru | or the portable communication terminal D like FIG.1 (b), and a client coordinate. It is a figure which shows typically an example of the operation log information to produce | generate. It is a figure which shows typically an example at the time of transmitting operation log information with respect to the portable communication terminal B thru | or the portable communication terminal D from the portable communication terminal A. It is a figure which shows typically the process which converts a physical coordinate into a client coordinate in each portable communication terminal. It is a figure which shows typically an example at the time of transmitting operation log information with respect to the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D from the portable communication terminal B. FIG. It is a figure which shows typically the process which converts a physical coordinate into a client coordinate in each portable communication terminal. It is a conceptual diagram which shows typically the process in the case of moving and displaying the image currently displayed with the portable communication terminal to another portable communication terminal. It is a figure which shows typically an example of the operation log information to produce | generate. It is a conceptual diagram which shows typically the case where the display processing of image information is performed with the portable communication terminal A. FIG. It is a conceptual diagram which shows typically the case where the image information of FIG. 17 is moved to the position where each portable communication terminal combines. It is a figure which shows typically an example of the operation log information produced | generated when performing operation to move. It is a figure which shows typically the process in the case of changing the display size of image information. It is a figure which shows typically an example of the operation log information to produce | generate. FIG. 4 is a diagram schematically illustrating an example of operation log information generated by the portable communication terminal B and operation log information generated by the portable communication terminal C when operations compete. It is a figure which shows typically an example of the operation log information memorize | stored in the operation log information storage part of each portable communication terminal. It is a figure which shows typically an example in the case of performing operation spanning a different portable communication terminal. It is a figure which shows typically an example of the process in the case of performing operation spanning a different portable communication terminal. It is a figure which shows an example of a determination table typically. It is a figure which shows typically the management of the upper and lower sides of a layer. It is a figure which shows typically the display in a touchscreen type display apparatus.

  An example of an overall conceptual diagram of the display system 1 of the present invention is shown in FIG. A conceptual diagram of an example of the system configuration of the display system 1 is shown in FIG.

  The display system 1 is realized in a portable communication terminal used by each learner. The portable communication terminal in the display system 1 is preferably a portable communication terminal provided with a touch panel display device 72. FIG. 3 shows an example of the hardware configuration of the portable communication terminal. The portable communication terminal includes a calculation device 70 such as a CPU that executes calculation processing of a program, a storage device 71 such as a RAM and a hard disk that stores information, and a touch panel display device such as a touch panel display that can be input by a touch panel. 72 and a communication device 73 for directly communicating the processing results of the arithmetic device 70 and information stored in the storage device 71 between terminals.

  The touch panel display device 72 is a device in which the functions of the display device and the input device are integrated in that input can be performed directly on the display by a predetermined input device (such as a touch panel pen) or a finger.

  Each means in the present invention is only logically distinguished in function, and may be physically or virtually the same area.

  The display system 1 of the present invention is provided in a portable communication terminal used by each learner. The display system 1 includes an inquiry information distribution processing unit 10, a correspondence determination processing unit 11, a connection establishment processing unit 12, a combination position determination processing unit 13, a timing synchronization processing unit 14, an operation input reception processing unit 15, and an operation log information storage unit. 16, an operation log information transmission processing unit 17, an operation log information reception processing unit 18, an operation execution processing unit 19, and a display processing unit 20. Each portable communication terminal preferably has the same function.

  The inquiry information distribution processing unit 10 receives an input for starting processing of the display system 1 of the present invention. For example, an input indicating that a region displayed at a predetermined location of the touch panel display device 72 in the portable communication terminal has been selected is accepted. Then, as shown in FIG. 5 described later, the inquiry information (key) is broadcasted to other portable communication terminals within a predetermined distance for executing the combination processing.

  The correspondence determination processing unit 11 receives response information (value) sent from another portable communication terminal, and is the response information (value) matching or corresponding to the inquiry information distributed by the inquiry information distribution processing unit 10 Determine.

  Inquiry information refers to whether a portable communication terminal that can be combined is within a communicable range (within a predetermined distance within which communication can be reached in ad hoc mode, or within a range in which network communication is possible in other communication modes). Information that is broadcast to inquire about whether or not. Instead of broadcast distribution, a method of performing unicast distribution specifying a plurality of terminals that can be connected may be used.

  The response information is information returned from the portable communication terminal that has received the inquiry information in response to the inquiry information.

  The correspondence determination processing unit 11 determines the correspondence based on the determination table shown in FIG. 26 in order to determine the correspondence between the inquiry information and the response information. However, the present invention is not limited to this.

  When the correspondence determination processing unit 11 determines that the correspondence between the inquiry information and the response information matches or corresponds, the connection establishment processing unit 12 and the portable communication terminal that sent the inquiry information (key) A connection is established with the portable communication terminal that has returned the response information (value).

  In this case, the form of direct connection (generally called “Peer to Peer”, “P2P”) between portable communication terminals is a direct short-range wireless communication between portable communication terminals ( In general, the connection is based on a communication connection form called “ad hoc mode”. Environments that cannot be connected to a local area network (LAN) by communicating in "ad hoc mode" (for example, observations and experiments performed outdoors if the display system of the present invention is used at school) Even in this case, the display system of the present invention can be realized. Note that the form of direct connection is not limited to the ad hoc mode, and any form of communication connection may be used as long as direct connection between portable communication terminals can be realized.

  When the connection establishment processing unit 12 establishes a connection between a plurality of portable communication terminals, the timing synchronization processing unit 14 executes processing for synchronizing the timing between the portable communication terminals. The date and time information is an example of the timing. However, the timing is not limited to this, and a timer that counts at a fixed timing may be used.

  As an example of specific processing, the timing (date / time information) of the portable communication terminal that delivered the inquiry information (key) is transmitted to other portable communication terminals, and the other information is aligned with the sent date / time information. There is a method of executing processing for synchronizing the timing (date and time information) of the portable communication terminal. In this case, with regard to the timing synchronization processing, the portable portable terminal that has transmitted the inquiry information (key) is a virtual “reference terminal” (terminal that is a reference for timing), and portable communication that has transmitted response information (value). The terminal is handled in the form of a virtual “correction target terminal” (terminal whose timing is corrected in accordance with the reference terminal). Note that any processing format may be used as long as the timing can be synchronized between the portable communication terminals with which the connection has been established.

  The operation input reception processing unit 15 receives an operation input to the portable communication terminal used by the learner. Operation log information is generated by receiving an operation input by the operation input reception processing unit 15 and is stored in an operation log information storage unit 16 described later. The operation log information includes information indicating timing, information indicating the operation content, and, in some cases, coordinate information.

  The operation log information storage unit 16 stores the operation log information generated by the operation input reception processing unit 15 and the operation log information received by the operation log information reception processing unit 18. FIG. 6 schematically shows an example of the operation log information storage unit 16. In addition to storing the operation log information itself indicating the operation content in the operation log information, information (object, material) of the object to be operated is stored as necessary. If the same object information is used for a plurality of operations, only one information need be stored.

  The operation log information transmission processing unit 17 transmits the operation log information generated by the operation input reception processing unit 15 to another portable communication terminal that has established a connection with the portable communication terminal. At this time, in addition to transmitting the operation log information itself, if there is object information to be operated, that information is also transmitted in association with the operation log information. If information on an object to be operated has already been transmitted and received and stored in each portable communication terminal, only the operation log information needs to be sent without sending the object information. As a result, the amount of information to be transmitted can be reduced. The object match may be configured to provide an object match determination function, and to notify an object transmission request to the terminal that has transmitted the operation log information in the case of a mismatch. The object match determination can be performed using object identification information (such as a file name) and information such as the object size and position. By adopting such a configuration, it is possible to eliminate the occurrence of deviations in the objects referred to between the portable communication terminals.

  The operation log information reception processing unit 18 receives operation log information for an operation input performed on the other portable communication terminal from another portable communication terminal that has established a connection with the portable communication terminal.

  The operation execution processing unit 19 is based on the operation log information received by the operation input reception processing unit 15, the operation log information received by the operation log information reception processing unit 18, and the operation log information stored in the operation log information storage unit 16. An operation process corresponding to the operation log information is executed. At the time of this operation, processing is executed on the object to be operated according to the operation content of the operation log information.

  The display processing unit 20 displays the region at the combination position based on the combination position determined by the combination position determination processing unit 13 based on the processing result in the operation execution processing unit 19 and the touch panel type display device 72 of the portable communication terminal. To do. At this time, the display is performed after the coordinate conversion of the image displayed on the touch panel type display device 72 is performed. The display processing unit 20 determines whether the client coordinates, which are relative coordinates in the operation log information, are within the display area of the portable communication terminal. If it is determined, the display processing unit 20 performs display processing. Does not display. If the processing content in the operation log information is processing that does not depend on the display position, the display processing is performed. After client coordinate conversion, it may be determined whether or not it is within the display area. If it is determined whether or not the physical coordinates that are absolute coordinates are within the display area and displayed, the client coordinates are displayed. A coordinate conversion process may be performed.

  A basic process for displaying in combination with a portable communication terminal will be described. In the display system 1 of the present invention, a physical coordinate that is an absolute coordinate indicating an editable region of content displayed on a screen and a client that is a relative coordinate indicating a range displayed on the touch panel display device 72 of each portable communication terminal. There are coordinates. This coordinate relationship is shown in FIG. In the case of FIG. 8, the origin (cx, cy) of the client coordinates is the physical coordinates (X = 500, Y = 300). Therefore, coordinate conversion between physical coordinates and client coordinates can be performed by adding or subtracting the client coordinates or physical coordinates.

  In other words, the client coordinates can be calculated by subtracting the distance from the physical coordinate origin to the origin of the portable communication terminal from the physical coordinate value, and the absolute coordinate is calculated by subtracting the physical coordinate value from the physical coordinate value. What is necessary is just to calculate the value which added the distance from the origin to the origin in the portable communication terminal.

For example, in FIG. 8, the physical coordinates are (X, Y), the client coordinates of the portable communication terminal A are (cx1, cy1), the client coordinates of the portable communication terminal B are (cx2, cy2), and the portable communication terminal C Assume that the client coordinates are (cx3, cy3) and the client coordinates of the portable communication terminal D are (cx4, cy4). In this case, the conversion process from client coordinates to physical coordinates is:
In portable communication terminal A,
X = cx1, Y = cy1
In portable communication terminal B,
X = cx2 + 500, Y = cy
In the portable communication terminal C,
X = cx3, Y = cy3 + 300
In the portable communication terminal D,
X = cx4 + 500, Y = cy4 + 300
It becomes.

The conversion process from physical coordinates to client coordinates is as follows:
In portable communication terminal A,
cx1 = X, cy1 = Y
In portable communication terminal B,
cx2 = X−500, cy2 = Y
In the portable communication terminal C,
cx3 = X, cy3 = Y-300
In the portable communication terminal D,
cx4 = X-500, cy4 = Y-300
It becomes.

  Therefore, for example, when the portable communication terminal A to the portable communication terminal D are combined as shown in FIG. 1B, the physical coordinates and the client coordinates are as shown in FIG. That is, the physical coordinates (X, Y) are changed from (0, 0) to (999, 599). When the display area of each portable communication terminal on the physical coordinates is indicated by physical coordinates, the portable communication terminal A is (cx1 , Cy1) from physical coordinates (0,0) to (499,299), portable communication terminal B from (500,0) to (999,299) as (cx2, cy2), and portable communication terminal C (Cx3, cy3) is changed from (0, 300) to (499, 599), and the portable communication terminal D is changed from (500, 300) to (999, 599) as (cx4, cy4).

  Each portable communication terminal displays the range included in the physical coordinates on each touch panel display device 72. In that case, the physical coordinates must be changed to the client coordinates (relative coordinates) of each touch panel display device 72. For example, in the case of FIG. 9, client coordinates (cx1, cy1) = (X, Y) of portable communication terminal A, client coordinates (cx2, cy2) = (X-500, Y), portable type of portable communication terminal B By calculating the client coordinates (cx3, cy3) = (X, Y−300) of the communication terminal C and the client coordinates (cx4, cy4) = (X−500, Y−300) of the portable communication terminal D, The information given by the coordinates is changed to the client coordinates and displayed on the touch panel display device 72.

  Next, an example of the processing process of the display system 1 of the present invention will be described with reference to the flowchart of FIG. 4 and the sequence diagram of FIG. In the following description, a case where date and time information is used as timing will be described. Moreover, it is group learning in a school lesson, Comprising: Four learners (students) (learners A thru | or learner D) connect each portable communication terminal (portable communication terminal A thru | or portable communication terminal D). The case of combining will be described. Further, in the first embodiment, processing when not depending on coordinates will be described.

  First, when the portable communication terminals used by each learner are combined and the connection is established between the portable communication terminals to perform direct communication, it is necessary to acquire identification information used for communication such as the other party's IP address. Therefore, when a selection to start the connection establishment process is accepted by the inquiry information distribution processing unit 10 in a portable communication terminal (S100), the connection establishment process shown in FIG. 5 is started (S110).

  For example, when learner A selects to start connection establishment processing in portable communication terminal A, it is accepted by inquiry information distribution processing unit 10 of portable communication terminal A, and the inquiry information that becomes the key is broadcast and distributed. . Here, since group learning processing is performed, for example, information on a group to which the learner A belongs (for example, “B group”) is distributed. It is preferable to use the information associated with the learner when the learner logs in to the portable communication terminal, but the inquiry information that becomes the key is not limited to this, and when the connection establishment process is started. Input may be accepted.

  The connection establishment processing unit 12 of the portable communication terminal B to the portable communication terminal D accepts the information that is the key broadcasted from the portable communication terminal A, and the portable communication terminal that has matching or corresponding information is the key. Is returned by broadcast distribution (in this embodiment, in particular, “ad hoc mode” is used).

  When the value information corresponding to the key information broadcast from the portable communication terminal A is received from the portable communication terminal B to the portable communication terminal D as described above, the connection of the portable communication terminal A is established. The processing unit 12 sends information used for network connection, such as an IP address, between the portable communication terminal A and the portable communication terminal B to the portable communication terminal D to establish a communication connection. In addition, the portable communication terminal A sends information used for network connection of the portable communication terminal B to the portable communication terminal D, such as an IP address, to the portable communication terminal B to the portable communication terminal D. As a result, the portable communication terminal B to the portable communication terminal D establish a communication connection with another portable communication terminal other than the portable communication terminal A.

  When determining whether the key or value is portable communication terminals belonging to the same group, the class information of the learner using the portable communication terminal as the key (for example, “1 -A ") or group identification information (for example," B group "), and other portable communication terminals use the class information (for example," 1-A ") of the learner who uses it or group identification information (for example," B "). "B group") is returned as value. Then, in the correspondence determination processing unit 11, if the key and value match or correspond to each other, it is determined as a valid communication partner.

  Further, when the contents of the screens displayed between the portable communication terminals (for example, the size of the mount and the type of the page) match or correspond to each other, it may be determined as a valid communication partner. In this case, the title of the mount displayed on the portable communication terminal is sent as the key, and the portable communication terminal that opens the corresponding mount displays the title of the mount, the cache of the mount data, and the data amount of the mount data as the value. And so on. Then, the correspondence determination processing unit 11 determines whether the key and value match or correspond to each other.

  When portable communication terminals are combined, it may be determined whether the communication partner is a valid communication partner depending on whether the combination position corresponds. In this case, as the key, a message indicating that the function for combining the portable communication terminals is in operation is sent, and the portable communication terminal that is activating the same function returns that value as value. Then, the position of the selected combination (upper, lower, left and right diagonal (for example, right) is sent as a key, and the portable communication terminal for which the corresponding position (for example, left) is selected as the combination returns that as value.

  The “ad hoc mode” is set so that the radio wave can reach only within a predetermined range, for example, within a range of several meters, so that broadcast distribution is not performed over a wide range.

  As described above, a direct communication connection can be established between portable communication terminals. That is, the portable communication terminal A to the portable communication terminal D can directly communicate with each other. This is schematically shown in FIG.

  If the inquiry information distribution processing unit 10 of the portable communication terminal A has not received the value information within a predetermined time after the key information has been distributed, the portable communication terminal It is determined that the terminal is not the target of establishment.

  The inquiry information distribution processing unit 10 performs a plurality of processes for broadcasting a key and receiving a value for the key in order for the correspondence determination processing unit 11 to determine whether the portable communication terminal that has received the value is a valid communication partner. It may be repeated times. In this case, different keys may be distributed.

  Next, in order to specify the combination direction of the portable communication terminals, the combination position determination processing unit 13 of each portable communication terminal specifies the combination direction. In this case, the combination direction is displayed on the display device of the portable communication terminal, and the selection is accepted, whereby the combination with the portable communication terminal in the corresponding combination direction is performed. For example, if the portable communication terminal A accepts the selection on the right as the combination direction, the portable communication terminal A is on the left and the portable communication terminal B is on the right if the portable communication terminal B accepts the left selection. Combined in relationship. Also, when the lower selection is accepted as the combination direction in the portable communication terminal A, if the upper selection is accepted in the portable communication terminal C, the portable communication terminal A is on the upper side, and the portable communication terminal C is on the lower side. Are combined in a positional relationship. When the portable communication terminal B accepts the lower selection as the combination direction, if the portable communication terminal D accepts the upper selection, the portable communication terminal B is the upper position and the portable communication terminal D is the lower position. Combined in relationship. As described above, the combination position determination processing unit 13 accepts the selection of the combination direction of the portable communication terminal, whereby the combination positional relationship can be set.

  In the connection establishment process, when information on the combination direction is used as the key and value, the positional relationship of the combination can be set simultaneously with the connection establishment process.

  FIG. 1B shows a state where the portable communication terminals are combined as described above. Note that the information in which the positional relationship is set is shared by transmitting and receiving information between each of the combined portable communication terminals A to D. Thereby, the positional relationship between the origin of its own portable communication terminal and the origin of physical coordinates can be specified.

  When the positional relationship of each portable communication terminal is specified in this way, the display processing unit 20 of each portable communication terminal uses physical coordinates (absolute coordinates) when displaying as one display device based on the specified positional relationship. And the positional relationship between the client coordinates (relative coordinates) in the display device of each portable communication terminal. In particular, the physical coordinate and the client coordinate conversion formula are generated by specifying the positional relationship between the physical coordinate origin and the relative coordinate origin in the display device of each portable communication terminal.

  Next, since it is necessary to align the date and time information between the portable communication terminals having established communication connections, the timing synchronization processing unit 14 of each portable communication terminal synchronizes the date and time information between the portable communication terminals. Processing is executed (S120). In this case, based on one of the portable communication terminals that have established a communication connection, the date / time information is also transmitted to other portable communication terminals, and the portable communication terminal that receives the date / time information receives the date / time information. Update.

  For example, when aligning the date and time information of the portable communication terminal A, the timing synchronization processing unit 14 of the portable communication terminal A sets its own date and time information “September 20, 2013, 12:00:36” to the portable communication terminal B. To the portable communication terminal D. The timing synchronization processing unit 14 of the portable communication terminal B to the portable communication terminal D receives the date and time information “September 20, 2013, 12:00:36” sent from the portable communication terminal A, and receives its own date and time. The information is updated to the received date and time information “September 20, 2013 12:00:36”.

  In the above, since the connection establishment process is performed when the portable communication terminal A to the portable communication terminal D are substantially the same, the date / time information is transferred from the portable communication terminal A to the portable communication terminal B to the portable communication terminal D. Although the case where transmission is performed and synchronization processing is executed has been described, when different times are performed, for example, after the synchronization processing of date and time information is performed in the portable communication terminal A and the portable communication terminal B, the portable communication terminal B and the portable communication terminal C may perform date / time information synchronization processing.

  The timing synchronization processing is schematically shown in FIG.

  When the date and time information synchronization processing is performed in this way by the timing synchronization processing unit, one display can be performed by bringing the combined portable communication terminals next to each other (the state in FIG. 1B). . This process will be described below.

  When the portable communication terminal A to the portable communication terminal D are combined as shown in FIGS. 1B and 9, for example, an operation “AAA” is performed on the touch panel display device 72 of the portable communication terminal A. Then, the operation input acceptance processing unit 15 accepts the operation “AAAA” (S130). Then, the operation input reception processing unit 15 converts the client coordinates where the received operation “AAAA” is performed into absolute coordinates, and generates operation log information illustrated in FIG. 10. For example, when the operation “AAAA” is performed on the client coordinates (cx1, cy1) = (100, 100), the operation is changed to the absolute coordinates (X, Y) = (100, 100) and includes the operation Log information is generated (S140). The operation input reception processing unit 15 stores the generated operation log information in the operation log information storage unit 16.

  And the operation log information transmission process part 17 of the portable communication terminal A transmits the produced | generated operation log information to the portable communication terminal B thru | or the portable communication terminal D with which the connection is established (S150). The operation log information sent in S140 is received by the operation log information reception processing unit 18 of the portable communication terminal B to the portable communication terminal D (S160). This is schematically shown in FIG.

  When the operation log information is received, the operation log information is stored in the operation log information storage unit 16 of each of the portable communication terminal B to the portable communication terminal D, and the operation execution processing unit 19 executes the corresponding operation process. The operation process is executed on all connected terminals. Then, each display processing unit 20 of the portable communication terminal B to the portable communication terminal D converts physical coordinates in the received operation log information into respective client coordinates (S170). For example, the display processing unit 20 of the portable communication terminal B converts the client coordinates (cx2, cy2) = (− 400, 100), and the display processing unit 20 of the portable communication terminal C converts the client coordinates (cx3, cy3) = ( 100, -200), and the display processing unit 20 of the portable communication terminal D converts the client coordinates (cx4, cy4) = (-400, -200). Since the portable communication terminal A originally accepts input in client coordinates, it can be used. This is schematically shown in FIG.

  Then, the operation “AAAA” is not within the display range of the respective touch panel type display devices 72 in the portable communication terminal B to the portable communication terminal D. However, since the operation “AAAA” is a process that does not depend on coordinates, each of the portable communication terminals A to D executes a display process corresponding to the operation “AAAA” (S180).

  Next, it is assumed that the operation “BBBB” is performed at the client coordinates (100, 200) of the touch panel type display device 72 of the portable communication terminal B. The operation input reception processing unit 15 of the portable communication terminal B uses the client coordinates (cx2, cy2) = (100, 200) where the received operation “BBBB” is performed, and the physical coordinates (X, Y) = (600, 200), and operation log information including it is generated (S130, S140). The operation input reception processing unit 15 of the portable communication terminal B stores the generated operation log information in the operation log information storage unit 16.

  Then, the operation log information transmission processing unit 17 of the portable communication terminal B transmits the generated operation log information to the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D with which the connection is established. (S150). This is schematically shown in FIG. The operation log information sent in S140 is received by the operation log information reception processing unit 18 of the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D (S160).

  When the operation log information is received, the operation log information is stored in the operation log information storage unit 16 of each of the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D, and an operation corresponding to the operation log information is performed by the operation execution processing unit 19. Execute the process. The operation process is executed on all connected terminals. Then, each display processing unit 20 of the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D converts the physical coordinates in the received operation log information into respective client coordinates (S170). For example, the display processing unit 20 of the portable communication terminal A converts the client coordinates (cx1, cy1) = (600, 200), and the display processing unit 20 of the portable communication terminal C converts the client coordinates (cx3, cy3) = (600 , −100), the display processing unit 20 of the portable communication terminal D converts the client coordinates (cx4, cy4) = (100, −100). This is schematically shown in FIG.

  Then, the operation “BBBB” is not within the display range of the touch panel display device 72 of each of the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D. However, since the operation “BBBB” is a process that does not depend on coordinates, each portable communication terminal A to portable communication terminal D executes a process corresponding to the operation “BBBB” (S180).

  By repeating the process as described above a plurality of times, the operation log information matches in the portable communication terminal A to the portable communication terminal D. And since it displays on each portable communication terminal based on operation log information, each operation can be reflected quickly. In addition, since it does not depend on the server, there is no processing delay and no processing concentration on the server.

  In the display process in the display processing unit 20, display is performed by performing layer management based on the operation log information. That is, layer management is performed so that objects in each operation are displayed at the top in the order of operation log information stored in the operation log information storage unit 16. The concept of this layer management is shown in FIGS. FIG. 27 is a diagram schematically showing the management of the upper and lower layers, and FIG. 28 is a diagram schematically showing the display on the touch panel type display device 72.

  In this way, all display of images is managed on the basis of operation log information, so display synchronization can be easily realized between the combined portable communication terminals, and there is no display shift. . In addition, it is not necessary to separately provide a component for managing layer information of drawings, and image display can be realized in a simpler form as compared with a conventional image display system.

  Next, in the second embodiment, processing in the case of depending on coordinates will be described. For example, a case will be described in which an image displayed on one portable communication terminal is moved to another portable communication terminal and displayed. FIG. 15 schematically shows a conceptual diagram in this case. In FIG. 15, in order to facilitate understanding of the coordinate relationship of the touch panel type display device 72 of the portable communication terminal, portions other than the touch panel type display device 72 of the portable communication terminal are omitted. In the following embodiment, as shown in FIG. 15, a case will be described in which the image information displayed on the portable communication terminal A is moved to the vicinity of the center of the four portable communication terminals combined. It is assumed that the portable communication terminal A to the portable communication terminal D have already performed connection establishment processing (S100, S110) and date / time synchronization processing (S120) in the same processing as in the first embodiment.

  First, the user of the portable communication terminal A performs an operation of displaying the image information “Flower1.jpg” on the portable communication terminal A. This operation input is received by the operation input reception processing unit 15 of the portable communication terminal A (S130), and operation log information as shown in FIG. 16 is generated (S140). In the operation log information of FIG. 16, the operation target is “display” with respect to “Flower1.jpg”. In this case, physical coordinates (Width) indicating the width of the image information are (300, 400), high It is shown that the physical coordinates indicating the height are (50, 250). As described above, the operation input is accepted in the client coordinates (relative coordinates) in the portable communication terminal A, but the operation log information is generated in the physical coordinates (absolute coordinates). After conversion to coordinates, it is included in the operation log information as physical coordinates.

  When the operation input reception processing unit 15 of the portable communication terminal A generates the operation log information in this way, the operation log information storage unit 16 stores the operation log information. This state is shown in FIG.

  Then, the operation log information transmission processing unit 17 of the portable communication terminal A uses the operation log information generated by the operation input reception processing unit 15 as the portable communication terminal B to the portable communication terminal combined with the portable communication terminal A. Transmit to terminal D (S150). At this time, image information to be processed is also sent. That is, the operation log information of FIG. 16 is transmitted.

  Each operation log information reception processing unit 18 of the portable communication terminal B to the portable communication terminal D receives the operation log information sent from the portable communication terminal A (S160). Then, each operation execution processing unit 19 of the portable communication terminal B to the portable communication terminal D executes an operation process corresponding to the operation log information, and each display processing unit 20 performs an operation based on the received operation log information. The physical coordinates in the log information are converted into the client coordinates of each portable communication terminal (S170).

  In the case of FIG. 16, since the physical coordinates in the operation log information are Width (X1, X2) = (300, 400) and Height (Y1, Y2) = (50, 250), the display processing unit 20 of the portable communication terminal B is As client coordinates, Width (cx21, cx22) = (− 200, −100), Height (cy21, cy22) = (50, 250), the display processing unit 20 of the portable communication terminal C uses Width (cx31) as client coordinates. , Cx32) = (300, 400), Height (cy31, cy32) = (− 250, −50), the display processing unit 20 of the portable communication terminal D uses the client coordinate Width (cx41, cx42) = (− 200, − 100), Height (cy41, cy42) = (− 250, −50).

  If it does so, since it is outside the display range of each touch panel type display device 72 of portable communication terminal B thru / or portable communication terminal D, each display processing part 20 does not display. Therefore, the display processing unit 20 of the portable communication terminal A performs display on the touch panel display device 72 of the portable communication terminal A (S180).

  Next, the user performs an operation of moving the image “Flower1.jpg” displayed on the portable communication terminal A to the position shown in FIG. This operation input is received by the operation input reception processing unit 15 of the portable communication terminal A (S130), and a moving operation is performed. Then, the operation input reception processing unit 15 generates operation log information as shown in FIG. 19 (S140). When the operation log information is generated, the operation log information is stored in the operation log information storage unit 16.

  Further, the operation log information transmission processing unit 17 of the portable communication terminal A transmits the operation log information to the portable communication terminal B to the portable communication terminal D combined with the portable communication terminal A. (S150).

  Each operation log information reception processing unit 18 of the portable communication terminal B to the portable communication terminal D receives the operation log information sent from the portable communication terminal A (S160). Then, each operation execution processing unit 19 of the portable communication terminal B to the portable communication terminal D executes an operation process corresponding to the operation log information, and each display processing unit 20 performs an operation based on the received operation log information. The physical coordinates in the log information are converted into the client coordinates of each portable communication terminal (S170).

  In the case of FIG. 19, since the physical coordinates in the operation log information are Width (X1, X2) = (450, 550) and Height (Y1, Y2) = (200, 400), the display processing unit 20 of the portable communication terminal B is As client coordinates, Width (cx21, cx22) = (− 50, 50), Height (cy21, cy22) = (200, 400), the display processing unit 20 of the portable communication terminal C uses Width (cx31, cx32) = (450, 550), Height (cy31, cy32) = (− 100, 100), the display processing unit 20 of the portable communication terminal D uses the client coordinates Width (cx41, cx42) = (− 50, 50), Height (cy41, cy42) = (− 100, 100) is calculated.

  Then, the display processing unit 20 of the portable communication terminal A displays the display of the portable communication terminal B in the range of Width (cx11, cx12) = (450, 500), Height (cy11, cy12) = (200, 300). The processing unit 20 is in the range of Width (cx21, cx22) = (0, 50), Height (cy21, cy22) = (200, 300), and the display processing unit 20 of the portable communication terminal C is Width (cx31, In the range of cx32) = (450,500), Height (cy31, cy32) = (0, 100), the display processing unit 20 of the portable communication terminal D has Width (cx41, cx42) = (0, 50), Since display is possible in the range of Height (cy41, cy42) = (0, 100), the corresponding area is displayed. (S180).

  That is, the display processing unit 20 of the portable communication terminal A has the image information “Flower1.jpg” in the area of Width (cx11, cx12) = (450, 500), Height (cy11, cy12) = (200, 300). The range of physical coordinates Width (X1, X2) = (450, 500), Height (Y1, Y2) = (200, 300) is displayed. In addition, the display processing unit 20 of the portable communication terminal B has the image information “Flower1.jpg” in the area of Width (cx31, cx32) = (0, 50), Height (cy31, cy32) = (200, 300). A range of physical coordinates Width (X1, X2) = (500, 550), Height (Y1, Y2) = (200, 300) is displayed. In addition, the display processing unit 20 of the portable communication terminal C has the image information “Flower1.jpg” in the area of Width (cx21, cx22) = (450,500), Height (cy21, cy22) = (0, 100). The range of physical coordinates Width (X1, X2) = (450, 500), Height (Y1, Y2) = (300, 400) is displayed. In addition, the display processing unit 20 of the portable communication terminal D has the image information “Flower1.jpg” in the area of Width (cx41, cx42) = (0, 50), Height (cy41, cy42) = (0, 100). ”Physical coordinates Width (X1, X2) = (500, 550), Height (Y1, Y2) = (300, 400).

  By executing the processing as described above in the display system 1 of each portable communication terminal, a plurality of portable communication terminals can be combined to form one display device. In addition, processing is not performed mainly by a specific portable communication terminal, but the portable communication terminal where the operation is performed sends information including operation log information to other portable communication terminals. The information is shared and made uniform between the portable communication terminals, and only the corresponding area is displayed on each portable communication terminal. For this reason, the processing is decentralized, and the concentration of processing does not occur in a specific portable communication terminal.

  Moreover, since it is the structure which communicates directly between portable communication terminals, a server becomes unnecessary. Therefore, group learning can be performed even in an environment where communication with the server is not possible, for example, outdoors without a wi-fi environment.

  In FIG. 15 described above, since the moving operation of the image information “Flower1.jpg” is performed in the portable communication terminal A, the user displays the image information on the touch panel display device 72 of the portable communication terminal A. After the operation of selecting and moving was completed, the end of the moving operation was detected, and the operation input reception processing unit 15 generated operation log information.

  However, in actual processing, as shown in FIG. 24, for example, an operation for moving image information from the touch panel display device 72 of the portable communication terminal A to the touch panel display device 72 of the portable communication terminal B may be performed. In this case, a series of movement operations are performed on the physical coordinates, but actually, as shown in FIG. 1B, the touch panel display device 72 continuously moves in different portable communication terminals. Rather, there is a frame of a portable communication terminal. Therefore, as shown in FIG. 25, the operation input reception processing unit 15 detects the end of the operation input when it is out of the frame, once generates operation log information, and transmits it to each portable communication terminal. Then, at the stage of entering the frame, the start of the operation input is detected again. After the movement is completed, the operation log information is generated and transmitted to each portable communication terminal. This is schematically shown in FIG.

  That is, when a movement operation is performed as shown in FIG. 25, when the operation input reception processing unit 15 of the portable communication terminal A detects the movement operation, the input device (such as a finger or a touch pen) is moved to the frame of the portable communication terminal A. When the position reaches the position, operation log information is generated assuming that the moving operation is completed. Then, the operation log information transmission processing unit 17 of the portable communication terminal A transmits to the portable communication terminal B to the portable communication terminal D. Further, the operation input reception processing unit 15 of the portable communication terminal B generates operation log information on the assumption that the moving operation has started when the input device reaches the frame position of the portable communication terminal B. Then, the operation log information transmission processing unit 17 of the portable communication terminal B transmits to the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D.

  By such processing, it is possible to execute processing across the touch panel display device 72 of the portable communication terminal.

  Next, a case where the display size of the image information displayed on the portable communication terminal is changed will be described. Here, a case where the display size is changed with respect to the image information in the state of the second embodiment will be described. FIG. 20 schematically shows this processing.

  The user of the portable communication terminal D performs an operation for enlarging the image information displayed on the portable communication terminal D. This operation input is received by the operation input reception processing unit 15 of the portable communication terminal D (S130). Then, operation log information as shown in FIG. 21 is generated (S140). In the operation log information of FIG. 21, the operation content “size change” is performed on the operation target “Flower1.jpg”. In this case, physical coordinates (Width) indicating the width of the image information are (450, 700), The physical coordinates indicating the height are (200, 550).

  When the operation input reception processing unit 15 of the portable communication terminal D generates the operation log information in this way, the operation log information storage unit 16 stores the operation log information.

  Then, the operation log information transmission processing unit 17 of the portable communication terminal D uses the operation log information generated by the operation input reception processing unit 15 as the portable communication terminal A to the portable communication terminal combined with the portable communication terminal D. It transmits to terminal C (S150).

  Each operation log information reception processing unit 18 of the portable communication terminal A to the portable communication terminal C receives the operation log information sent from the portable communication terminal D (S160). Then, each operation execution processing unit 19 of the portable communication terminal A to the portable communication terminal C executes an operation process (a process for changing the display size) corresponding to the operation log information, and each display processing unit 20 receives Based on the operation log information, physical coordinates in the operation log information are converted into client coordinates of each portable communication terminal (S170).

  In the case of FIG. 21, since the physical coordinates in the operation log information are Width (X1, X2) = (450, 700) and Height (Y1, Y2) = (200, 550), the display processing unit 20 of the portable communication terminal A is As client coordinates, Width (cx11, cx12) = (450, 700), Height (cy11, cy12) = (200, 550), the display processing unit 20 of the portable communication terminal B uses Width (cx21, cx22) as client coordinates. ) = (− 50, 200), Height (cy21, cy22) = (200, 550), the display processing unit 20 of the portable communication terminal C has client coordinates, Width (cx31, cx32) = (450, 700), Height (cy31, cy32) = (− 100, 250) is calculated.

  Then, the display processing unit 20 of the portable communication terminal A displays the display of the portable communication terminal B in the range of Width (cx11, cx12) = (450, 500), Height (cy11, cy12) = (200, 300). The processing unit 20 is in the range of Width (cx21, cx22) = (0, 200), Height (cy21, cy22) = (200, 300), and the display processing unit 20 of the portable communication terminal C is Width (cx31, In the range of cx32) = (450,500), Height (cy31, cy32) = (0,250), the display processing unit 20 of the portable communication terminal D has Width (cx41, cx42) = (0,200), Since each display is possible within the range of Height (cy41, cy42) = (0, 250), the corresponding area is displayed.(S180).

  That is, the display processing unit 20 of the portable communication terminal A has the image information “Flower1.jpg” in the area of Width (cx11, cx12) = (450, 500), Height (cy11, cy12) = (200, 300). The range of physical coordinates Width (X1, X2) = (450, 500), Height (Y1, Y2) = (200, 300) is displayed. Further, the display processing unit 20 of the portable communication terminal B has the image information “Flower1.jpg” in the area of Width (cx21, cx22) = (0, 200), Height (cy21, cy22) = (200, 300). A range of physical coordinates Width (X1, X2) = (500, 700), Height (Y1, Y2) = (200, 300) is displayed. In addition, the display processing unit 20 of the portable communication terminal C includes the image information “Flower1.jpg” in the area of Width (cx31, cx32) = (450, 500), Height (cy31, cy32) = (0, 250). The range of physical coordinates Width (X1, X2) = (450, 500), Height (Y1, Y2) = (300, 550) is displayed. In addition, the display processing unit 20 of the portable communication terminal D has the image information “Flower1.jpg” in the area of Width (cx41, cx42) = (0, 200), Height (cy41, cy42) = (0, 250). ”Physical coordinates Width (X1, X2) = (500, 700), Height (Y1, Y2) = (300, 550).

  By executing the above process on the display system 1 of each portable communication terminal, even if the display size is changed, the display can be performed as one display device.

  In the present embodiment, processing when operations conflict will be described. Here, contention refers to a case where a plurality of operations are performed at the same or different portable communication terminals. In the present invention, as described above, operation log information is generated for an operation performed at a certain portable communication terminal, and transmitted to other operation log information. Each portable communication terminal performs display based on the generated or received operation log information.

  For example, assume that the size change operation of the image information “Flower1.jpg” is performed at the portable communication terminal B, and the movement operation of the image information “Flower1.jpg” is performed at almost the same timing in the portable communication terminal C. In this case, the operation input information processing unit 15 of the portable communication terminal B has the operation log information shown in FIG. 22A, and the operation input reception processing unit 15 of the portable communication terminal C has the operation log shown in FIG. Each piece of information is generated.

  Then, the operation log information transmission processing unit 17 of the portable communication terminal B transmits the operation log information shown in FIG. 22A to the portable communication terminal A, the portable communication terminal C, and the portable communication terminal D. The operation log information transmission processing unit 17 of the communication terminal C transmits the operation log information shown in FIG. 22B to the portable communication terminal A, the portable communication terminal B, and the portable communication terminal D, respectively. Each operation log information reception processing unit 18 of the portable communication terminal A to the portable communication terminal D receives the operation log information transmitted from the portable communication terminal B and the portable communication terminal C, and receives an operation log information storage unit. 16, the operation execution processing unit 19 and the display processing unit 20 perform processing in order from the operation log information of the old date and time information among the operation log information stored in the operation log information storage unit 16. FIG. 23 schematically shows an example of operation log information stored in the operation log information storage unit 16 of each portable communication terminal.

  As described above, in the display processing system according to the present invention, display processing is performed based on the operation log information, so that in principle there is no conflict. In the case of operation log information having the same date and time information, the operation log information is processed in order from the earliest stored. Therefore, all operation log information can be processed appropriately.

  In each embodiment, the connection between portable communication terminals may be temporarily disconnected. In that case, the portable communication terminal that detects disconnection requests the other portable communication terminals to transmit the operation log information until after disconnection after detecting the recovery of the connection. For example, an operation log information transmission request and disconnected date / time information are transmitted to another portable communication terminal.

  The other portable communication terminal that has received the request extracts the operation log information after the disconnected date and time information from the operation log information storage unit 16 and returns it to the portable communication terminal that sent the transmission request. The portable communication terminal that has sent the transmission request stores the sent operation log information in the operation log information storage unit 16. Then, the corresponding processing is executed by the operation execution processing unit 19 and displayed by the display processing unit 20.

  As a result, the operation log information can always be matched between the portable communication terminals, so that even if the connection is temporarily disconnected, it can be recovered immediately.

  In the fifth embodiment, the case where the connection between the portable communication terminals is temporarily disconnected has been described. However, some of the connected portable communication terminals may disconnect the connection themselves. In this case, the disconnected portable communication terminal executes its own editing work.

  When the disconnected portable communication terminal performs connection again, the connection establishment process is performed again by executing the same process as in the above-described embodiments. When the connection establishment processing unit 12 of one of the connected portable communication terminals recognizes reconnection with a new portable communication terminal, the operation log information transmission processing unit 17 of the terminal The operation log information stored in the operation log information storage unit 16 is extracted and transmitted to the portable communication terminal that has established the reconnection.

  The operation log information reception processing unit 18 of the reconnected portable communication terminal stores the received operation log information in the operation log information storage unit 16, and the operation execution processing unit 19 performs an operation process corresponding to the operation log information. Display processing in the display processing unit 20 is executed. By executing such processing, operation log information can be synchronized even when reconnection is performed, so each connected portable communication terminal is treated as a single display device. Can function.

  With the processing of the sixth embodiment, even if the connection relationship is canceled from the group collaborative work state and the individual editing work is performed and then the group collaborative work is performed again, the smooth operation can be performed. Work migration can be realized. In other words, the disconnected portable communication terminal notifies the other portable communication terminals that have reconnected the operation log information executed from the disconnection to the reconnection and the object information related to the operation log information after the reconnection. It is only necessary to do so, and the amount of data to be communicated can be minimized.

  On the other hand, when using a server, the operation log information sent from the reconnected portable communication terminal to the server and the related object information are combined with the operation log information and object information of other portable communication terminals. Since it is necessary to report the merged contents to all connected portable communication terminals, the amount of data to be communicated increases. In other words, in the case of a system via a server, each portable communication terminal sends its editing results to the server once during reconnection processing, and performs integrated processing on the server. It is necessary to wait for the distribution of the image as a result of integrating the operations for each portable communication terminal.

  On the other hand, according to the present invention, it is possible to smoothly realize the reconnection to the collaborative work of the group only by performing the process of sharing the information added during the individual editing work or the operation log information. Specifically, in this embodiment, since the operation log information is already stored in each portable communication terminal, the operation log information executed by other portable communication terminals during individual editing operations, etc. The process is executed at each portable communication terminal according to the timing synchronized at the time of reconnection, so that the entire data to be operated can be processed without performing the process of distributing the entire data to be operated. Can be shared. Thereby, it is possible to shift between individual editing work and group cooperation work at an arbitrary timing.

  In particular, in the education field of schools, in general, teachers give instructions on the next action to learners at any time according to the progress of classes and the progress of learning. At this time, as in the present embodiment, the smooth transition of work at an arbitrary timing (particularly between the individual editing work and the collaborative work of the group without shifting the work continuity) ) Can be said to have a great effect in terms of ease of use for teachers who conduct classes.

  By using the display system 1 of the present invention, no load is applied to a specific portable communication terminal, and when one screen is displayed on a plurality of portable communication terminals, which portable communication terminal Even if editing is done on the screen, it can be reflected.

1: display system 10: inquiry information distribution processing unit 11: correspondence determination processing unit 12: connection establishment processing unit 13: combination position determination processing unit 14: timing synchronization processing unit 15: operation input reception processing unit 16: operation log information storage Unit 17: Operation log information transmission processing unit 18: Operation log information reception processing unit 19: Operation execution processing unit 20: Display processing unit 70: Computing device 71: Storage device 72: Touch panel type display device 73: Communication device

Claims (6)

A display system combining a plurality of portable communication terminals and displaying as one display device,
A combination position determination processing unit that identifies a portable communication terminal that is a communication partner of the portable communication terminal and determines a combination position with the portable communication terminal;
An operation input reception processing unit that receives operation input in a portable communication terminal and generates operation log information;
An operation log information transmission processing unit for directly transmitting the generated operation log information to the portable communication terminal of the communication partner;
An operation log information reception processing unit for directly receiving operation log information based on an operation input at the portable communication terminal from the communication partner portable communication terminal;
An operation execution processing unit that executes an operation corresponding to the generated operation log information or the received operation log information;
A display processing unit that executes display processing of an area corresponding to the determined combination position;
A display system comprising: The display system is:
A reference information distribution processing unit that distributes predetermined reference information upon receiving input of connection establishment processing start for direct communication between portable communication terminals;
A correspondence determination processing unit that receives response information sent from a portable communication terminal of a communication partner corresponding to the inquiry information, and determines whether the inquiry information and the response information match or correspond;
A connection establishment processing unit that establishes a connection with the communication partner portable communication terminal based on the determination result;
The display system according to claim 1, further comprising: The combination position determination processing unit
Based on the determination result by the correspondence determination processing unit, the portable communication terminal of the communication partner is specified, and the combination position with the portable communication terminal is determined.
The display system according to claim 2. The display system is:
A timing synchronization processing unit for transmitting timing information for synchronizing timing between portable communication terminals to the communication partner portable communication terminal;
The display system according to claim 1, further comprising: The display system is:
An operation log information storage unit that stores operation log information generated by the operation input reception processing unit and / or operation log information received by the operation log information reception processing unit,
The operation log information transmission processing unit
When the connection establishment processing unit establishes a connection with a portable communication terminal of a new communication partner, the operation log information stored in the operation log information storage unit before the connection is established can be stored in the new communication partner. Send to portable communication terminal,
The display system according to claim 1, wherein the display system is a display system. A display program for a display system that displays a single display device by combining a plurality of portable communication terminals,
The display program displays the portable communication terminal,
A combination position determination processing unit that identifies a portable communication terminal that is a communication partner of the portable communication terminal and determines a combination position with the portable communication terminal;
An operation input reception processing unit that receives operation input in a portable communication terminal and generates operation log information,
An operation log information transmission processing unit for directly transmitting the generated operation log information to the portable communication terminal of the communication partner;
An operation log information reception processing unit for directly receiving operation log information based on an operation input at the portable communication terminal from the communication partner portable communication terminal;
An operation execution processing unit that executes an operation corresponding to the generated operation log information or the received operation log information;
A display processing unit that executes display processing of an area corresponding to the determined combination position;
Display program characterized by functioning as

Images