JP2015184876A - Terminal device, cooperation method of portable terminal device, information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To cooperatively operate a plurality of terminals while suppressing power consumption of the terminals by a simple operation.SOLUTION: A terminal device which cooperatively operates with another terminal device includes a communication part for communicating with the other device, and a processing part for transmitting operation information showing contents of an operation when the number of operations from the start of the operation of an input device for receiving an operation to an end part of an operation area till interruption becomes equal to or more than the prescribed number of times, and executing a cooperative operation with the other terminal device when receiving from the other terminal device cooperation information generated on the basis of collation between the operation information and the operation information transmitted to show that the terminal device and the other terminal device cooperatively operate.

Description

  The present invention relates to a terminal device, a terminal device cooperation method, an information processing device, and a program.

  A technique for operating a plurality of terminal devices (hereinafter referred to as terminals as appropriate) in cooperation with each other has been proposed. As the cooperative operation, for example, there is an operation in which content displayed on the tablet terminal is simultaneously displayed (also referred to as shared display) on the display screen of another tablet terminal that operates in cooperation.

JP 2011-198316 JP 2011-227821 A Special table 2012-502374 JP 2000-98989 A

  In the conventional method, an instruction operation for operating a plurality of terminals in cooperation is complicated. For example, the conventional method requires an instruction operation to place a plurality of terminals adjacent to each other and shift the plurality of terminals from the normal input mode to the cooperation start mode. The conventional method further requires an instruction operation to perform a cooperative operation after shifting to the cooperative start mode.

  Furthermore, when a plurality of terminals that operate in a coordinated manner are driven by a rechargeable battery, for example, if the power consumed during an instruction operation for performing a coordinated operation is reduced, it is possible to suppress a reduction in drive time. However, the conventional method does not consider any reduction in power consumption.

  One aspect of the present embodiment is to allow a plurality of terminals to operate in a coordinated manner while suppressing power consumption of the terminals by a simple operation.

  A first aspect of the present embodiment is a terminal device that operates in cooperation with another terminal device, and the terminal device includes a communication unit that communicates with another device and an end of an operation region of an input device that receives an operation. The operation information indicating the content of the operation is transmitted when the number of operations from the start of the operation to the unit until the operation is interrupted exceeds a predetermined number, and the operation information transmitted from the other terminal device And receiving a link information generated based on the collation and indicating that the terminal device and the other terminal device operate in cooperation with each other, the processing unit executes a cooperative operation with the other terminal device.

  According to the first aspect, it is possible to operate a plurality of terminals in a coordinated manner with a simple operation while suppressing power consumption of the terminals.

FIG. 1 is an external view of a terminal described in the first embodiment. FIG. 2 is a diagram for schematically explaining the instruction operation of the cooperative operation of the terminal. FIG. 3 is a diagram for explaining inter-terminal cooperation in the first embodiment. FIG. 4 is a hardware block diagram of the first terminal 1 or the second terminal 2 in the first embodiment. FIG. 5 is a hardware block diagram of the server 3 according to the first embodiment. FIG. 6 is a software block diagram of the first terminal 1 and the second terminal 2 in the first embodiment. FIG. 7 is a software block diagram of the server 3 according to the first embodiment. FIG. 8 is a first flowchart illustrating processing executed by the first terminal 1 and the second terminal 2 to determine whether or not it is an opportunity to shift the two terminals to the linked state. FIG. 9 is a diagram showing an example of determination information in the first terminal 1. FIG. 10 is a diagram illustrating an example of determination information in the second terminal 2. As illustrated in FIG. FIG. 11 is a flowchart for explaining the process executed by the server 3 to determine the linked terminal. FIG. 12 is a flowchart for explaining the process of determining the cooperation terminal executed by the first terminal 1 and the second terminal 2. FIG. 13 is a second flowchart for explaining processing for determining the timing of terminal cooperation, which is executed by the first terminal 1 and the second terminal 2. FIG. 14 is a software block diagram of the first terminal 1 or the second terminal 2 in the third embodiment. FIG. 15 is a software block diagram of the server 3 according to the third embodiment. FIG. 16 is a flowchart for explaining the transmission process of the passing position designation information executed by the server 3. FIG. 17 is a flowchart for explaining guide display processing executed by the first terminal 1 and the second terminal 2. FIG. 18 is a diagram for explaining guide display processing in the first terminal 1 and the second terminal 2. FIG. 19 is a software block diagram of the first terminal 1 provided with the function of the server 3.

[First Embodiment]
(Device description)
FIG. 1 is an external view of a terminal described in this embodiment. This terminal is a portable computer such as a so-called tablet terminal, a smartphone, or a mobile phone.

  FIG. 1 (A) is a diagram schematically showing the display surface side of the first terminal 1 operating in cooperation with other terminals. On the display surface side of the first terminal 1, a display surface DSP and a housing BDY are provided. The display surface DSP is a white portion, and the housing BDY is a portion indicated by hatching with diagonal lines. The display surface DSP has a display area having a horizontal width L1 and a vertical width L2. This display area is an operation area for receiving an operation, for example.

  A touch panel (not shown) is provided on the display surface DSP. The operator operates the first terminal 1 by touching the display surface DSP with a finger or an operating instrument (for example, a stylus).

  FIG. 1B is a diagram showing an end R of the display surface DSP described in FIG. In FIG. 1 (B), the end R on the housing BDY side of the display surface DSP is indicated by dot hatching. The lateral width L3 on the left and right sides of the end R is, for example, 5% of the lateral width L1. The vertical width L4 at the upper and lower sides of the end R is 5% of the vertical width L2. Note that the lengths of the widths L1 to L4 are exaggerated.

(Instruction operation for cooperative operation of terminals)
FIG. 2 is a diagram for schematically explaining the instruction operation of the cooperative operation of the terminal. The operator of the terminal performs an instruction operation for the cooperative operation in order to cause the first terminal 1 and the second terminal 2 to cooperate. The second terminal 2 shown in FIG. 2 also has the same appearance and function as the first terminal 1 described in FIG.

  The operator places the first terminal 1 and the second terminal 2 adjacent to each other when performing an instruction operation of the cooperative operation. For example, as shown in FIG. 2, the distance between the side wall of the casing BDY in the vertical direction of the first terminal 1 and the side wall of the casing BDY in the vertical direction of the second terminal 2 is a predetermined distance. In this state, the first terminal 1 and the second terminal 2 are arranged so as to be within the distance L5. Here, the predetermined distance L5 is, for example, 0 mm to 5 mm. In addition, the term “adjacent” means that the distance between the side wall of the casing BDY in the lateral direction of the first terminal 1 and the side wall of the casing BDY in the lateral direction of the second terminal 2 is within the predetermined distance L5. In this state, the first terminal 1 and the second terminal 2 are arranged. In addition, the term “adjacent” means that the distance between the side wall of the casing BDY in the vertical direction of the first terminal 1 and the side wall of the casing BDY in the horizontal direction of the second terminal 2 is within a predetermined distance L5. In this state, the first terminal 1 and the second terminal 2 are arranged.

  That is, the operator places the first terminal 1 and the second terminal 2 so that the display surface DSP of the first terminal 1 and the display surface DSP of the second terminal 2 are arranged adjacent to each other. Deploy.

  Next, the operator performs a scratch operation on the end R of the first terminal 1 and the end R of the second terminal 2 with a finger or an operating tool.

(Scratch operation)
The scratch operation will be described with reference to FIG. An example of the scratch operation is that the operator performs a zigzag contact operation on the display surface DSP of the first terminal 1 and the display surface DSP of the second terminal 2, for example. In FIG. 2, for example, the operator touches a point P1 on the display surface DSP of the first terminal 1 and keeps this contact, and slides the finger toward the point P2 in the direction of the arrow AR1 (see ST11). .

  Further, the operator slides his / her finger from the display screen DSP of the first terminal 1 to the display screen DSP of the second terminal 2 (see symbols ST11 and ST21). Then, the operator slides the finger along the points P3 and P4, turns back at the point P4, and moves the finger to the point P5 (see reference ST21).

  Then, the operator slides his / her finger toward the first terminal 1, moves the points P6 and P7 and the finger, turns back at the point P7, and moves the finger to the point P8 (see reference ST12).

  Thereafter, the operator slides his / her finger from the display surface DSP of the first terminal 1 to the display surface DSP of the second terminal 2 in the direction of the arrow AR1 (see reference numerals ST12 and ST22). Then, as indicated by reference numerals ST22, ST13, ST23, ST14, ST24, the operator performs a zigzag contact operation on the display screen DSP of the first terminal 1 and the display screen DSP of the second terminal 2. Note that AR1 need not always be constant.

  The scratch operation described above is an instruction operation that triggers two terminals to operate in a coordinated manner.

(System configuration)
FIG. 3 is a diagram for explaining the inter-terminal cooperation in the present embodiment. Whether the server (information processing apparatus) 3 operates the first terminal 1 and the second terminal 2 in cooperation with each other based on the determination information transmitted from the first terminal 1 and the second terminal 2. It is a computer that determines whether or not. Hereinafter, the server (information processing apparatus) 3 is referred to as a server 3 as appropriate.

  The wireless access point AP performs wireless communication with the first terminal 1, the second terminal 2, and the server 3. The first terminal 1 communicates with the server 3 via the wireless access point AP (see the dotted line indicated by reference numeral CN1). The second terminal 2 communicates with the server 3 via the wireless access point AP (see the dotted line indicated by reference numeral CN2). When the first terminal 1 and the second terminal 2 shift to the linked state, they communicate with each other via the wireless access point AP (see code CN3). Note that the first terminal 1 and the second terminal 2 may communicate with each other without going through the wireless access point AP when transitioning to a linked state.

(Device hardware block diagram)
FIG. 4 is a hardware block diagram of first terminal 1 or second terminal 2 in the present embodiment. In the following figure, "/" means "or". Specifically, when FIG. 4 shows the hardware configuration of the first terminal 1, the storage (storage unit) 102 stores the first determination information J1 and the linked terminal information CP. When FIG. 4 shows the hardware configuration of the second terminal 2, the storage (storage unit) 102 stores the second determination information J2 and the linked terminal information CP.

  The first terminal 1 includes a CPU (processing unit) 101, a storage (storage unit) 102, a RAM 103, a display device (liquid crystal panel) 104, an input device (touch panel) 105, connected to the bus B, A communication device (communication unit) 106 and an external connection interface device 107 are included. CPU is an abbreviation for “Central Processing Unit”, and RAM is an abbreviation for “Random Access Memory”. Hereinafter, the CPU (processing unit) 101 is referred to as CPU 101, and the storage (storage unit) 102 is referred to as storage 102 as appropriate. The display device (liquid crystal panel) 104 is referred to as a display device 104, the input device (touch panel) 105 is referred to as an input device 105, and the communication device (communication unit) 106 is referred to as a communication device 106 as appropriate.

  The CPU 101 is a central processing unit that controls the entire first terminal 1. The storage 102 is a mass storage device such as a hard disk drive (HDD) and a solid state drive (SSD).

  The storage 102 stores first determination information J1 and linkage terminal information CP. Furthermore, the storage 102 stores an execution file (program) of the terminal cooperation software SF1.

  The RAM 103 temporarily stores processing executed by the CPU 101, data generated (calculated) at each step executed by the terminal cooperation software SF1, and the like. The RAM 103 is a semiconductor memory such as a DRAM (Dynamic Random Access Memory).

  When the first terminal 1 is activated, the CPU 101 reads the execution file of the terminal cooperation software SF1 from the storage 102 and develops it in the RAM 103. Note that this execution file may be stored in the external storage medium MD.

  The display device 104 has the display screen DSP described with reference to FIGS. 1 and 2, and displays various information such as images and characters on the display screen DSP. The display device 104 is, for example, a liquid crystal panel.

  The input device 105 inputs operation information to the first terminal 1. The input device 105 is a touch panel, for example, and detects coordinates indicating the contact position of the object with respect to the display surface DSP. The input device 105 executes this coordinate detection at a predetermined timing (also called sampling timing).

  The communication device 106 connects to another device, for example, a wireless access point AP, and performs wireless communication.

  The external connection interface device 107 is a device that functions as an interface for connecting the own terminal (for example, the first terminal 1) and the external storage medium MD. The external connection interface device 107 is, for example, a card slot or a USB (Universal Serial Bus) port.

  The external storage medium MD is a portable nonvolatile memory such as a USB memory. Note that a configuration may be adopted in which a storage medium reading device (not shown) for reading data stored in the storage medium is connected via the external connection interface device 107. This storage medium (also called a recording medium) is a portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) and a DVD (Digital Versatile Disc).

  In FIG. 4, for example, the display device 104 and the input device 105 of the first terminal 1 may be provided in a separate housing from the first terminal 1. That is, the display device 104 and the input device 105 may be separated from the first terminal 1 and connected to the first terminal 1 by wired communication or wireless communication (hereinafter referred to as a separated configuration). In the case of the separated configuration, the operator places the input device 105 of the first terminal 1 and the input device 105 of the second terminal 2 adjacent to each other.

  The second terminal 2 also has the same hardware configuration as the first terminal 1.

(Server hardware block diagram)
FIG. 5 is a hardware block diagram of the server (information processing apparatus) 3 in the present embodiment. The server 3 includes a CPU (processing unit) 301, a storage (storage unit) 302, a RAM 303, a communication device (communication unit) 304, and an external connection interface device 305 connected to the bus B. Hereinafter, the CPU (processing unit) 301 is referred to as CPU 301, the storage (storage unit) 302 as storage 302, and the communication device (communication unit) 304 as communication device 304 as appropriate.

  The CPU 301 is a central processing unit that controls the entire server 3. The storage 302 is a mass storage device such as a hard disk drive (HDD) and a solid state drive (SSD).

  The storage 302 stores first determination information J1, second determination information J2, and linked terminal information CP. Furthermore, the storage 302 stores an execution file (program) of the terminal cooperation software SF1.

  The RAM 303 temporarily stores processing executed by the CPU 301, data generated (calculated) at each step executed by the cooperation determination software SF3, and the like. The RAM 303 is a semiconductor memory such as a DRAM (Dynamic Random Access Memory).

  The CPU 301 reads the executable file of the linkage determination software SF3 from the storage 302 when the server 3 is activated, and expands it in the RAM 303. Note that this execution file may be stored in the external storage medium MD.

  The communication device 304 connects to the wireless access point AP and performs wireless communication.

  The external connection interface device 305 is a device that functions as an interface for connecting the server 3, the input device IN, the display device DP, and the external storage medium MD. The external connection interface device 305 is, for example, a card slot, a USB (Universal Serial Bus) port, or a connector for connecting to the display device DP.

  The input device IN is a device that inputs operation information to the server 3. The input device IN is, for example, a keyboard or a mouse.

  The display device DP is a device that displays images and characters output from the server 3. The display device DP is, for example, a liquid crystal display device.

  The external storage medium MD is a portable nonvolatile memory such as a USB memory. Note that a configuration may be adopted in which a storage medium reading device (not shown) for reading data stored in the storage medium is connected via the external connection interface device 305. This storage medium is, for example, a portable storage medium such as a CD-ROM or DVD.

(Terminal software block diagram)
FIG. 6 is a software block diagram of first terminal 1 and second terminal 2 in the present embodiment.

  The first terminal 1 and the second terminal 2 are a screen edge entrance / exit estimation unit 11, a judgment information storage unit 12, a screen edge entry / exit continuation judgment unit 13, a judgment information transmission unit 14, and a judgment information erasure Unit 15 and a linked terminal information storage unit 16. The screen edge in / out estimation unit 11 determines whether a screen edge in / out locus has been detected. The determination information storage unit 12 stores (also referred to as storage) determination information for determining whether or not the server 3 operates a plurality of terminals in cooperation with each other. The determination information is, for example, operation information indicating the content of the operation on the display surface DSP.

  The screen edge entry / exit continuation determination unit 13 determines whether the number of operations from the start of the operation to the edge R of the display surface DSP (operation area) of the input device 105 that receives the operation until the interruption is greater than or equal to a predetermined number.

  The determination information transmission unit 14 transmits the determination information of the storage apparatus 102 to the server 3 when the number of operations is a predetermined number or more. The determination information erasure unit 15 erases the determination information in the storage apparatus 102. The cooperation terminal information storage unit 16 receives the cooperation information generated by the server 3 based on the collation between the operation information and the operation information transmitted from the second terminal 2 (another terminal), and stores it in the storage device 102 To do. This link information (hereinafter referred to as link terminal information CP as appropriate) is information indicating that the first terminal 1 and the second terminal 2 perform a link operation.

  When the terminal link software SF1 receives the link terminal information CP, the terminal link software SF1 executes a link operation with another terminal. Note that the storage device 102, the input device 105, and the communication device 106, which are hardware, are indicated by a one-dot chain line.

(Server software block diagram)
FIG. 7 is a software block diagram of the server 3 in the present embodiment. The server 3 includes a determination information storage unit 31, a determination information extraction unit 32, a link terminal search unit 33, a link terminal information storage unit 34, and a link terminal information transmission unit 35.

  The determination information storage unit 31 receives determination information from the first terminal 1 and the second terminal 2 and stores the determination information in the storage device 302. The determination information extraction unit 32 extracts information for determination of the storage apparatus 302. The linked terminal search unit 33 searches for two or more terminals that are linked to each other by collating the extracted determination information.

  The linked terminal information storage unit 34 stores the linked terminal information CP in the storage 302. The link terminal information transmission unit 35 transmits link terminal information to the first terminal 1 and the second terminal 2.

  Note that the storage device 302 and the communication device 304, which are hardware, are indicated by a one-dot chain line.

(Flow chart of terminal link trigger determination processing)
FIG. 8 is a first flowchart illustrating processing executed by the first terminal 1 and the second terminal 2 to determine whether or not it is an opportunity to shift the two terminals to the linked state.

  Prior to the processing described in FIG. 8, the operator arranges the first terminal 1 and the second terminal 2 adjacent to each other as shown in FIG. Then, the operator performs a touch operation on the display surface DSP.

  Step S1: The input device 105 shown in FIG. 6 inputs a contact trajectory (also referred to as handwriting) to the screen edge input / output estimation unit 11. Specifically, when a contact operation is performed by the operator, the input device 105 detects the coordinates at the time of contact and inputs them to the screen edge input / output estimation unit 11.

  Step S2: The screen edge in / out estimation unit 11 determines whether a screen edge in / out locus has been detected. Specifically, the screen edge entrance / exit estimation unit 11 determines the coordinates of the end point corresponding to the input start point or the input end point of the handwriting among the input coordinates (hereinafter referred to as input coordinates as appropriate). Judge whether it is within. Here, the screen edge entrance / exit estimation unit 11 detects a point at which the contact operation starts (for example, touchdown) as a handwriting input start point. Further, the screen edge exit / entry estimation unit 11 detects a point at which the contact operation is completed (for example, touch-up) as a handwriting input end point.

  If the screen edge entry / exit locus is not detected in step S2 (S2 / NO), the process proceeds to step S3. If a screen edge entry / exit locus is detected in step S2 (S2 / YES), the process proceeds to step S4.

  Step S3: The screen edge exit / entry estimation unit 11 inputs input coordinates to a handwriting application (not shown) that performs normal handwriting processing. The handwriting application executes normal handwriting processing, for example, displaying a mark or a cross mark at the position of the input coordinates.

  Step S4: The screen edge exit / entry estimation unit 11 inputs the input coordinates to the determination information storage unit 12, and the determination information storage unit 12 stores the trajectory information (also referred to as operation information) in the storage device. In the case of the screen edge in / out estimation unit 11 of the first terminal 1, the screen edge in / out estimation unit 11 stores the trajectory information in the storage device 102 as the first determination information J1. In the case of the screen edge in / out estimation unit 11 of the second terminal 2, the screen edge in / out estimation unit 11 stores the trajectory information in the storage device 102 as the second determination information J2. Further, the screen edge in / out estimation unit 11 inputs the detection result to the screen edge in / out continuation determination unit 13.

  Step S5: When the detection result is input from the screen edge in / out estimation unit 11, the screen edge in / out continuation determining unit 13 determines whether or not there is a screen edge in / out more than a certain number of times within the first certain time.

  If it is determined in step S5 that the screen edge has entered and exited a certain number of times within the first fixed time (S5 / YES), the process proceeds to step S6. In step S5, if there is no screen edge entry / exit more than a certain number of times within the first certain time (S5 / NO), the process proceeds to step S7.

  Step S6: The determination information transmission unit 14 transmits the determination information stored in the storage device 102 to the server 3. In the case of the determination information transmission unit 14 of the first terminal 1, the determination information transmission unit 14 transmits the first determination information J1 of the storage device 102 to the server 3. In the case of the determination information transmitting unit 14 of the second terminal 2, the determination information transmitting unit 14 transmits the second determination information J2 of the storage device 102 to the server 3.

  Step S7: The screen edge entrance / exit estimation unit 11 determines whether or not the second predetermined time has elapsed since the coordinates were not input from the input device 105. If the above-mentioned time has not elapsed (S7 / NO), the process returns to step S1. If the above-described time has elapsed (S7 / YES), the process proceeds to step S8. After step S6, the process proceeds to step S8.

  Step S8: The determination information deleting unit 15 deletes the determination information in the storage apparatus 102. In the case of the determination information erasure unit 15 of the first terminal 1, the determination information erasure unit 15 erases the first determination information J1 of the storage apparatus 102. In the case of the determination information erasure unit 15 of the second terminal 2, the determination information erasure unit 15 erases the second determination information J2 of the storage apparatus 102.

(Specific example of terminal cooperation trigger determination processing)
A specific example of the terminal cooperation trigger determination process will be described with reference to FIGS. 2 and 8 to 10. FIG. 9 is a diagram showing an example of determination information in the first terminal 1. Note that “A” is shown as the identifier ID for identifying the first terminal 1 in the top row of the table of FIG. FIG. 10 is a diagram illustrating an example of determination information in the second terminal 2. As illustrated in FIG. Note that “B” is shown as the identifier ID for identifying the second terminal 2 in the top row of the table of FIG.

  As shown in FIG. 2, the operator scratches the display surface DSP of the first terminal 1 and the second terminal 2 with the first terminal 1 and the second terminal 2 arranged adjacent to each other. Perform the operation.

(First terminal 1)
As shown in FIG. 2, for example, the operator touches a point P1 on the display surface DSP of the first terminal 1 and slides the finger to the point P2 while maintaining this contact (see line ST11).

  The input device 105 of the first terminal 1 inputs the coordinates at the time of contact to the screen edge entrance / exit estimation unit 11. These coordinates are X and Y coordinates with the lower left corner of the display surface DSP as the origin. Further, the input device 105 of the first terminal 1 has an identifier for identifying the coordinates input to the screen edge input / output estimation unit 11 from the contact start (handwriting input start point) to the contact end (handwriting input end point). The stroke number (for example, an integer equal to or greater than 1) is input to the screen edge entrance / exit estimation unit 11. When the input device 105 is a touch panel, the start of contact indicates the start of contact with the display surface DSP by the operator's finger or operating tool. When the input device 105 is a touch panel, the end of contact indicates the end of contact with the display surface DSP by the operator's finger or operating tool.

  The screen edge exit / entry estimation unit 11 stores, in the RAM 103, the stroke number, input coordinates, coordinate input time, and two coordinate angles determined from the two consecutive coordinates. Further, the screen edge entrance / exit estimation unit 11 determines whether the screen edge entrance / exit locus is present, specifically, determines whether the coordinates of the end point of the handwriting are within the edge portion R (S2). Here, when the coordinates of the end point of the handwriting are within the end portion R, the screen end input / output estimation unit 11 stores the screen end flag “1” in the RAM 103 corresponding to the input coordinates stored in the RAM 103. On the other hand, when the coordinates of the end point of the handwriting are outside the end portion R, the screen end input / output estimation unit 11 stores the screen end flag “0” in the RAM 103 corresponding to the input coordinates stored in the RAM 103.

  FIG. 9 schematically shows the stroke number, input coordinates, coordinate input time, angle of two consecutive coordinates, and screen edge flag in the first terminal 1. The downward direction from the top of the drawing shows the passage of time.

  The table of FIG. 9 shows a stroke number column, an X coordinate column, a Y coordinate column, a time column, a direction (angle) column, and a screen edge column. The stroke number column stores the stroke number. The X coordinate field and the Y coordinate field store the X coordinate and Y coordinate of the input coordinates. The time column stores the input time of coordinates. The direction (angle) column stores the angle of two consecutive coordinates. The screen field stores a screen edge flag.

  In the above example, when the operator slides his / her finger as shown by the line ST11, the input coordinate, the input time of the coordinate, the angle of the two consecutive coordinates, and the screen edge flag in the row indicated by the stroke number “1” Shown in content.

  When the input coordinate is determined to be within the end R (S2 / YES), the determination information storage unit 12 of the first terminal 1 stores the stroke number, input coordinate, coordinate input time, and continuous stored in the RAM 103. The two coordinate angles and the screen edge flag are stored in the storage device 102 as the first determination information J1 (S4). As described above, the determination information (for example, the above-described operation information) includes a plurality of continuous coordinates indicating operation positions when the display surface DSP is continuously operated.

  The screen edge entry / exit continuation determination unit 13 determines whether there is a screen edge entry / exit more than a certain number of times (for example, three times) within a first certain time period (for example, 1 second) (S5). Here, entering and exiting the screen edge means the following first and second cases. The first case is when the handwriting input start point is within the end R. In the first case, it is also called screen edge. The second case is when the handwriting input end point is within the end R. In the second case, it is also called screen edge. In other words, the direction going out of the terminal from the display surface DSP area is called screen edge, and the direction coming from outside the terminal into the display surface DSP area is called screen edge insertion.

  In other words, the screen edge entry / exit continuation determination unit 13 determines whether the number of operations from the start of the operation on the edge R of the display surface DSP (operation area) to the interruption is greater than or equal to a predetermined number (S5). This predetermined number of times is the above-mentioned fixed number of times. Specifically, the screen edge entry / exit continuation determination unit 13 detects screen edge entry / exit, and determines that the number of operations described above is greater than or equal to the predetermined number of times when the number of detections is greater than or equal to the predetermined number (S5). .

  Note that the display surface DSP other than the end portion R is, for example, the housing BDY portion and the white portion of the display surface DSP in the example of FIG.

  In the example of FIG. 9, the screen edge entry / exit continuation determination unit 13 detects screen edge entry when the screen edge flag of the first coordinate of the handwriting is "1", and detects screen edge and screen edge of the handwriting end coordinate. Detected when the flag is "1".

(Second terminal 2)
In FIG. 2, the operator further slides the finger toward the second terminal 2 and moves the points P3, P4, P5 and the finger.

  The input device 105 of the second terminal 2 inputs the coordinates at the time of contact to the screen edge entrance / exit estimation unit 11. These coordinates are X and Y coordinates with the lower left corner of the display surface DSP as the origin. Furthermore, the input device 105 of the second terminal 2 uses the identifier for identifying the coordinates input to the screen edge input / output estimation unit 11 between the start of contact and the end of contact as a stroke number (for example, an integer of 1 or more). Input to the estimation unit 11.

  The screen edge exit / entry estimation unit 11 stores, in the RAM 103, the stroke number, input coordinates, coordinate input time, and two coordinate angles determined from the two consecutive coordinates. Further, the screen edge entrance / exit estimation unit 11 determines whether the screen edge entrance / exit locus is present, specifically, determines whether the coordinates of the end point of the handwriting are within the edge portion R (S2). Here, when the coordinates of the end point of the handwriting are within the end portion R, the screen end input / output estimation unit 11 stores the screen end flag “1” in the RAM 103 corresponding to the input coordinates stored in the RAM 103. On the other hand, when the coordinates of the end point of the handwriting are outside the end portion R, the screen end input / output estimation unit 11 stores the screen end flag “0” in the RAM 103 corresponding to the input coordinates stored in the RAM 103. All the screen edge flags of the input coordinates that are not the end points of the handwriting are “0”.

  FIG. 10 schematically shows the stroke number, input coordinates, coordinate input time, angle of two consecutive coordinates, and screen edge flag in the second terminal 2. The downward direction from the top of the drawing shows the passage of time. Since the configuration of the table in FIG. 10 is the same as the configuration of the table in FIG. 9, the description of the configuration is omitted.

  In the above example, when the operator slides his / her finger as shown by the line ST21, the input coordinate, the input time of the coordinate, the angle of the two consecutive coordinates, and the screen edge flag in the row indicated by the stroke number "1" Shown in content.

  When the information storage unit for determination 12 of the second terminal 2 detects the screen edge entry / exit locus (S2 / YES), the stroke number, input coordinates, input time of coordinates, and two consecutive coordinates stored in the RAM 103 Are stored in the storage device 102 as second determination information J2 (S4).

  The screen edge entry / exit continuation determination unit 13 determines whether there is a screen edge entry / exit more than a certain number of times (for example, three times) within a first certain time period (for example, 1 second) (S5).

  In the example of FIG. 10, the screen edge entry / exit continuation determination unit 13 detects screen edge entry when the screen edge flag of the first coordinate of the handwriting is "1", and detects screen edge and screen edge of the handwriting end coordinate. Detected when the flag is "1".

(First terminal 1)
In FIG. 2, the operator further slides his / her finger toward the first terminal 1, and moves the points P6, P7 and P8 and the finger.

  The input device 105 of the first terminal 1 displays the coordinates at the time of contact and the identifier (stroke No.) identifying the coordinates input to the screen edge input / output estimation unit 11 between the start of contact and the end of contact. Enter in 11.

  The screen edge exit / entry estimation unit 11 stores, in the RAM 103, the stroke number, input coordinates, coordinate input time, and two coordinate angles determined from the two consecutive coordinates. Further, the screen edge entrance / exit estimation unit 11 determines whether the screen edge entrance / exit locus (S2).

  In the above example, when the operator slides his / her finger as shown by the line ST12, the input coordinates, the input time of the coordinates, the angle of two consecutive coordinates, and the screen edge flag are indicated by the stroke No. 2 in FIG. This is indicated by the contents of the indicated line.

  The determination information storage unit 12 of the first terminal 1 detects a screen edge entry / exit locus (S2 / YES), stroke number, input coordinates, coordinate input time, angle of two consecutive coordinates, screen edge flag. Is stored in the storage apparatus 102 as first determination information J1 (S4).

  The screen edge entry / exit continuation determination unit 13 determines whether or not there is a screen edge entry / exit more than a certain number of times within the first certain time (S5).

  The entry / exit continuation determination unit 13 determines that there is no screen edge entry / exit more than a certain number of times within the first certain time (S5 / NO). Then, the screen edge exit / entry estimation unit 11 determines that the second predetermined time has not elapsed since the coordinates were not input from the input device 105 (S7 / NO).

  Thereafter, as described in FIG. 2, the operator sets the display surface DSP of the first terminal 1 and the display surface DSP of the second terminal 2 as indicated by reference numerals ST22, ST13, ST23, ST14, and ST24. Perform contact operation in a zigzag pattern.

  In the process of this contact operation, the screen edge in / out continuation determining unit 13 of the first terminal 1 determines that there are screen edge in / out more than a certain number of times within the first certain time (S5 / YES). Then, the determination information transmission unit 14 of the first terminal 1 transmits the first determination information J1 stored in the storage device 102 to the server 3 via the communication unit 106. The first determination information J1 is, for example, various information shown in FIG. Note that the determination information transmission unit 14 of the first terminal 1 transmits an identifier (for example, an IP (Internet Protocol) address) that uniquely identifies the terminal on the network when the first determination information J1 is transmitted. Send to 3.

  Similarly, in the process of the touch operation, the screen edge in / out continuation determining unit 13 of the second terminal 2 determines that there are screen edge in / out more than a certain number of times within the first certain time (S5 / YES). . Then, the determination information transmission unit 14 of the second terminal 2 transmits the second determination information J2 stored in the storage device 102 to the server 3 via the communication unit 106. The second determination information J2 is, for example, various information shown in FIG. The determination information transmitting unit 14 of the second terminal 2 transmits to the server 3 an identifier for uniquely identifying the own terminal on the network when transmitting the second determination information J2.

(Flow chart of terminal linkage determination processing)
FIG. 11 is a flowchart for explaining the process executed by the server 3 to determine the linked terminal. The communication unit 304 of the server 3 is waiting to receive the first determination information J1 and the second determination information J2.

  Step S11: The communication unit 304 of the server 3 receives the first determination information J1 transmitted from the first terminal 1 and the second determination information J2 transmitted from the second terminal 2, and performs determination. Input to the information storage unit 31.

  Step S12: The determination information storage unit 31 stores the received first determination information J1 and second determination information J2 in the storage device 302 (see FIG. 7).

  Step S13: The determination information extraction unit 32 extracts the determination information (for example, the first determination information J1 and the second determination information J2) of the storage device 302, and the cooperation terminal search unit 33 extracts Collate judgment information and search for two or more terminals to be linked. That is, the linked terminal search unit 33 collates the first determination information J1 and the second determination information J2, and the first operation on the first terminal and the second operation on the second terminal are linked. It is determined whether or not the operation is an instruction operation. This instruction operation is, for example, the scratch operation described with reference to FIG. A specific example of this determination will be described later.

  Step S14: The linked terminal search unit 33 determines whether two or more terminals to be linked are searched. If the search is successful (step S14 / YES), the process proceeds to step S15. If the search is not possible (step S14 / NO), the process proceeds to step S16.

  Step S15: The linked terminal information storage unit 34 stores information related to the searched terminal (hereinafter referred to as linked terminal information CP as appropriate) in the storage device 302, and the linked terminal information transmission unit 35 transmits the linked terminal to the searched terminal. Terminal information CP is transmitted. That is, when it is determined that the first operation and the second operation described in step S13 are scratch operations performed between both terminals, the link terminal information transmitting unit 35 sets the link terminal information CP to the first terminal. Transmit to the first and second terminals 2. A specific example of this storage and transmission will be described later.

  Step S16: The link terminal searching unit 33 deletes the determination information in the storage device 302. By this erasing process, unnecessary determination information is eliminated, so that the free capacity of the storage apparatus 302 can be increased.

(Specific example of collation and search in step S13)
A specific example of collation and search in step S13 in FIG. 11 will be described with reference to FIG. 2, FIG. 7, FIG. 9, and FIG.

  First, the determination information extraction unit 32 extracts the determination information (for example, the first determination information J1 in FIG. 9 and the second determination information J2 in FIG. 10) of the storage device 302 (S13).

  The linked terminal search unit 33 collates the extracted first determination information J1 in FIG. 9 and second determination information J2 in FIG. This collation will be described in detail. The linked terminal search unit 33 has the same stroke number in the first determination information J1 and the second determination information J2, and extracts the angle at the entry / exit point. Here, the entry / exit point is a coordinate point at which the screen edge flag is “1” in the first terminal 1 and the second terminal 2.

  In the example of the first determination information J1 in FIG. 9, for example, the linked terminal search unit 33 extracts the following angles. Angle "AnglA1m" of the point with stroke No. "1" and screen edge flag "1". This angle corresponds to, for example, the angle of a straight line connecting the end point P2 and the immediately preceding sampling point. Angle "AnglA21" of the point with stroke No. "2" and screen edge flag "1". This angle corresponds to the angle of a straight line connecting the point P6 in FIG. 2 and the sampling point immediately before (or immediately after) the point P6. Angle "AnglA2n" at the point with stroke No. "2" and screen edge flag "1". Similarly, this angle corresponds to the angle of a straight line connecting point P8 in FIG. 2 and the sampling point immediately preceding it.

  In the example of the second determination information J2 in FIG. 10, the cooperation terminal search unit 33 extracts the following angles, for example. Angle "AnglB11" of the point with stroke No. "1" and screen edge flag "1". This angle corresponds to the angle of a straight line connecting the point P3 in FIG. 2 and the sampling point immediately thereafter. Angle "AnglB1m" with a stroke No. "1" and a screen edge flag of "1". This angle corresponds to the angle of the straight line connecting the point P5 in FIG. 2 and the sampling point immediately before it. Angle "AnglB21" with a stroke No. "2" and a screen edge flag of "1". This angle corresponds to the angle on the arrow AR1 side in the line ST22 in FIG.

  The linkage terminal search unit 33 collates each of the plurality of angles extracted from the first determination information J1 and the plurality of angles extracted from the second determination information J2, respectively, to determine a predetermined range (for example, 0.5 degrees). ) Pair the angle inside.

  In the example of FIG. 2, the linked terminal search unit 33 pairs the angle “AnglA1m” (angle at the point P2) and the angle “AnglB11” (angle at the point P3). Further, in the example of FIG. 2, the link terminal search unit 33 pairs the angle “AnglA21” (angle at the point P6) and the angle “AngB1m” (angle at the point P5). Further, in the example of FIG. 2, the link terminal search unit 33 pairs the angle “AnglA2n” (angle at the point P8) and the angle “AnglB21” (angle on the arrow AR1 side in the line ST22 in FIG. 2). .

  Then, the linked terminal search unit 33 calculates the difference between the paired angles, and when the accumulated value of the differences is within a predetermined value, the linked terminal search unit 33 operates the first terminal 1 and the second terminal 2 in a linked operation. Search as a terminal (S13). Here, the predetermined value is a value determined by the developer through experiments.

  That is, the link terminal searching unit 33 collates the first feature of the coordinates included in the first determination information J1 with the second feature of the coordinates included in the second determination information J2. The coordinate feature indicates, for example, an angle indicated by two coordinates.

  Then, when the first feature and the second feature match, the cooperation terminal search unit 33 determines that the first operation and the second operation are instruction operations (for example, scratch operation) of the cooperation operation. In the case of the angle, the first feature and the second feature in the case where the above-described feature is used are in the case where the cumulative value of the differences between the paired angles is within a predetermined value.

  In addition, the feature of the coordinates is the coordinates of the point located most at the end R. These coordinates are points P2, P3, P5, P6, and P8 in the example of FIG. In the example of the feature of this coordinate, the link terminal searching unit 33 uses the Y coordinate of the point located at the end in the first determination information J1 and the second determination information J2 that is closest to this coordinate. Pair with the Y coordinate of the point located at the extreme end of. At this time, in the example of FIG. 2, the linked terminal search unit 33 pairs the points P2 and P3. Point P5 is paired with point P6. Next, the linked terminal search unit 33 assumes that the first feature and the second feature match if the accumulated value of the difference between the paired Y coordinates is within a predetermined value.

  Note that the link terminal searching unit 33 may collate the speed, acceleration, and coordinates of the locus at the entry / exit point instead of the angle. In addition, the linked terminal search unit 33 may collate the difference between the number of screen edge in / out at the first terminal 1 and the number of screen edge in / out at the second terminal 2. These elements may be used in combination.

  As a result of the search, the linked terminal search unit 33 determines that the first terminal 1 and the second terminal 2 to be linked are searched (S14 / YES).

  The linked terminal information storage unit 34 stores the searched linked terminal information CP of the first terminal 1 and the second terminal 2 in the storage device 302 (S15). The linked terminal information CP of the searched first terminal 1 and second terminal 2 includes an identifier for uniquely identifying the first terminal 1 and the second terminal 2 that operate in cooperation on the network, and the first terminal 1. And information indicating that the second terminal 2 is a terminal that is a target of the cooperative operation.

  The link terminal information transmission unit 35 transmits link terminal information CP to the first terminal 1 and the second terminal 2 via the communication unit 304.

(Linked terminal confirmation process in the terminal)
FIG. 12 is a flowchart for explaining the process of determining the cooperation terminal executed by the first terminal 1 and the second terminal 2.

  Step S21: The communication unit 106 of the first terminal 1 and the second terminal 2 receives the linked terminal information CP from the server 3 and inputs it to the linked terminal information storage unit 16.

  Step S22: The link terminal information storage unit 16 stores the input link terminal information CP in the storage device 102, and the terminal link software SF1 is a terminal that is a target of the link operation included in the input link terminal information CP. Transition to the linked state. That is, the terminal cooperation software SF1 operates in cooperation with the terminal identified by the terminal identifier included in the cooperation terminal information CP.

  Step S22 will be specifically described. Based on the identifier of the second terminal 2 that is the target of the cooperative operation included in the input cooperative terminal information CP, the terminal cooperation software SF1 of the first terminal 1 uses the wireless access point AP in FIG. Communicate with terminal 2 of 2. This communication is schematically indicated by reference numeral CN3 in FIG.

  Similarly, the terminal cooperation software SF1 of the second terminal 2 passes through the wireless access point AP of FIG. 3 based on the identifier of the first terminal 1 that is the target of the cooperation operation included in the input cooperation terminal information CP. , Communicate with the first terminal 1; This communication is schematically indicated by reference numeral CN3 in FIG.

  The terminal cooperation software SF1 of the first terminal 1 notifies the terminal cooperation software SF1 of the second terminal 2 that it has been detected as a cooperation target.

  Based on this notification, the second terminal 2 confirms that the terminal user has established the cooperation, and if the confirmation is confirmed, the second terminal 2 transitions to the cooperation state with the first terminal 1, and indicates that the state has entered the cooperation state. Notify terminal 1 of 1. At this time, the terminal cooperation software SF1 of the second terminal 2 displays on the display device 104 a symbol indicating that the state has shifted to the cooperation state with the first terminal 1. This symbol is, for example, an image such as a character string “cooperation state” or a tally indicating that the state has shifted to the cooperation state.

  Upon receiving this notification, the terminal link software SF1 of the first terminal 1 shifts to a link state with the second terminal 2, and displays a symbol indicating that the switch to the link state with the second terminal 2 is made on the display device 104. To display.

  Thereafter, the first terminal 1 and the second terminal 2 perform a terminal cooperation operation. The terminal cooperation operation will be exemplified. For example, when the browser is activated in the first terminal 1, the terminal cooperation software SF1 of the first terminal 1 notifies the terminal cooperation software SF1 of the second terminal 2 to that effect. Upon receiving this notification, the terminal cooperation software SF1 of the second terminal 2 activates the browser.

  When a browser activated on the first terminal 1 accesses a site indicated by a URL (Uniform Resource Locator), the terminal cooperation software SF1 of the first terminal 1 informs the second terminal of that fact. 2 is notified to the terminal cooperation software SF1. Upon receiving this notification, the terminal cooperation software SF1 of the second terminal 2 accesses the site indicated by this URL.

  In addition, the terminal cooperation operation includes an operation in which the first terminal 1 and the second terminal 2 share the same information. Sharing the same information means, for example, simultaneously creating or displaying the same content (character information or image information).

  In particular, information can be easily shared between students and teacher students by cooperating a plurality of terminals of a student and sharing the same information among the plurality of terminals in an educational setting. Therefore, the teacher can easily grasp the understanding information of the student and easily provide various information to the student for the understanding of the student.

  According to the cooperation process described in this embodiment, the operator can instruct the terminal to shift a plurality of terminals to the cooperation state by a simple operation called a scratch operation. That is, according to the linkage processing described in the present embodiment, there is no need to perform an operation for switching from the normal input operation mode to the mode in which the normal input operation is disabled and the mode is shifted to the linkage state.

  This switching operation is, for example, an operation in which the user finds an icon for instructing the transition to the linked state and touches this icon, and this operation is complicated for the operator. However, according to the present embodiment, the operator can move a plurality of terminals to the linked state with a simple operation (for example, a scratch operation), so that the operability is improved.

  Furthermore, according to the present embodiment, it is not necessary to provide dedicated hardware (for example, a camera) in order to perform an instruction operation for the cooperative operation, so that the manufacturing cost does not increase.

  Note that instead of a scratch operation, a contact operation that draws a straight line on the display screen DSP of the first terminal 1 and the second terminal 2 may be used. However, in this contact operation, an operator may make an erroneous operation. . In this embodiment, in order to eliminate this erroneous operation, the operator is caused to perform a scratch operation.

  In addition, the first terminal 1 and the second terminal 2 use the judgment stored in the storage device 102 when the screen edge has entered and exited more than a certain number of times within the first certain time (S5 / YES in FIG. 8). Information is transmitted to the server 3 (S6). That is, the first terminal 1 and the second terminal 2 do not transmit the determination information to the server 3 every time the determination information is input. Therefore, according to the present embodiment, it is possible to reduce the power required for transmitting the determination information compared to the case where the determination information is transmitted to the server 3 each time the determination information is input. As a result, the power consumption of the terminal can be suppressed. In particular, when a plurality of terminals that are linked are driven by a rechargeable battery, the drive time can be suppressed from being shortened by this power reduction.

[Second Embodiment]
In the first embodiment, the determination information transmission unit 14 of the first terminal 1 and the second terminal 2 transmits the operation information stored in the storage device 102 to the server 3 (S6 in FIG. 8), The determination information deleting unit 15 deletes the determination information of the storage apparatus 102 (S8).

  And the cooperation terminal search part 33 of the server 3 collated the information for determination, and searched two or more terminals to operate | move in cooperation (refer S13 of FIG. 11). This search is executed after receiving the first determination information J1 and the second determination information J2 (S11). In this collation, the greater the amount of information for determination, the more accurate collation is possible. Therefore, in order to enable accurate verification, the determination information transmitting unit 14 of the first terminal 1 and the second terminal 2 transmits the determination information to the server 3 (S6), and then deletes the determination information. Do not do. Then, after transmitting the determination information, the first terminal 1 and the second terminal 2 again detect the coordinates at the time of contact (S1), and further store the operation information.

  Then, the first terminal 1 and the second terminal 2 perform processing for determining whether or not there is a screen edge entry / exit more than a certain number of times within the first certain time after transmitting the determination information. Then, the first terminal 1 and the second terminal 2 transmit the determination information stored after transmission of the determination information to the server 3 together with the determination information already transmitted.

  FIG. 13 is a second flowchart for explaining processing for determining the timing of terminal cooperation, which is executed by the first terminal 1 and the second terminal 2. The flowchart of FIG. 13 is a flowchart in which step S5 is replaced with step S5a in the flowchart of FIG. Further, the flowchart of FIG. 13 shows a process of returning to step S1 without moving to step S8 after the determination information is transmitted (S6).

  Step S5a: When the detection result is input from the screen edge in / out estimation unit 11, the screen edge in / out continuation determining unit 13 determines whether or not there is a screen edge in / out more than a certain number of times within the first certain time. Alternatively, the screen edge entry / exit continuation determination unit 13 may receive a detection result from the screen edge input / output estimation unit 11 after transmitting the determination information, and the screen edge input / output more than a certain number of times within the first fixed time. Determine if there is any. If there are more than a certain number of screen edge in / out within the first fixed time (S5a / YES), the process proceeds to step S6. After the determination information is transmitted, if there is more than a certain number of screen edge in / out within the first certain time (S5a / YES), the process proceeds to step S6.

  If step S5a is not YES (S5a / NO), the process proceeds to step S7.

  The determination information transmitting unit 14 transmits the determination information stored in the storage apparatus 102 after transmission together with the determination information already transmitted (S7).

  Here, after the determination information transmitting unit 14 transmits the determination information stored in the storage device 102 to the server 3 (S6), unlike the flowchart of FIG. 8 described in the first embodiment, FIG. Return to step S1. Therefore, the determination information in the storage apparatus 102 is not erased.

  Therefore, the determination information transmission unit 14 of the first terminal 1 and the second terminal 2 stores the determination information stored after transmission of the determination information X together with the determination information already transmitted (hereinafter referred to as determination information X). Information (hereinafter referred to as determination information Y) can be transmitted to the server 3. Hereinafter, determination information obtained by adding determination information Y to determination information X is referred to as determination information Z.

  Therefore, the linked terminal search unit 33 of the server 3 can search for two or more terminals to be linked by collating the determination information Z of the first terminal 1 and the determination information Z of the second terminal 2. (S13 in FIG. 11). As described above, the collaborative terminal search unit 33 of the server 3 can collate the determination information having a larger amount of information compared to the first embodiment, so that accurate collation is possible. As a result, the search accuracy of the terminal that operates in cooperation is improved.

[Third Embodiment]
In the third embodiment, in order to improve the search accuracy of two or more terminals that the server 3 operates in cooperation with each other, a guide image (hereinafter referred to as “guide” as appropriate) is displayed to the operator, and this guide is passed. Let the operation.

(Terminal software block diagram)
FIG. 14 is a software block diagram of first terminal 1 or second terminal 2 in the present embodiment.

  FIG. 14 is a diagram in which a guide processing unit 17 is added to the configuration of the first terminal 1 and the second terminal 2 described in FIG. The guide processing unit 17 transmits the position information (hereinafter referred to as passing position designation information as appropriate) transmitted from the server 3 and indicating the display position of the guide indicating the position of the operation on the end R to the operator, and the communication unit 106. Receive via. Then, the guide processing unit 17 analyzes this information and displays a guide in the passing position designation information at this display position.

(Server software block diagram)
FIG. 15 is a software block diagram of the server 3 in the present embodiment. FIG. 15 is a diagram in which a passage position designation information generation unit 36 is added to the configuration of the server 3 described in FIG. The passing position designation information generating unit 36 transmits passing position designation information including the coordinates of the guide to the first terminal 1 and the second terminal 2 via the communication unit 304.

(Flow chart of passing position designation information transmission process)
FIG. 16 is a flowchart for explaining the transmission process of the passing position designation information executed by the server 3. The flowchart in FIG. 16 shows a flow in which step S17 is executed after it is determined that two or more terminals to be linked in the flowchart in FIG. 11 are not searched (S14 / NO). Note that, as shown in the flowchart of FIG. 16, after step S17 is executed, processing for deleting the judgment information (S16) is executed.

  Step S17: The passing position designation information generating unit 36 of the server 3 generates passing position designation information for the first terminal 1 including the coordinates of the guide based on the first determination information J1, and the first terminal 1 Send to. Similarly, the passing position designation information generating unit 36 of the server 3 generates passing position designation information for the second terminal 2 including the coordinates of the guide based on the second determination information J2, and the second terminal 2 Send to.

(Specific example of generation processing of passing position specification information)
Hereinafter, the generation processing of the passage position designation information will be specifically described. The passing position designation information generation unit 36 performs a touch operation on the second terminal 2 based on the coordinates included in the second determination information J2 of the second terminal 2 and the screen edge flag is “1”. The edge R is specified to belong to one of the four sides (upper, lower, right, left) on the display surface DSP.

  In the example of FIG. 2, it can be seen that in the second terminal 2, the contact operation was performed at the end R on the left side. Therefore, the passing position designation information generating unit 36 specifies that the end R where the contact operation has been performed on the second terminal 2 belongs to the left side of the display surface DSP. Then, when the specified side is the right side or the left side, the passing position designation information generation unit 36 is based on the coordinate X where the screen edge flag included in the second determination information J2 of the second terminal 2 is “1”. , Determine the X coordinate of the guide coordinates. For example, when the coordinates X whose screen edge flag is “1” are distributed within a predetermined range with reference to 0, the X coordinate of the guide coordinates is determined to be 0. If the average of the coordinates X whose screen edge flag is “1” is Xav, the X coordinate of the guide coordinates may be determined as Xav. Since the guide is displayed at the edge of the screen, you can basically set the minimum or maximum value of the screen X coordinate according to which end of the screen.

  Next, the passing position designation information generating unit 36 randomly determines the Y coordinates of the guide coordinates by the number of guides. At this time, the passing position designation information generation unit 36 may make the difference between the Y coordinates of two consecutive guides constant.

  For example, when the number of guides is three, the Y coordinate is determined to be 400, 300, and 200. Then, the passage position designation information generation unit 36 generates passage position designation information (hereinafter referred to as passage position designation information B) including the determined guide coordinates. The determined guide coordinates are Pb1 (0, 400), Pb2 (0, 300), and Pb3 (0, 200) in the above example.

  Note that when the specified side is the upper side or the lower side, the passing position designation information generation unit 36 is based on the coordinate Y in which the screen edge flag included in the second determination information J2 of the second terminal 2 is “1”. , Determine the Y coordinate of the guide coordinates. The subsequent processing is processing in which the X coordinate and the Y coordinate are exchanged in the guide coordinate determination processing described with the specified side as the left side or the right side.

  Next, the passing position designation information generation unit 36 performs the touch operation on the first terminal 1 based on the coordinates included in the first determination information J1 of the first terminal 1 and the screen edge flag is “1”. It is specified to which side of the four sides of the display surface DSP the performed end R belongs.

  In the example of FIG. 2, it can be seen that in the first terminal 1, the contact operation was performed at the end R on the right side. Therefore, the passing position designation information generation unit 36 specifies that the end R where the contact operation has been performed on the first terminal 1 belongs to the right side of the display surface DSP. Then, when the specified side is the right side or the left side, the passing position designation information generation unit 36 is based on the coordinate X where the screen edge flag included in the first determination information J1 of the first terminal 1 is “1”. , Determine the X coordinate of the guide coordinates. For example, when the coordinate X having the screen edge flag “1” is distributed within a predetermined range with reference to 1024, the X coordinate of the guide coordinate is determined to be 1024. If the average of the coordinates X whose screen edge flag is “1” is Xav, the X coordinate of the guide coordinates may be determined as Xav. Since the guide is displayed at the edge of the screen, you can basically set the minimum or maximum value of the screen X coordinate according to which end of the screen.

  Next, the passing position designation information generating unit 36 may generate the Y coordinate of the guide coordinates at random as in the case of the first terminal 1, or the Y coordinate of the guide coordinates in the second terminal 2 may be generated. It may be determined correspondingly. In this case, the passing position designation information generation unit 36 determines the Y coordinate of the guide coordinate obtained by subtracting a predetermined coordinate from the Y coordinate of the guide coordinate in the second terminal 2. The predetermined coordinate is 10, for example.

  For example, the passage position designation information generation unit 36 determines the Y coordinate of the guide as 390 (400-10), 290 (300-10), and 190 (200-10). Then, the passage position designation information generation unit 36 generates passage position designation information including the determined guide coordinates (hereinafter referred to as passage position designation information A). The determined guide coordinates are Pa1 (1024, 390), Pa2 (1024, 290), and Pa3 (1024, 190) in the above example.

  The passing position designation information generation unit 36, based on the coordinate Y where the screen edge flag included in the first determination information J1 of the first terminal 1 is “1” when the specified side is the upper side or the lower side. , Determine the Y coordinate of the guide coordinates. The subsequent processing is processing in which the X coordinate and the Y coordinate are exchanged in the guide coordinate determination processing described with the specified side as the left side or the right side.

  The passing position designation information generation unit 36 transmits the generated passing position designation information B to the second terminal 2, and transmits the generated passing position designation information A to the first terminal 1 (S17).

  As described above, the passage position designation information generation unit 36 identifies the end R on which the operation has been performed based on the plurality of coordinates included in the first determination information J1, and the first end R at the identified end R is specified. Passing position designation information A indicating the display position of the guide is transmitted to the first terminal 1. Then, the passing position designation information generation unit 36 identifies the end R where the operation has been performed based on the plurality of coordinates included in the second determination information J2, and displays the second guide at the identified end R. Passing position designation information B indicating the position is transmitted to the second terminal 2.

(Guide display processing flowchart)
FIG. 17 is a flowchart for explaining guide display processing executed by the first terminal 1 and the second terminal 2. The first terminal 1 receives the passage position designation information A transmitted from the server 3. Further, the second terminal 2 receives the passing position designation information B transmitted from the server 3.

  Step S31: The guide processing unit 17 receives the passing position designation information transmitted from the server 3.

  Step S32: When an input operation is performed, the guide processing unit 17 displays a guide at one coordinate included in the passing position designation information transmitted from the server 3. The guide processing unit 17 determines that an input operation has been performed when coordinates are input from the input device 105 to the screen edge input / output estimation unit 11.

  Step S33: The guide processing unit 17 erases the displayed guide when the contact point goes out of the display surface DSP area and there is no information input from the input device 105.

  Step S34: The guide processing unit 17 determines whether to end the guide display process. Specifically, when the determination information transmitting unit 14 transmits determination information or when a guide is displayed at all coordinates included in the passing position designation information, the guide processing unit 17 terminates the guide display process. Determine (S34 / YES). When the guide display process is to be ended (S34 / YES), the guide processing unit 17 ends the guide display process. If the guide display process is not terminated (S34 / NO), the process returns to step S32. When the process returns to step S32, the guide processing unit 17 displays the guide at coordinates corresponding to the undisplayed guide.

(Specific example of guide display processing)
FIG. 18 is a diagram for explaining guide display processing in the first terminal 1 and the second terminal 2. In FIG. 18, a dotted line indicates a locus when the operator touches the display surface DSP. A one-dot chain line arrow indicates the direction of the contact operation.

  FIG. 18 (A) is a diagram illustrating a process for displaying a guide on the second terminal 2. When the contact operation is performed, the guide processing unit 17 of the second terminal 2 performs the guide coordinates Pb1 (0, 400), Pb2 (0, 300), Pb3 (0) included in the received passing position designation information B. , 200), the guide G1 is displayed at the first coordinate Pb1 (0, 400) (S32).

  The shape of the guide may be various shapes, for example, a semicircle, a rectangle, or a star shape. At this time, the operator performs a contact operation from the first terminal 1 to the second terminal 2 as indicated by a dotted line in FIG. In the following description, every time the operator performs a touch operation on the display surface DSP of the first terminal 1 and the second terminal 2, the processing described in the flowchart of FIG. 8 is executed.

  FIG. 18 (B) is a diagram for explaining the process of deleting the guide in the second terminal 2. The operator performs a contact operation toward the guide graphic G1. When the contact point goes out of the display surface DSP area, the guide processing unit 17 deletes the guide G1 as indicated by reference numeral G2 (S33).

  FIG. 18C is a diagram for explaining processing for displaying a guide on the first terminal 1. The guide processing unit 17 of the first terminal 1 selects 1 of the guide coordinates Pa1 (1024, 390), Pa2 (1024, 290), Pa3 (1024, 190) included in the received passage position designation information A. The guide G3 is displayed at the first coordinate Pa1 (1024, 390) (S32).

  FIG. 18D is a diagram for explaining a contact operation performed by the operator toward the guide G3 displayed on the first terminal 1. The operator performs a contact operation toward the guide G3. The guide processing unit 17 deletes the guide G3 when the contact point goes out of the display surface DSP area (S33).

  Thereafter, the guide processing unit 17 of the second terminal 2 displays the guide at the second coordinate Pb2 (0, 300) (S32), deletes the displayed guide (S33), and then the third coordinate. A guide is displayed on Pb3 (0, 200) (S32). Here, after step S32, the guide processing unit 17 of the second terminal 2 determines whether to end the guide display processing (S34), and if it determines to end the guide display processing (S34 / YES), The guide display process ends.

  Similarly, the guide processing unit 17 of the first terminal 1 displays the guide at the second coordinate Pa2 (1024, 290) (S32), deletes the displayed guide (S33), and then the third coordinate. A guide is displayed on Pa3 (1024, 190) (S32). Here, after step S32, the guide processing unit 17 of the first terminal 1 determines whether or not to end the guide display processing (S34), and if it determines to end the guide display processing (S34 / YES) The guide display process ends.

  The server 3 includes first determination information J1 including coordinates along the guide transmitted from the first terminal 1, and second determination information including coordinates along the guide transmitted from the second terminal 2. Receive J2. Then, the server 3 collates the first determination information J1 and the second determination information J2, and searches for two or more terminals to be operated in cooperation (S13).

  By displaying this guide, the operator can visually recognize where the touch operation should be performed on the display surface DSP when performing the scratch operation. As a result, the scratch operation is easier than when the guide is not displayed.

  In addition, the display of this guide can cause the operator to execute a scratch operation. For this reason, the possibility that the coordinate trajectory change in the first determination information coincides with the coordinate trajectory change in the second determination information is increased, and the search accuracy of the terminal to be linked is increased. This locus change is, for example, an angle between two coordinates.

  The search process executed by the server 3 may be the same as in the above-described embodiment. In the case of the present embodiment, the operator's operation can be guided to some extent by the guide (the operator may not be able to protect it), so that the separation of the terminal pair in the search and collation of the terminals to be linked is further improved. Can do.

[Fourth Embodiment]
In the first to third embodiments, the server 3 is provided separately, but the function executed by the server 3 may be provided in the terminal. For example, the function executed by the server 3 is provided in the first terminal 1.

  FIG. 19 is a software block diagram of the first terminal 1 provided with the function of the server 3.

  The determination information transmission unit 14 of the first terminal 1 does not transmit the determination information stored in the storage device 102 to the server 3 because the own terminal executes the function of the server 3.

  Further, the linked terminal information storage unit 34 of the first terminal 1 does not transmit the information related to the searched terminal to the first terminal 1 but to the terminal of the second terminal 2.

  The determination information transmission unit 14 of the second terminal 2 transmits the second determination information J2 stored in the storage device 102 to the first terminal 1 having the function of the server 3. The determination information storage unit 31 of the first terminal 1 stores the received second determination information J2 in the storage device 102.

  According to the present embodiment, since the terminal has the function of the server 3, it is not necessary to provide the server 3 separately.

[Other embodiments]
In the first to third embodiments, the touch panel is illustrated as an example of an input device for inputting operation information in the first terminal 1 and the second terminal 2. However, the input device for inputting operation information may be a camera, for example. For example, the first terminal 1 displays the captured image of the camera on the display surface DSP. Then, the operator performs an instruction operation (for example, moving the finger in the vicinity of the camera lens) toward the camera with the operator's finger or an operation tool. The camera (input device) of the first terminal 1 inputs the captured image to the screen edge entrance / exit estimation unit 11. The screen edge exit / entry estimation unit 11 analyzes this image and calculates the coordinate pointed on the display screen DSP. Thereafter, the first terminal 1 and the second terminal 2 perform the processing described in the first to third embodiments based on the coordinates.

  The above embodiment is summarized as follows.

(Appendix 1)
A terminal device that operates in cooperation with another terminal device,
A communication unit that communicates with other devices;
When the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number, operation information indicating the content of the operation is transmitted
When receiving the cooperation information indicating that the operation of the terminal device and the other terminal device, which is generated based on the comparison between the operation information and the operation information transmitted from the other terminal device, is performed, A terminal device having a processing unit that executes a cooperative operation with another terminal device.

(Appendix 2)
In Appendix 1,
The terminal device in which the operation region of the terminal device and the operation region of the other terminal device are arranged adjacent to each other when the operation information is transmitted.

(Appendix 3)
In Appendix 1,
The operation information includes a plurality of continuous coordinates indicating an operation position when the operation area is continuously operated,
The processing unit detects a start point or an end point of the coordinates included in the end, and determines that the number of operations is equal to or greater than a predetermined number when the number of detections is equal to or greater than the predetermined number.

(Appendix 4)
In Appendix 2,
When the operation on the end is started, the processing unit stores the operation information in a storage unit,
A terminal device that transmits the operation information stored in the storage unit after transmission together with the transmitted operation information when the number of operations exceeds the predetermined number after the operation information is transmitted.

(Appendix 5)
In Appendix 1,
The processing device, when receiving position information indicating a display position of a guide image indicating an operation position with respect to the end portion to an operator, displays the guide image at the display position.

(Appendix 6)
A terminal device cooperation method in a system having a first operation terminal and a second terminal device that operate in cooperation, and an information processing device that communicates with the first operation terminal and the second terminal device,
The first terminal device receives first operation information indicating the content of the operation when the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number. Sent to the information processing device;
The second terminal device having the input device arranged adjacent to the input device of the first terminal device is from the start of the operation to the end of the operation area of the input device that accepts the operation until interrupted. When the number of operations becomes a predetermined number or more, second operation information indicating the content of the operation is transmitted to the information processing device,
The information processing device collates the first operation information with the second operation information, and the first operation for the first terminal device and the second operation for the second terminal device are performed. When it is determined as an instruction operation of cooperative operation, the cooperative information indicating that the first terminal device and the second terminal device operate cooperatively is transmitted to the first operating terminal and the second terminal device,
When the first terminal device receives the cooperation information, the first terminal device performs a cooperation operation with the second terminal device,
When the second terminal device receives the cooperation information, the terminal device cooperation method executes a cooperation operation with the first terminal device.

(Appendix 7)
In Appendix 6,
The first and second operation information includes a plurality of continuous coordinates indicating operation positions when the operation area is continuously operated,
The first operation terminal and the second terminal device detect that a start point or an end point of the coordinates is included in the end, and when the number of detections is equal to or greater than the predetermined number, the number of operations is A terminal device cooperation method for determining that the number of times is equal to or greater than a predetermined number.

(Appendix 8)
In Appendix 7,
The information processing apparatus collates a first feature of a plurality of coordinates included in the first operation information with a second feature of a plurality of coordinates included in the second operation information, and The terminal device cooperation method in which the first operation and the second operation are determined to be the instruction operation of the cooperation operation when the second feature matches the second feature.

(Appendix 9)
In Appendix 6,
The linkage information includes an identifier for identifying a terminal device that is a target of the linkage operation,
The first terminal device operates in cooperation with the second terminal device identified by the identifier of the cooperation information;
The second terminal device is a terminal device cooperation method that operates in cooperation with the first terminal device identified by the identifier of the cooperation information.

(Appendix 10)
In Appendix 7,
If the information processing apparatus determines that the operation is not an instruction operation of the cooperative operation, the information processing apparatus identifies an end portion where the operation is performed based on a plurality of coordinates included in the first operation information, and the first end portion at the identified end portion The first position information indicating the display position of the guide image is transmitted to the first terminal device, and the end where the operation is performed is specified based on the plurality of coordinates included in the second operation information. , A terminal device cooperation method for transmitting second position information indicating the display position of the second guide image at the specified end portion to the second terminal device.

(Appendix 11)
An information processing apparatus that communicates with a first operation terminal and a second terminal apparatus that operate in cooperation with each other,
Received from the second terminal device, the first operation information received from the first terminal device, the first operation information indicating the content of the operation on the end of the operation area of the input device that accepts the operation in the first terminal device In the second terminal device, a storage unit that stores second operation information indicating the content of the operation on the end of the operation region of the input device that receives the operation;
The first operation information and the second operation information are collated, and the first operation on the first terminal device and the second operation on the second terminal device are linked operation instruction operations If it is determined, the information processing unit includes a processing unit that transmits cooperation information indicating that the first terminal device and the second terminal device operate in cooperation to the first operation terminal and the second terminal device. apparatus.

(Appendix 12)
In Appendix 11,
The processing unit collates a first feature of a plurality of coordinates included in the first operation information with a second feature of a plurality of coordinates included in the second operation information, and the first operation information An information processing apparatus that determines that the first operation and the second operation are instruction operations for the cooperative operation when a feature matches the second feature.

(Appendix 13)
When the processing unit determines that the operation is not an instruction operation of the cooperation operation, the processing unit specifies an end portion where the operation is performed based on a plurality of coordinates included in the first operation information, and the first end at the specified end portion is specified. First position information indicating the display position of the guide image is transmitted to the first terminal device, and further, an end where the operation is performed is identified based on a plurality of coordinates included in the second operation information, An information processing apparatus that transmits second position information indicating a display position of a second guide image at the identified end to the second terminal apparatus.

(Appendix 14)
A program executed on a computer operating in cooperation with another computer,
In the computer,
A process of transmitting operation information indicating the content of the operation when the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number of times;
When receiving the cooperation information indicating that the computer and the other computer operate in cooperation, generated based on the comparison between the operation information and the operation information transmitted from the other computer, the other computer A program that executes a process that works in conjunction with the.

(Appendix 15)
In the first terminal device, the first operation information indicating the content of the operation on the end of the operation area of the input device that accepts the operation is received from the first terminal device,
In the second terminal device, the second operation information indicating the content of the operation on the end of the operation area of the input device that receives the operation is received from the second terminal device,
Collating the first operation information with the second operation information;
When the first operation on the first terminal device and the second operation on the second terminal device are determined to be cooperative operation instruction operations, the first terminal device and the second terminal device A program for executing processing for transmitting cooperative information indicating that cooperative operation is performed to the first operating terminal and the second terminal device.

DESCRIPTION OF SYMBOLS 1 ... 1st terminal, 2 ... 2nd terminal, 3 ... Information processing apparatus, 101, 301 ... CPU (processing part), 102, 302 ... Storage apparatus (storage part) ..., 103, 303 ... RAM, 104 ... Display device (display panel), 105 ... Input device (touch panel), 106, 304 ... Communication device (communication unit), 107,305 ... External connection interface device, 11 ... Screen edge entrance / exit estimation unit, 12 ... Information storage unit for determination , 13 ... Screen edge continuation determination unit, 14 ... Determination information transmission unit, 15 ... Determination information deletion unit, 16 ... Cooperation terminal information storage unit, 17 ... Guide processing unit, 31 ... Determination information storage unit, 32 ... Determination information extraction unit, 33 ... cooperation terminal search unit, 34 ... cooperation terminal information storage unit, 35 ... cooperation terminal information transmission unit, 36 ... passage position designation information generation unit.

Claims (10)

A terminal device that operates in cooperation with another terminal device,
A communication unit that communicates with other devices;
When the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number, operation information indicating the content of the operation is transmitted
When receiving the cooperation information indicating that the operation of the terminal device and the other terminal device, which is generated based on the comparison between the operation information and the operation information transmitted from the other terminal device, is performed, A terminal device having a processing unit that executes a cooperative operation with another terminal device. In claim 1,
The operation information includes a plurality of continuous coordinates indicating an operation position when the operation area is continuously operated,
The processing unit detects that the start point or end point of the coordinates is included in the end, and determines that the number of operations is equal to or greater than a predetermined number when the number of detections is equal to or greater than the predetermined number . In claim 2,
When the operation on the end is started, the processing unit stores the operation information in a storage unit,
A terminal device that transmits the operation information stored in the storage unit after transmission together with the transmitted operation information when the number of operations exceeds the predetermined number after the operation information is transmitted. In claim 1,
The processing device, when receiving position information indicating a display position of a guide image indicating an operation position with respect to the end portion to an operator, displays the guide image at the display position. A terminal device cooperation method in a system having a first operation terminal and a second terminal device that operate in cooperation, and an information processing device that communicates with the first operation terminal and the second terminal device,
The first terminal device receives first operation information indicating the content of the operation when the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number. Sent to the information processing device;
The second terminal device having the input device arranged adjacent to the input device of the first terminal device is from the start of the operation to the end of the operation area of the input device that accepts the operation until interrupted. When the number of operations becomes a predetermined number or more, second operation information indicating the content of the operation is transmitted to the information processing device,
The information processing device collates the first operation information with the second operation information, and the first operation for the first terminal device and the second operation for the second terminal device are performed. When it is determined as an instruction operation of cooperative operation, the cooperative information indicating that the first terminal device and the second terminal device operate cooperatively is transmitted to the first operating terminal and the second terminal device,
When the first terminal device receives the cooperation information, the first terminal device performs a cooperation operation with the second terminal device,
When the second terminal device receives the cooperation information, the terminal device cooperation method executes a cooperation operation with the first terminal device. In claim 5,
The first and second operation information includes a plurality of continuous coordinates indicating operation positions when the operation area is continuously operated,
The first operation terminal and the second terminal device detect that a start point or an end point of the coordinates is included in the end, and when the number of detections is equal to or greater than the predetermined number, the number of operations is A terminal device cooperation method for determining that the number of times is greater than or equal to a predetermined number. In claim 6,
When the information processing apparatus determines that the operation is not an instruction operation for the cooperative operation, the information processing apparatus identifies an end portion where the operation is performed based on a plurality of coordinates included in the first operation information, and the first end portion at the identified end portion is identified. The first position information indicating the display position of the guide image is transmitted to the first terminal device, and the end where the operation is performed is specified based on the plurality of coordinates included in the second operation information. , A terminal device cooperation method for transmitting second position information indicating the display position of the second guide image at the specified end portion to the second terminal device. An information processing apparatus that communicates with a first operation terminal and a second terminal apparatus that operate in cooperation with each other,
Received from the second terminal device, the first operation information received from the first terminal device, the first operation information indicating the content of the operation on the end of the operation area of the input device that accepts the operation in the first terminal device In the second terminal device, a storage unit that stores second operation information indicating the content of the operation on the end of the operation region of the input device that receives the operation;
The first operation information and the second operation information are collated, and the first operation on the first terminal device and the second operation on the second terminal device are linked operation instruction operations If it is determined, the information processing unit includes a processing unit that transmits cooperation information indicating that the first terminal device and the second terminal device operate in cooperation to the first operation terminal and the second terminal device. apparatus. A program executed on a computer operating in cooperation with another computer,
In the computer,
A process of transmitting operation information indicating the content of the operation when the number of operations from the start of the operation to the end of the operation area of the input device that accepts the operation to the interruption is greater than or equal to a predetermined number of times;
When receiving the cooperation information indicating that the computer and the other computer operate in cooperation, generated based on the comparison between the operation information and the operation information transmitted from the other computer, the other computer A program that executes a process that works in conjunction with the. Computer
In the first terminal device, the first operation information indicating the content of the operation on the end of the operation area of the input device that accepts the operation is received from the first terminal device,
In the second terminal device, the second operation information indicating the content of the operation on the end of the operation area of the input device that receives the operation is received from the second terminal device,
Collating the first operation information with the second operation information;
When the first operation on the first terminal device and the second operation on the second terminal device are determined to be cooperative operation instruction operations, the first terminal device and the second terminal device A program for executing processing for transmitting cooperative information indicating that cooperative operation is performed to the first operating terminal and the second terminal device.

Images