JP5217782B2 - Camera-based pairing method, system and apparatus, and camera-based pairing program - Google Patents

Description

  The present invention relates to a so-called terminal-to-terminal pairing method in which a call terminal such as a mobile phone and an information processing terminal such as a personal computer (PC) are associated with each other and cooperate with each other. The present invention relates to a camera-use pairing method, system, and apparatus, and a camera-use pairing program that can be reliably paired as described above and can be realized at low cost.

  Patent Document 1 describes a so-called terminal pairing method in which a user of a call terminal such as a telephone that performs a voice call associates the call terminal with an information processing terminal such as a PC and cooperates with each other. . In the method described in Patent Document 1, by doing so, it is possible to simultaneously use a voice call and data sharing for viewing and collaboratively editing document data including text and images with other users in real time. Supports communication between users.

  In the method described in Patent Document 1, first, both users who are making a voice call using a telephone terminal manually input (manually input) terminal identification information such as the telephone number of their telephone terminal into the information processing terminal. Thus, the call terminal and the information processing terminal are paired on the server side, and two pairs are created. Next, data sharing is automatically started between the information processing terminal of one pair and the information processing terminal of the other pair.

  Patent Document 2 describes a pairing method between wireless devices having a wireless communication function. The pairing method described in Patent Document 2 is intended to simplify the pairing operation between a television receiver (hereinafter simply referred to as a television) and a video camera. First, the terminal identification information of the television is displayed on the television sub-screen, and the user takes a picture with the video camera. Then, the video camera extracts the terminal identification information of the television from the captured video, encodes the extracted terminal identification information of the television and the terminal identification information of the video camera together, and uses the wireless communication function for the television. Send. As shown in FIG. 1 of Patent Document 2, the television is equipped with a wireless antenna (antenna 180). When encoded data is received using the wireless antenna, the received encoded data is decoded. If the terminal identification information can be detected, pairing with the video camera is performed.

JP 2005-012380 A JP 2006-135930 A

  However, the method described in Patent Document 1 places a heavy burden on the user during the pairing operation, and thus cannot be easily paired. The reason is that the method described in Patent Document 1 forces a user during a voice call to manually input the telephone number of his / her call terminal into the information processing terminal during the pairing operation. It is. Manual entry of a telephone number is a laborious operation. Further, there are cases where the user remembers the phone number of another person but does not remember his own phone number. Some call terminals have a function to display their own phone number by performing a specific operation, but since many models cannot use functions other than volume adjustment during a call, the user operates the call terminal. You can't even know your phone number. Therefore, a user who does not remember his / her telephone number needs to find out his / her telephone number using other means.

  Further, in the method described in Patent Document 1, a pair different from the user's intention may be assembled, and the pairing reliability is poor. The reason is that in the method described in Patent Document 1, if the user manually inputs an incorrect telephone number into the information processing terminal without noticing the error, the information processing terminal and the other person's call terminal are This is because they are unintentionally paired.

  As an example, consider a scene in which user A and user B, and user C and user D are talking. For example, if the user A manually inputs an incorrect telephone number and happens to be the telephone number of the user C, the information processing terminal of the user A and the telephone terminal of the user C are paired. . As a result, data sharing is started between the information processing terminal of user A and the information processing terminal of user D.

  Therefore, the present invention does not place a heavy burden on the user during the pairing operation, prevents a pair that is different from the user's intention from being formed easily, and can perform pairing reliably. An object is to provide a ring method, a system, an apparatus, and a camera-use pairing program.

  The camera-based pairing system according to the present invention includes a first terminal equipped with a camera and a second terminal. When the first terminal is moved by the user, the camera is Including a first terminal trajectory calculating means for calculating trajectory information indicating a trajectory of movement of the first terminal based on the photographed image, and the second terminal is operated by the user in conjunction with the movement of the first terminal. Including a second terminal trajectory calculating means for calculating trajectory information indicating a trajectory of input coordinates in the second terminal when an input operation is performed from the second terminal; Pairing determination means for determining whether or not to pair the first terminal and the second terminal by comparing the locus information calculated by and the locus information calculated by the second terminal locus calculating means. Further features To.

The camera-based pairing method according to the present invention calculates trajectory information indicating a trajectory of the first terminal based on an image captured by the camera when the first terminal equipped with the camera is moved by the user. A first terminal trajectory calculating step and a trajectory indicating a trajectory of input coordinates in the second terminal when an input operation from the second terminal is performed by the user in conjunction with the movement of the first terminal. By comparing the trajectory information calculated in the second terminal trajectory calculation step , the trajectory information calculated in the first terminal trajectory calculation step, and the trajectory information calculated in the second terminal trajectory calculation step , the first terminal And a pairing determination step for determining whether or not to pair with the second terminal.

  A pairing device according to the present invention is a pairing device that performs pairing between a first terminal equipped with a camera and a second terminal, and when the first terminal is moved by a user, the camera is When the user performs an input operation from the second terminal in conjunction with the movement of the first terminal in conjunction with the movement of the first terminal, the locus information obtained by calculating the locus of movement of the first terminal based on the photographed image And a pairing determination means for determining whether or not to pair the first terminal and the second terminal by comparing with the locus information obtained by calculating the locus of the input coordinates in the second terminal. It is characterized by that.

A pairing program according to the present invention is a pairing program for performing pairing between a first terminal equipped with a camera and a second terminal, and the first terminal is moved to the computer by the user. The trajectory information obtained by calculating the trajectory of the movement of the first terminal based on the image taken by the camera and the second terminal from the second terminal by the user in conjunction with the movement of the first terminal. When an input operation is performed, it is determined whether or not to pair the first terminal and the second terminal by comparing with the locus information obtained by calculating the locus of the input coordinates in the second terminal. This is for executing the pairing determination process.

  ADVANTAGE OF THE INVENTION According to this invention, it does not place a big burden on a user in the case of a pairing operation | work, and it can prevent that the pair different from a user's intention is built, and can perform pairing reliably.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an example of a network configuration in the embodiment of the present invention. As shown in FIG. 1, the pair management device 100, the call terminal A10, the call terminal B11, the information processing terminal A20, and the information processing terminal B21 are mutually connected via a communication network such as the Internet or a mobile phone network. It is connected. In this embodiment, a case where there are two call terminals and two information processing terminals is shown, but any number (for example, three or more) may be used. In this embodiment, a call terminal and an information processing terminal are used as terminals, but a terminal that does not have a call function (for example, a PDA (Personal Digital Assistant) or the like) may be used.

  FIG. 2 is a functional block diagram showing a functional configuration of the camera-based pairing method. FIG. 3 is an explanatory diagram showing an example of a data structure in the embodiment of the present invention. This will be described below with reference to these drawings.

  Specifically, the pair management apparatus 100 is realized by an information processing apparatus such as a personal computer that operates according to a program. As shown in FIG. 2, the pair management apparatus 100 includes a trajectory comparison unit 101, a pair storage unit 102, and a trajectory storage unit 103.

  The trajectory comparison unit 101 is specifically realized by a CPU of an information processing apparatus that operates according to a program. The trajectory comparison means 101 generates a trajectory 210 based on the trajectory 220 received from the call terminal and the trajectory 230 received from the information processing terminal, and based on the generated trajectory 210, a pair of the call terminal and the information processing terminal. And a function for generating the pair 200 when it is determined that a pair is to be formed.

  Specifically, the pair storage unit 102 is realized by a storage device such as a magnetic disk device or an optical disk device. The pair storage unit 102 stores a plurality of pairs 200. There are as many pairs 200 as there are established pairs. This is information corresponding to a pair of “terminal number” and “cooperative PC IP address” shown in FIG. 2 and FIG.

  The pair 200 includes a call terminal identifier 201 and an information processing terminal identifier 202.

  The call terminal identifier 201 is address information that identifies the call terminal A10 and the call terminal B11. Specifically, for example, a telephone number, a SIP address, an IP address, a MAC address, etc., are not limited thereto.

  The information processing terminal identifier 202 is address information that identifies the information processing terminal A20 and the information processing terminal B21. Specifically, for example, a telephone number, a SIP address, an IP address, a MAC address, etc., are not limited thereto.

  Specifically, the trajectory storage means 103 is realized by a storage device such as a magnetic disk device or an optical disk device. The trajectory storage unit 103 stores a plurality of trajectories 210. The trajectory storage means 103 collectively manages a series of trajectories 210 for each vibration operation of the call terminal and the input means 121 by the user. For example, when two users perform vibration operations separately, two sets of a series of trajectories 210 are generated.

  The trajectory 210 includes a time 211, a call terminal X coordinate 212, a call terminal Y coordinate 213, an information processing terminal X coordinate 214, and an information processing terminal Y coordinate 215.

  Time 211 is the time when the call terminal or the information processing terminal collects the trajectory information. However, the time 211 is not the time on the clock mounted on the call terminal or the information processing terminal, but the time on the clock mounted on the pair management apparatus 100.

  The call terminal X coordinate 212 is the value of the X coordinate of the position of the call terminal on the XY plane collected by the call terminal at time 211.

  The call terminal Y coordinate 213 is the value of the Y coordinate of the position of the call terminal on the XY plane collected by the call terminal at time 211.

  The information processing terminal X coordinate 214 is the value of the X coordinate of the position of the information processing terminal on the XY plane collected by the information processing terminal at time 211.

  The information processing terminal Y coordinate 215 is the value of the Y coordinate of the position of the information processing terminal on the XY plane collected by the information processing terminal at time 211.

  Specifically, the call terminal A10 is realized by a mobile phone, a smart phone, a fixed phone, a street multimedia terminal, an in-vehicle terminal, a game machine, or other similar device having a function of exchanging information with the outside, which operates according to a program. The Since the configuration of the call terminal B11 is the same as that of the call terminal A10, the description is omitted.

  As shown in FIG. 2, the call terminal A10 includes a photographing unit A111, a moving image decomposition unit A112, a still image difference detection unit A113, a trajectory calculation unit A114, a vibration monitoring unit A115, a time correction unit A116, and a trajectory storage unit A117. Including.

  Specifically, the photographing unit A111 is realized by a camera capable of photographing a moving image.

  Specifically, the moving image decomposing means A112 is realized by the CPU of the portable terminal A that operates according to a program. The moving image decomposing unit A112 has a function of analyzing a moving image captured by the image capturing unit A111 and extracting a series of still images constituting the moving image.

  More specifically, the still image difference detection means A113 is realized by the CPU of the portable terminal A that operates according to a program. The still image difference detection unit A113 has a function of analyzing a series of still images extracted by the moving image decomposition unit A112 and calculating a physical movement direction and a movement distance of the photographing unit A111 itself. Specifically, for example, the still image difference detection unit A113, for a moving image shot while the shooting unit A111 is physically moving, a certain still image extracted by the moving image decomposition unit A112, and a still image extracted next. The difference from the image is detected. As a result, the still image difference detection means A113 determines the physical movement direction and movement distance of the photographing means A111 itself on the XY plane with the horizontal direction of the still image as the X axis and the vertical direction as the Y axis. calculate.

  The locus calculation means A114 is specifically realized by the CPU of the portable terminal A that operates according to a program. The trajectory calculating means A114 successively displays a series of physical movement directions and distances of the photographing means A111 itself obtained as a result of the still image difference detection means A113 analyzing a series of still images. A function of calculating the trajectory of the physical movement of the photographing means A111 itself by being connected is provided.

  Specifically, the vibration monitoring unit A115 is realized by the CPU of the portable terminal A that operates according to a program. The vibration monitoring means A115 monitors the trajectory calculated by the trajectory calculating means A114, and that the photographing means A111 has started to be moved by the user's hand (vibration start) and that the vibration has ended and the user's hand has stopped (vibration). End). The vibration monitoring unit A115 has a function of calculating in which direction on the XY plane the imaging unit A111 starts moving (initial inclination) when the vibration start is detected.

  Specifically, the time correction means A116 is realized by the CPU of the portable terminal A that operates according to a program. The time correction means A116 calculates the time difference between the clocks by exchanging the time information of the clock installed in the pair management device 100 and the time information of the clock installed in the call terminal A10, and is originally installed in the call terminal A10. A function of converting the time 221 stored as the time on the clock to be converted into the time on the clock mounted on the pair management apparatus 100.

  Specifically, the trajectory storage unit A117 is realized by a storage device such as a magnetic disk device or an optical disk device. The trajectory storage unit A117 includes a plurality of trajectories 220 and an initial inclination 224. The trajectory storage unit A117 calculates, as the trajectory 220, the trajectory calculation unit A114 based on the physical movement direction and distance of the photographing unit A111 itself calculated by the still image difference detection unit A113 from two still images. The position of the photographing means A111 at that time on the XY plane is stored.

  The locus 220 includes a time 221, an X coordinate 222, and a Y coordinate 223 as shown in FIG. 3.

  The time 221 is the time when the second still image of the two still images was taken. The time 221 is initially stored as the time on the clock mounted on the call terminal A10, but later converted to the time on the clock mounted on the pair management apparatus 100 and is overwritten and stored.

  The X coordinate 222 is the value of the X coordinate of the position of the photographing unit A111 on the XY plane at the time 221.

  The Y coordinate 223 is the value of the Y coordinate of the position of the imaging unit A111 on the XY plane at the time 221.

  The initial inclination 224 is calculated when the vibration monitoring unit A115 detects the start of vibration, and is information indicating in which direction on the XY plane the imaging unit A111 starts to move. Specifically, the initial inclination 224 corresponds to the inclination of a straight line when the direction in which the photographing unit A111 starts moving is expressed by a straight line passing through the origin of the XY plane.

  Specifically, the information processing terminal A20 includes a personal computer (PC), a portable computer (PDA), a mobile phone, a smartphone, a fixed phone, a street multimedia terminal, an in-vehicle terminal, a TV with a network connection function, which operates according to a program, It is realized by a set-top box with a network connection function, a game machine, and other similar devices having a function of exchanging information with the outside. Since the configuration of the information processing terminal B21 is the same as that of the information processing terminal A20, the description thereof is omitted.

  As shown in FIG. 3, the information processing terminal A20 includes input means A123, trajectory calculation means A124, vibration monitoring means A125, time correction means A126, and trajectory storage means A127.

  Specifically, the input means A123 has a touch pad provided in a mouse, a notebook PC, etc., a touch panel display device provided in a PDA, a smartphone, a street multimedia terminal, or the like, or a function that allows an input operation with a user's hand or finger. It is realized by other similar devices provided.

  Hereinafter, a mouse cursor on a mouse or a touch pad, or a location where a user touches the display in a touch panel display device is referred to as an “operation target point”.

  The trajectory calculation means A124 is specifically realized by a CPU of an information processing terminal that operates according to a program. The trajectory calculation unit A124 moves the “operation target point” by successively connecting a series of movement directions and distances of the “operation target point” obtained as a result of the user operating the input unit A123. Has a function to calculate the trajectory.

  Specifically, the vibration monitoring unit A125 is realized by a CPU of an information processing terminal that operates according to a program. The vibration monitoring unit A125 monitors the trajectory calculated by the trajectory calculating unit A124, and the fact that the “operation target point” has started to be moved by the user's hand (vibration start) and that the user's hand has stopped after the vibration has ended. (Vibration end) is detected. When the vibration monitoring unit A125 detects the start of vibration, the “operation target point” indicates which of the XY planes the X-axis is the horizontal direction of the screen of the information processing terminal A20 and the Y-axis is the vertical direction. It has a function to calculate whether it has started moving in the direction (initial inclination).

  Specifically, the time correction means A126 is realized by a CPU of an information processing terminal that operates according to a program. The time correction means A126 calculates the time difference between the clocks by exchanging the time information of the clock mounted on the pair management device 100 and the time information of the clock mounted on the information processing terminal A20, and originally the information processing terminal A20. Has a function of converting the time 231 stored as the time on the clock mounted on the watch to the time on the clock mounted on the pair management apparatus 100.

  The trajectory storage means A127 is specifically realized by a storage device such as a magnetic disk device or an optical disk device. The trajectory storage unit A127 stores a plurality of trajectories 230 and an initial inclination 234.

  The trajectory storage unit A127 stores, as the trajectory 220, the position on the XY plane of the “operation target point” calculated by the trajectory calculation unit A124 and the time at that time. As shown in FIG. 3, the locus 220 includes a time 231, an X coordinate 232, and a Y coordinate 233.

  Time 231 is the time when the locus calculation means A124 calculated the position of the “operation target point”. The time 231 is initially stored as the time on the clock mounted on the call terminal A10, but later converted to the time on the clock of the pair management apparatus 100 and is overwritten and stored.

  The X coordinate 232 is the value of the X coordinate of the position of the “operation target point” on the XY plane at the time 231.

  The Y coordinate 233 is the value of the Y coordinate of the position of the “operation target point” on the XY plane at the time 231.

  The initial inclination 224 is calculated when the vibration monitoring unit A125 detects the start of vibration, and is information indicating in which direction on the XY plane the “operation target point” starts to move. Specifically, the initial inclination 224 corresponds to the inclination of a straight line when the direction in which the “operation target point” starts to move is expressed by a straight line passing through the origin of the XY plane.

  Next, the operation will be described. FIG. 4 is a flowchart illustrating an example of processing of the camera-based pairing method. In the present embodiment, description will be made on the assumption that the user A owns the call terminal A10 and the information processing terminal A20, and the user B owns the call terminal B11 and the information processing terminal B21.

  First, when it is desired to perform pairing of the terminal, the user A operates the call terminal A10 to display a pairing execution screen. Then, the photographing unit A111 automatically starts photographing, and the call terminal A10 displays the moving image being photographed on the pairing execution screen. Next, the moving image decomposing unit A112 analyzes the moving image captured by the imaging unit A111, and starts a process of extracting a series of still images constituting the moving image from the analyzed moving image.

  Next, the still image difference detection unit A113 analyzes a series of still images extracted by the moving image decomposition unit A112, and based on the analysis result, determines the physical movement direction and movement distance of the photographing unit A111 itself. The calculation process is started. Next, the trajectory calculating means A114 connects the series of physical movement directions and movement distances of the imaging means A111 itself, calculated by the still image difference detection means A113, one after another, so that the imaging means A111 itself The process for calculating the trajectory of physical movement is started. The trajectory calculation process by the trajectory calculation unit A114 is continuously executed while the photographing unit A111 is capturing a moving image.

  Next, the vibration monitoring unit A115 starts a process of monitoring the start of the vibration operation by the user A based on the locus calculated by the locus calculating unit A114. Specifically, first, the vibration monitoring unit A115 obtains the latest position of the photographing unit A111 (coordinates on the XY plane) and a predetermined time from the information on the locus calculated by the locus calculating unit A114. Two positions are obtained (extracted), the position of the photographing means A111 to be used. Next, the vibration monitoring unit A115 calculates the distance between the two positions, and checks whether or not it exceeds a predetermined value. Further, the call terminal A10 starts a timer.

  Next, the user A operates the information processing terminal A20 to perform an operation for displaying a screen for executing pairing. Then, the trajectory calculation unit A124 connects the series of movement directions and movement distances of the “operation target points” obtained as a result of the user operating the input unit A123, one after another, thereby obtaining the “operation target points”. The process of calculating the movement trajectory is started. The trajectory calculation unit A124 continues to execute the trajectory calculation process while the user operates the input unit A123.

  Next, the vibration monitoring unit A125 starts a process of monitoring the start of the vibration operation by the user A based on the locus calculated by the locus calculating unit A124. Specifically, the vibration monitoring unit A125 first obtains the latest position of the input unit A123 (coordinates on the XY plane) and a predetermined time from the information on the locus calculated by the locus calculating unit A124. Two positions, ie, the position of the input means A123 to be obtained are acquired (extracted). Next, the vibration monitoring unit A125 calculates the distance between the two positions, and checks whether or not it exceeds a predetermined value. Thereafter, the user A brings the call terminal A10 onto the input means A121 and holds it together (step S300).

  When the predetermined time has elapsed, the timer of the call terminal A10 activated in step S300 times out, and the call terminal A10 starts a vibration operation to the user A by sounding an electronic sound or the like. Tell them that it ’s okay. As a notification means, instead of sounding a sound, if a sub-display device or the like is provided on the same surface as the installation surface of the photographing means A111, a message may be displayed on the sub-display device. Further, the vibration may be generated at the call terminal A10 using the vibrator function. In response to this, the user A starts moving the call terminal A10 and the input unit A121 left and right together (vibration start) (step S301).

  The vibration monitoring means A115 detects the start of vibration of the call terminal A10. Then, the vibration monitoring means A115 passes through the coordinates on the XY plane of the latest photographing means A111 at the time when the vibration start is detected and the coordinates of the photographing means A111 obtained by going back a predetermined time. The inclination of the straight line on the plane is calculated, and the value is stored in the initial inclination 224. Further, the trajectory calculation unit A114 starts saving the time 221, the X coordinate 222, and the Y coordinate 223 to the trajectory 220 (step S302). Thereby, the locus | trajectory 220 is newly produced | generated one after another during the vibration operation of the user A. Here, the time 221 is a time on a clock mounted on the call terminal A10.

  Next, the vibration monitoring unit A115 starts processing for monitoring the end of the vibration operation by the user A based on the locus calculated by the locus calculating unit A114. Specifically, first, the vibration monitoring unit A115 acquires (extracts) two positions as in the process of step S300. Next, the vibration monitoring means A115 calculates the distance between the two positions and checks whether or not the distance is below a predetermined value.

  The vibration monitoring unit A125 detects the start of vibration of the input unit A121. Then, the vibration monitoring unit A125 stores the initial inclination 234 in the same manner as the vibration monitoring unit A115. In addition, the trajectory calculation unit A124 starts saving the time 231, the X coordinate 232, and the Y coordinate 233 to the trajectory 230. As a result, new trajectories 230 are successively generated during the vibration operation of the user A. Here, the time 231 is a time on a clock mounted on the information processing terminal A20.

  Next, the vibration monitoring unit A125 starts processing for monitoring the end of the vibration operation by the user A based on the trajectory calculated by the trajectory calculating unit A124. Specifically, first, the vibration monitoring unit A125 acquires (extracts) two positions in the same manner as in the process of step S300. Next, the vibration monitoring unit A125 calculates the distance between the two positions and checks whether or not the distance is below a predetermined value.

  The user A finishes moving the call terminal A10 and the input unit A121 left and right several times together, and stops moving (end of vibration) (step S303).

  The vibration monitoring means A115 detects the end of vibration of the call terminal A10. Then, the trajectory calculation unit A114 normalizes the X coordinate 222 and the Y coordinate 223 for all the trajectories 220. Even when the resolution of the moving image of the imaging unit A111 and the resolution of the screen of the information processing terminal A20 are different, the scale of the coordinates can be made uniform by this normalization processing. Therefore, in step S307, the trajectory comparison unit 101 The trajectories can be compared.

  Specifically, the normalization processing by the trajectory calculation unit A acquires (extracts) | maximum value | and | minimum value | for all the X coordinates 222, and sets each X coordinate 222 to (X coordinate 222). ) ÷ (Overwrite with the value of | maximum value | and | minimum value | the larger value). Here, | * | is the absolute value of *. For example, when maximum value = 20, minimum value = −30, and X coordinate 222 = 15, | minimum value | = 30 is larger than | maximum value | = 20, so 15 ÷ 30 = 0. 5 to overwrite. Similar processing is performed for the Y coordinate 223.

  Next, the time correction means A116 calculates the time difference between the clocks by exchanging the time information of the clock mounted on the pair management device 100 and the time information of the clock mounted on the call terminal A10. The method for calculating the time difference is described in, for example, a document (RFC 2030, “Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI”, October 1996, http://tools.ietf.org/rfc/rfc2030). .txt).

  Next, using the calculated time difference, the time correction unit A116 converts the time 221 stored as the time on the clock mounted on the call terminal A10 into the time on the clock mounted on the pair management apparatus 100 and overwrites it. save. For example, in the case where it is calculated that the clock mounted on the pair management apparatus 100 is delayed by 1 second from the clock mounted on the call terminal A10, the conversion processing is performed using the time 221 for all the trajectories 220. A process of delaying by 1 second is performed.

  Thereafter, the call terminal A10 transmits the identifier of the call terminal A10, the initial inclination 224, and all the trajectories 220 to the trajectory comparison means 101. Finally, the call terminal A10 updates the screen of the call terminal A10 and displays a message indicating that processing is in progress. In addition, the information processing terminal A20 performs the same process as the call terminal A10 (step S304).

  When the trajectory comparison unit 101 receives the initial inclination 224, the trajectory 220, the initial inclination 234, and the trajectory 230, the trajectory comparison unit 101 acquires (extracts) the first time 221 and time 231 from the trajectory 220 and the trajectory 230, and compares the time difference. It is determined whether or not is within a predetermined value. Further, the trajectory comparison unit 101 acquires (extracts) the last time 221 and time 231 and determines whether or not the time difference is within a predetermined value (step S305).

  If either one exceeds the predetermined value, the trajectory comparison unit 101 determines that the pair of the call terminal A10 and the information processing terminal A20 is not established (step S309). Thereafter, the trajectory comparing means 101 notifies both terminals of that fact, and the processing returns to the beginning as shown in FIG.

  When both are within the predetermined value, the trajectory comparison unit 101 extracts and extracts only the trajectory 220 and the trajectory 230 in which the time 221 and the time 231 overlap from the received trajectory 220 and the trajectory 230. Based on the trajectory 220 and the trajectory 230, a trajectory 210 is newly generated. Thereafter, as shown in FIG. 4, the process proceeds to step S306.

  Specifically, first, the trajectory comparison means 101 acquires (extracts) the later time of the first time 221 and the time 231, and the trajectory 220 having the time 221 earlier than this time, All the trajectories 230 having the earlier time 231 are discarded. Next, the trajectory comparison unit 101 acquires (extracts) the earlier time 221 between the last time 221 and the time 231, and obtains a trajectory 220 having a time 221 later than this time and a time 231 later than this time. All the trajectories 230 that are held are discarded. As a result, only the trajectory 220 and the trajectory 230 in which the time 221 and the time 231 overlap are left.

  By this processing, even if the detection timing of the vibration start in step S302 and the vibration end in step S304 is not accurately synchronized between the call terminal A10 and the information processing terminal A20, the trajectory comparison unit 101 is as accurate as possible. In step S307, it is possible to determine the coincidence of the locus.

  Next, the trajectory comparison means 101 counts the number of the trajectories 220 and 230, calculates the value of (the larger number) / (the smaller number), and rounds off the decimal part. ". For example, when the number of trajectories 220 = 100 and the number of trajectories 230 = 50, “sampling speed ratio” = 100 ÷ 50 = 2. Next, the trajectory comparison unit 101 copies the larger one of the trajectories 220 and 230 to the trajectory 210. In the above example, 100 trajectories 210 are newly generated, values are filled in the time 211, the call terminal X coordinate 212, and the call terminal Y coordinate 213, and the information processing terminal X coordinate 214 and the information processing terminal Y coordinate 215 are The value is empty.

  Next, the trajectory comparison unit 101 copies the X coordinate and the Y coordinate of the trajectory 220 and the trajectory 230 which are smaller in number to the trajectory 210. However, during this copying, the trajectory comparison unit 101 extracts the trajectory 210 in ascending order of time 211 as many times as the “sampling speed ratio”, and the extracted one or more trajectories 210 are empty. Of the trajectory 220 and the trajectory 230, the smaller X coordinate and Y coordinate are copied to the X coordinate and Y coordinate. In the above example, two of the trajectory 210 having the earliest time 211 and the trajectory 210 having the second earliest time 211 are extracted, and the trajectory 230 having the earliest time 231 is displayed on both of the two information processing terminal X coordinates 214. The X coordinate 232 is copied, and the Y coordinate 233 of the locus 230 with the earliest time 231 is copied to both of the two information processing terminal Y coordinates 215. Sequentially, this process is performed on subsequent trajectories 210, such as trajectory 210 with the earliest time 211 and trajectory 210 with the fourth earliest time 211.

  By this process, the trajectory comparison unit 101 temporarily determines the number of times (sampling speed) that the moving image decomposition unit A112 decomposes into a still image per unit time and the input unit A121 determines the position of the “operation target point” per unit time. Even if the number of times of sampling (sampling speed) is different, the difference between the two sampling speeds can be absorbed.

  The trajectory comparison unit 101 performs processing for converting the values of the call terminal X coordinate 212 and the call terminal Y coordinate 213 for all the trajectories 210 using the initial inclination 224 and the initial inclination 234 (step S306). The reason for this conversion process will be described with reference to FIGS. 5 and 6 are diagrams for explaining the reason why the coordinate system needs to be rotated. In the following description, it is assumed that the call terminal A10 is a mobile phone provided with a camera on the back surface, and the information processing terminal A20 is a PC provided with a mouse.

  As shown in FIGS. 5A and 5B, the mobile phone and the mouse have different coordinate systems on the XY plane. In the case of a mobile phone, the central portion of the camera lens is the origin of the coordinate system, and in the case of a mouse, the portion where the ball in the mouse is grounded is the origin of the coordinate system.

  Here, it is considered that the user rarely holds the mobile phone and the mouse together so that the Y coordinate of the mobile phone and the Y coordinate of the mouse are exactly parallel as shown in FIG. . Rather, in most cases, the user will hold the mobile phone and mouse together, as shown in (b), in such a way that the two Y coordinates are not parallel.

  In the case of (a), it is considered that it is determined that the locus of the mobile phone and the mouse match in step S307 without particularly converting the values of the call terminal X coordinate 212 and the call terminal Y coordinate 213. Therefore, there is no particular problem in the case of (a).

  On the other hand, in the case of (b), it is erroneously determined that the locus of the mobile phone and the mouse do not match in step S307. This is because (b-1) in FIG. 6 separates the coordinate system of the mouse and the coordinate system of the mobile phone so that the coordinate system of the mobile phone tilted in (b) is not tilted. However, in the coordinate system of the mouse, the locus that starts to move in the 2 o'clock direction of the watch is the locus that starts to move in the 5 o'clock direction of the watch in the coordinate system of the mobile phone. As a matter of course, in this state, it is determined in step S307 that the two loci do not match, and an incorrect determination result is obtained.

  Therefore, rotate the coordinate system of the mobile phone to be point-symmetric about the origin so that the trajectory of the mobile phone that starts to move in the 5 o'clock direction of the watch becomes the trajectory that starts to move in the 2 o'clock direction of the watch. To solve this problem. This rotation processing can be realized using a method widely known as “affine transformation”. That is, in this step, a process of converting the values of the call terminal X coordinate 212 and the call terminal Y coordinate 213 using “affine transformation” is performed.

  First, the trajectory comparison unit 101 uses the initial inclination 224 and the initial inclination 234 to calculate an angle for rotating the coordinate system of the call terminal A10. Next, the trajectory comparison unit 101 converts the values of the call terminal X coordinate 212 and the call terminal Y coordinate 213 using “affine transformation” for all the trajectories 210, and converts the converted values to the call terminal X coordinates. 212 and the call terminal Y coordinate 213 are overwritten and saved.

  The trajectory comparison unit 101 determines whether or not the trajectories match by comparing the trajectory of the call terminal A10 and the trajectory of the information processing terminal A20 in time series (step S307). FIG. 7 is an explanatory diagram schematically showing a time-series comparison of the trajectory between the call terminal and the information processing terminal. For simplification of description, the description will be made on the assumption that “sampling speed ratio” = 1.

  As shown in FIG. 7, the dotted line is the locus of the call terminal A10, and the solid line is the locus of the information processing terminal A20. The coordinates of the black dot on the dotted line are given by (call terminal X coordinate 212. call terminal Y coordinate 213), respectively. Similarly, the coordinates of the black point on the solid line are given by (information processing terminal X coordinate 214. information processing terminal Y coordinate 215), respectively. (Call terminal X coordinate 212. Call terminal Y coordinate 213) and (Information processing terminal X coordinate 214. Information processing terminal Y coordinate 215) having a common time 211 are connected by a straight line. Here, the length of each straight line is defined as “distance between tracks”.

  First, the trajectory comparison unit 101 calculates “distance between trajectories” for all the trajectories 210. Next, the trajectory comparison unit 101 calculates a ratio when the “distance between trajectories” is within a predetermined value with respect to the total number of trajectories 210. Then, the trajectory comparison unit 101 determines whether or not the calculated ratio is below a predetermined value.

  Here, for example, let us consider a case where the timings of operations for vibrating the call terminal and the mouse between the user A and the user B overlap. Even in this case, the trajectory information received by the pair management device is different for each user. Therefore, the pair management device can form a pair of a call terminal held by each user with a mouse and an information processing terminal to which the mouse is connected, as intended by each user.

  If the calculated ratio falls below the specified value, the trajectory comparison unit 101 determines that a pair of the call terminal A10 and the information processing terminal A20 is established (step S308). First, the trajectory comparing means 101 notifies both terminals of that fact. Next, the trajectory comparison unit 101 newly generates the pair 200 and stores the identifiers transmitted by the call terminal A10 and the information processing terminal A20 in step S304 as the call terminal identifier 201 and the information processing terminal identifier 202, respectively.

  If the calculated ratio does not fall below the specified value, the trajectory comparison unit 101 determines that the pair of the call terminal A10 and the information processing terminal A20 is not established (step S309). Thereafter, the trajectory comparing means 101 notifies both terminals of that fact, and the processing returns to the beginning as shown in FIG.

  As described above, according to the present embodiment, simple pairing can be performed without placing a heavy burden on the user during the pairing operation. The reason is that the user does not need to manually enter the phone number of his / her call terminal, and the user can hold his / her call terminal together with the mouse of his / her information processing terminal and vibrate. It is because it can do.

  Further, according to the present embodiment, it is possible to perform reliable pairing that can prevent a pair different from the user's intention from being formed. The reason is that even if the timing of the operation of vibrating the call terminals and mice of a plurality of users overlaps, the trace information of the call terminals and mouse obtained by vibration is compared in the pair management device to form a pair. This is because determining whether or not makes it possible to form a pair of a call terminal held together with the mouse by the user and an information processing terminal to which the mouse is connected as intended by the user.

  Furthermore, according to the present embodiment, there is an effect that a pair can be assembled even if the resolution of the moving image captured by the call terminal is different from the resolution of the screen of the information processing terminal (see step S304).

  Furthermore, according to the present embodiment, there is an effect that a pair can be assembled even if the time between the call terminal and the information processing terminal is not synchronized in advance (see step S304).

  Furthermore, according to this embodiment, even if the detection timing of vibration start and vibration end is not accurately synchronized between the call terminal and the information processing terminal, an effect that a pair can be assembled is obtained. (See step S305).

  Furthermore, according to the present embodiment, there is an effect that a pair can be assembled even if the sampling rate of the moving image decomposition unit 112 and the sampling rate of the input unit 121 are not matched in advance. (See step S305).

  Furthermore, according to the present embodiment, there is an effect that a pair can be assembled even if the user does not bother the direction of the call terminal and the information processing terminal. (See step S306).

  In addition, the method described in Patent Document 2 has a problem in that an information processing terminal that does not include wireless communication hardware cannot perform pairing, and thus cannot implement a system at low cost. The reason is described in Patent Document 2 on the pairing of a TV and a video camera. However, when the TV is replaced with an information processing terminal such as a PC and the video camera is replaced with a call terminal such as a mobile phone, the desktop This is because a PC usually does not include hardware for wireless communication, and therefore, using the method described in Patent Document 2, pairing cannot be performed with a desktop PC. Therefore, in order to enable pairing even with a desktop PC, it is necessary to purchase an expansion board for wireless communication. As a result, the system cannot be realized at low cost.

  According to the present embodiment, even an information processing terminal that does not include hardware for wireless communication can perform pairing, and an inexpensive pairing method can be provided. This is because the information processing terminal does not need to have wireless communication hardware for communication, and can communicate using wired communication such as a LAN.

  Next, an embodiment of the present invention will be described with reference to FIGS. When the user A performs pairing between the call terminal A10 and the information processing terminal A20, what kind of screen transition occurs or what kind of operation is performed will be described with reference to FIG. FIG. 8 is an explanatory diagram showing a screen example and an operation example.

  FIG. 8A is a pairing execution screen displayed by the user A operating the call terminal A10 and the information processing terminal A20 in step S300. FIG. 8 (2) shows an operation in which the user A vibrates the call terminal A10 and the input unit A121 left and right together in step S301.

  FIG. 8 (3) is a screen on which a message indicating that processing is in progress is displayed in step S304. FIG. 8 (4) is a screen on which a message indicating that a pair has been established is displayed in step S308.

  FIG. 9 is an explanatory diagram showing how to pair an information processing terminal having input means other than a mouse. As shown in FIG. 9 (5), when pairing an information processing terminal equipped with a touch pad such as a notebook PC and a call terminal, hold the call terminal with the hand and use the thumb or middle finger of the same hand. The touchpad can be operated. As shown in FIG. 9 (6), when pairing an information processing terminal equipped with a touch panel display device such as a PDA, a smartphone, or a street multimedia terminal with a call terminal, while holding the call terminal by hand, The touch panel display device may be operated with the thumb or middle finger of the same hand.

  Next, the minimum configuration of the camera-based pairing system according to the present invention will be described. FIG. 10 is a block diagram illustrating a minimum configuration example of the camera-based pairing system. As shown in FIG. 10, the camera-based pairing system includes a first terminal 10 equipped with a camera, a second terminal 20, and a pair management device 100 as the minimum components. The first terminal 10 includes a first terminal trajectory calculation unit A114, the second terminal 20 includes a second terminal trajectory calculation unit A124, and the pair management apparatus 100 includes a trajectory comparison unit 101 and including.

  In the camera-based pairing system with the minimum configuration shown in FIG. 10, the first terminal trajectory calculating means A114 is based on the image captured by the camera when the first terminal 10 is moved by the user. 10 has a function of calculating trajectory information indicating the trajectory moved. Further, the second terminal trajectory calculating means A124 calculates the input coordinates of the second terminal 20 when the user performs an input operation from the second terminal 20 in conjunction with the movement of the first terminal 10. A function of calculating trajectory information indicating the trajectory is provided. In addition, the trajectory comparison unit 101 compares the trajectory information calculated by the first terminal trajectory calculation unit with the trajectory information calculated by the second terminal trajectory calculation unit, so that the first terminal 10 and the second terminal 10 A function of determining whether to pair with the terminal 20 is provided.

  10, the pair management apparatus 100 compares the trajectory information received from the first terminal 10 with the trajectory information received from the second terminal 20, and compares the trajectory information received from the second terminal 20. If it is determined that the two pieces of trajectory information are close, a pair of the first terminal 10 and the second terminal 20 is established. Therefore, pairing can be easily performed without burdening the user such as inputting a number. In addition, for example, even when the timings of operations for vibration of a plurality of users overlap, the trajectory information received by the pair management apparatus 100 differs for each user. Therefore, the first terminal 10 and the second terminal 20 can be reliably paired as intended by the user.

  In addition, in said embodiment and Example, the characteristic structure of the camera utilization pairing method as shown to the following (1)-(7) is shown.

(1) The camera-based pairing method is performed when the first terminal (for example, realized by the call terminal A10) equipped with a camera (for example, realized by the photographing unit A111) is moved by the user. A first terminal trajectory calculating step (for example, realized by the trajectory calculating means A114) for calculating trajectory information (eg, trajectory 220) indicating the trajectory of the movement of the first terminal based on the image taken by When an input operation from the second terminal (for example, realized by the information processing terminal A20) is performed by the user in conjunction with the movement of the first terminal, the locus of the input coordinates on the second terminal is A second terminal trajectory calculating step (for example, realized by the trajectory calculating means A124) for calculating the trajectory information (for example, the trajectory 230), and a first end By comparing the trajectory information calculated by the trajectory calculation means for use with the trajectory information calculated by the trajectory calculation means for the second terminal, it is determined whether to pair the first terminal and the second terminal. And a pairing determination step (for example, realized by the trajectory comparison means 101).

(2) In the camera-based pairing method, in the first terminal trajectory calculation step, the movement position (for example, the X coordinate 222 and the Y coordinate 223) to which the first terminal has moved and the first terminal at the movement position. Trajectory information including the time of movement (for example, time 221) is calculated, and the movement position (for example, X coordinate 232 and Y coordinate 233) to which the second terminal has moved in the second terminal trajectory calculation step. And the time when the second terminal moves to the moving position (for example, time 231) in association with each other is calculated, and in the pairing determination step, the first terminal trajectory calculating means calculates at the same time. When it is determined that the difference between the movement position included in the trajectory information and the movement position included in the trajectory information calculated by the second terminal trajectory calculation means is within a predetermined distance, the first terminal and the second terminal Terminal The may be configured to determine that the pairing.

(3) In the camera-based pairing method, the first terminal and the second terminal display a pairing execution screen and hold the input means of the first terminal and the second terminal together. A pairing method using a camera for performing pairing between the first terminal and the second terminal based on an operation of vibrating the terminal and the input means, wherein the vibration between the first terminal and the input means A vibration monitoring step (for example, realized by the vibration monitoring means A115) that detects the start of operation, calculates an initial inclination between the first terminal and the second terminal, and starts storing the trajectory information; When the operation of vibrating the terminal and the input unit is finished, the end of the vibration operation between the first terminal and the input unit is detected (for example, realized by the vibration monitoring unit A115), and the stored trajectory information Normalize the X coordinate and the Y coordinate, and the first terminal The time difference between the clock mounted and the clock mounted on the pair management device is calculated, and the storage time of the trajectory information is converted into the time on the clock mounted on the pair management device based on the calculated time difference (for example, time correction means A116). The transmission step of transmitting the initial inclination (for example, the initial inclination 224) and the trajectory information to the pair management device, and the difference between the vibration start time and the vibration end time based on the received trajectory information. A time difference determination step (for example, realized by the locus comparison means 101) for determining whether or not both are within a predetermined value, and a coordinate system rotation step (for example, locus comparison) for rotating the coordinate system of the first terminal using the initial inclination. And a pair of the first terminal and the second terminal is established by determining whether or not the trajectory information of the first terminal and the second terminal matches. Pair determination step of determining whether (e.g., as implemented by the trajectory comparing means 101), characterized in that it comprises a.

(4) In the camera-based pairing method, the trajectory information of the first terminal and the trajectory information of the second terminal are respectively configured in a form in which pairs of storage times and points on the XY plane are connected in series. In the pair determination step, for a series of points, the distance between the first terminal point and the second terminal point having a common storage time (for example, the distance between the trajectories) is calculated, and the total number of sets If the calculated distance is below the default value, it is determined that a pair of the first terminal and the second terminal is established. It may be configured to determine that is not established.

(5) In the camera-based pairing method, in the coordinate system rotation step, in the coordinate system of the XY plane between the first terminal and the second terminal, the inclination of the first terminal passes through the origin of the XY plane. For a straight line expressed by the initial inclination value of the line and a straight line passing through the origin of the XY plane and whose inclination is expressed by the initial inclination value of the second terminal, one of the straight lines is point-symmetric about the origin Even if it is configured to calculate the angle required to match the other straight line by rotating so that one coordinate system is point-symmetric about the origin Good.

(6) In the camera-based pairing method, the trajectory information of the first terminal and the trajectory information of the second terminal are each configured in a form in which pairs of storage times and points on the XY plane are connected in series. In the time difference determination step, the number of pairs constituting the trajectory information of the first terminal and the pair constituting the trajectory information of the second terminal is counted, and (the larger number) / (the smaller number) ), The value after the decimal point is rounded down, this value is used as the sampling rate ratio, and when calculating the distance between the first terminal point and the second terminal point having a common storage time, From the terminal of the second terminal and the second terminal, the point of the locus information of the terminal having the larger number is extracted by extracting one point of the locus information of the terminal having the smaller number of pairs constituting the locus information in order of the oldest storage time. Are taken out by the number indicated by the sampling rate ratio from the oldest storage time. 1 and point extracted from the lesser, the process of obtaining the distance between the points extracted from the side number is large, the retention time may be configured to repeat the oldest.

(7) In the camera-based pairing method, in the time difference determination step, only the portion where the respective storage times overlap is extracted from the trajectory information of the first terminal and the trajectory information of the second terminal, In the pair determination step, it is determined whether or not the trajectory information between the first terminal and the second terminal matches based on the extracted trajectory information of the first terminal and the trajectory information of the second terminal. It may be configured.

  According to the present invention, since a call terminal and an information processing terminal at hand can be paired ad hoc, it is possible to provide a telephone service only for voice calls for SI carriers, communication carriers, and ISP carriers in a corporate network. Value-added services can be created.

It is a block diagram which shows an example of the network structure in embodiment of this invention. It is a functional block diagram which shows the function structure of the camera utilization pairing method. It is explanatory drawing which shows an example of the data structure in embodiment of this invention. It is a flowchart which shows an example of a process of a camera utilization pairing method. It is FIG. (1) for demonstrating the reason why rotation of a coordinate system is required. FIG. 6 is a diagram (part 2) for explaining the reason why the rotation of the coordinate system is necessary. It is explanatory drawing which represented typically the comparison on the time series of the locus | trajectory of a telephone call terminal and an information processing terminal. It is explanatory drawing which shows the example of a screen, and the example of operation. It is explanatory drawing which shows the method of pairing of the information processing terminal provided with input means other than a mouse | mouth. It is a block diagram which shows the example of the minimum structure of the camera utilization pairing method.

Explanation of symbols

10 Call terminal A
11 Call terminal B
20 Information processing terminal A
21 Information processing terminal B
DESCRIPTION OF SYMBOLS 100 Pair management apparatus 101 Trajectory comparison means 102 Pair storage means 103 Trajectory storage means 200 Pair 201 Calling terminal identifier 202 Information processing terminal identifier 210, 220, 230 Trajectory 211, 221, 231 Time 212 Calling terminal X coordinate 213 Calling terminal Y coordinate 214 Information processing terminal X-coordinate 215 Information processing terminal Y-coordinate 222,232 X-coordinate 223,233 Y-coordinate 224,234 Initial inclination A111, B111 Imaging means A112, B112 Moving image decomposition means A113, B113 Still image difference detection means A114, A124, B114, B124 Trajectory calculation means A115, A125, B115, B125 Vibration monitoring means A116, A126, B116, B126 Time correction means A117, A127, B117, B127 Trajectory storage means A121, B 121 Input means

Claims (14)

A first terminal equipped with a camera;
A second terminal,
For the first terminal, the first terminal calculates trajectory information indicating a trajectory of movement of the first terminal based on an image captured by the camera when the first terminal is moved by a user. Including a trajectory calculating means,
The second terminal is trajectory information indicating a trajectory of input coordinates in the second terminal when an input operation from the second terminal is performed by the user in conjunction with the movement of the first terminal. Including a second terminal trajectory calculating means for calculating
The first terminal and the second terminal are paired by comparing the trajectory information calculated by the first terminal trajectory calculating means with the trajectory information calculated by the second terminal trajectory calculating means. A camera-based pairing system, further comprising pairing determination means for determining whether or not to perform. The first terminal trajectory calculating means calculates trajectory information including the movement position at which the first terminal has moved and the time at which the first terminal has moved to the movement position in association with each other,
The second terminal trajectory calculating means calculates trajectory information that includes the movement position at which the second terminal has moved and the time at which the second terminal has moved to the movement position in association with each other,
The pairing determination means determines a difference between the movement position included in the locus information calculated by the first terminal locus calculation means and the movement position included in the locus information calculated by the second terminal locus calculation means at the same time. The camera-based pairing system according to claim 1, wherein when it is determined that the distance is within a predetermined distance, it is determined that the first terminal and the second terminal are paired. A first terminal;
A second terminal;
A pair management device that manages pairing between the first terminal and the second terminal;
The pair management device
A trajectory comparison means for determining whether or not to pair the first terminal and the second terminal by comparing trajectory information received from the first terminal and the second terminal;
Pair storage means for storing pair information;
Trajectory storage means for storing trajectory information between the first terminal and the second terminal,
The first terminal is
Photographing means which is a camera capable of photographing moving images;
Moving image decomposition means for extracting a series of still images from the moving image;
A still image difference detection means for calculating a physical movement direction and a movement distance of the photographing means from the series of still images;
Trajectory calculating means for calculating trajectory information of physical movement of the photographing means on the XY plane with the horizontal direction of the still image as the X axis and the vertical direction as the Y axis;
Vibration monitoring means for monitoring the trajectory information and detecting a vibration operation start and vibration operation end for the photographing means by a user;
The time difference between the clock mounted on the pair management device and the clock mounted on the first terminal is calculated, and the storage time of the trajectory information is set to the time on the clock mounted on the pair management device based on the calculated time difference. Time correction means for conversion;
Trajectory storage means for the first terminal that stores trajectory information of the first terminal and an initial inclination indicating in which direction on the XY plane the photographing means starts when the user starts a vibration operation on the photographing means. Including
The second terminal is
Input means which is a two-dimensional coordinate input device;
A trajectory for a second terminal that calculates trajectory information of the movement of a two-dimensional coordinate input by an input unit on an XY plane with the horizontal direction of the screen of the second terminal as the X axis and the vertical direction as the Y axis. A calculation means;
Vibration monitoring means for a second terminal for monitoring the trajectory information and detecting a vibration operation start and vibration operation end for the input means by a user;
The time difference between the clock mounted on the pair management device and the clock mounted on the second terminal is calculated, and the storage time of the trajectory information is set to the time on the clock mounted on the pair management device based on the calculated time difference. Second terminal time correction means for conversion;
A second terminal that stores trajectory information of the second terminal and an initial inclination indicating in which direction on the XY plane the two-dimensional coordinates input when the user starts a vibration operation on the input means. A camera-use pairing system comprising: a trajectory storage means. A first terminal trajectory calculating step of calculating trajectory information indicating a trajectory of the movement of the first terminal based on an image captured by the camera when the first terminal equipped with the camera is moved by the user; ,
A trajectory for a second terminal that calculates trajectory information indicating a trajectory of input coordinates in the second terminal when a user performs an input operation from the second terminal in conjunction with the movement of the first terminal. A calculation step;
Pairing the first terminal and the second terminal by comparing the trajectory information calculated in the first terminal trajectory calculation step with the trajectory information calculated in the second terminal trajectory calculation step. A pairing determination step for determining whether or not to perform the camera-based pairing method. In the first terminal trajectory calculation step, trajectory information including the movement position at which the first terminal has moved and the time at which the first terminal has moved to the movement position in association with each other is calculated,
In the second terminal trajectory calculation step, trajectory information including the movement position at which the second terminal has moved and the time at which the second terminal has moved to the movement position in association with each other is calculated,
Pairing determination step, the difference between the movement position included in the path information calculated by the second terminal for locus calculation step with the movement position included in the first calculated trajectory information terminal for locus calculation step at the same time The camera-based pairing method according to claim 4, wherein when it is determined that the distance is within a predetermined distance, the first terminal and the second terminal are determined to be paired. A pairing execution screen is displayed on the first terminal and the second terminal to hold together the input means of the first terminal and the second terminal, and the first terminal and the input means Based on the operation of vibrating the first terminal, when the first terminal equipped with the camera is moved by the user, the trajectory information indicating the trajectory that the first terminal has moved is calculated based on the image captured by the camera. And a camera-based pairing method for performing pairing between the first terminal and the second terminal,
Detecting the vibration operation start of the first terminal and the input means, to calculate the initial slope at said first terminal and said second terminal, and vibration monitoring initiating storage of the trace information ,
When the operation to vibrate the first terminal and the input unit is completed, the end of the vibration operation between the first terminal and the input unit is detected, and the stored X and Y coordinates of the trajectory information The time difference between the clock mounted on the first terminal and the clock mounted on the pair management apparatus is calculated, and the storage time of the trajectory information is calculated based on the calculated time difference on the clock mounted on the pair management apparatus. A transmission step for converting to time and transmitting the initial inclination and trajectory information to the pair management device;
Based on the received trajectory information, a time difference determination step for determining whether both the difference in vibration start time and the difference in vibration end time are within a predetermined value;
A coordinate system rotation step of rotating the coordinate system of the first terminal using the initial inclination;
It is determined whether or not the pair of the first terminal and the second terminal is established by determining whether or not the trajectory information of the first terminal and the second terminal matches. A pair determination method using a camera. The trajectory information of the first terminal and the trajectory information of the second terminal are respectively configured in a form in which a set of a storage time and a point on the XY plane is continuous.
In the pair determination step, the distance between the point of the first terminal having a common storage time and the point of the second terminal is calculated for a series of points, and the distance is within a predetermined value with respect to the total number of sets If the calculated ratio is lower than the predetermined value, it is determined that the pair of the first terminal and the second terminal is established, and if it is higher, the pair is not established. The camera-use pairing method according to claim 6. In the coordinate system rotation step, in the coordinate system of the XY plane between the first terminal and the second terminal, a straight line that passes through the origin of the XY plane and is represented by the initial tilt value of the first terminal. And a straight line that passes through the origin of the XY plane and whose slope is represented by the initial slope value of the second terminal, the other straight line is rotated so that it is point-symmetric about the origin. 8. The camera-based pairing method according to claim 6, wherein an angle required to match the straight line is calculated, and one coordinate system is rotated by the angle so as to be point-symmetric about the origin. The trajectory information of the first terminal and the trajectory information of the second terminal are respectively configured in a form in which a set of a storage time and a point on the XY plane is continuous.
In the time difference determination step, the number of pairs constituting the trajectory information of the first terminal and the pair constituting the trajectory information of the second terminal is counted, and (the larger number) / (the smaller number) When calculating the distance between the first terminal point and the second terminal point having a common storage time, the fractional part is rounded down and this value is used as the sampling rate ratio. Among the first terminal and the second terminal, one point of the locus information of the terminal having the smaller number of sets constituting the locus information is extracted in order from the oldest storage time, and the terminal having the larger number The number of points of the trajectory information is extracted by the number indicated by the value of the sampling rate in order from the oldest storage time, and the distance between the point extracted from the smaller number and the point extracted from the larger number is obtained. The process is repeated in order of the oldest storage time. Camera Use pairing method according to any one of claims 8. In the time difference determination step, only the portion where the respective storage times overlap is extracted from the trajectory information of the first terminal and the trajectory information of the second terminal,
Whether or not the trajectory information of the first terminal and the second terminal matches based on the trajectory information of the first terminal and the trajectory information of the second terminal extracted in the pair determination step. The camera pairing method according to any one of claims 6 to 9, wherein the determination is performed. A pairing device that performs pairing between a first terminal equipped with a camera and a second terminal,
Trajectory information obtained by calculating a trajectory of movement of the first terminal based on an image captured by the camera when the first terminal is moved by the user, and the second terminal of the first terminal When the user performs an input operation from the second terminal in conjunction with the movement of the first terminal, the first terminal is compared by comparing with the locus information obtained by calculating the locus of the input coordinates in the second terminal. And a pairing determination means for determining whether or not to pair the second terminal with the second terminal. In the pairing determination means, the difference between the movement position included in the trajectory information calculated by the first terminal and the movement position included in the trajectory information calculated by the second terminal at the same time is within a predetermined distance. The pairing device according to claim 11, wherein it is determined that the first terminal and the second terminal are paired when it is determined. A pairing program for pairing a first terminal equipped with a camera and a second terminal,
On the computer,
Trajectory information obtained by calculating a trajectory of movement of the first terminal based on an image captured by the camera when the first terminal is moved by the user, and the second terminal of the first terminal When the user performs an input operation from the second terminal in conjunction with the movement of the first terminal, the first terminal is compared by comparing with the locus information obtained by calculating the locus of the input coordinates in the second terminal. A pairing program for executing a pairing determination process for determining whether or not to pair with the second terminal. On the computer,
In the pairing determination process, the difference between the movement position included in the trajectory information calculated by the first terminal at the same time and the movement position included in the trajectory information calculated by the second terminal is within a predetermined distance. and when it is determined, the first terminal and the second terminal and the claim 1 3 program pairing according to execute the process of determining the pairing.

Images