JP6188070B2 - Pairing system, authentication server and terminal - Google Patents

Description

  The present invention relates to a pairing system between terminals, an authentication server, and a terminal that do not require a special module in the terminal and do not increase a user's burden.

  Examples of a general home / television medical support system include those described in Patent Document 1 (module terminal for home medical service) and Non-Patent Document 1 (Niimi Medical Association Remote Home Medical Support System). In such systems, videophones are made using fixed terminals such as PCs and set-top boxes to complement face-to-face medical treatment at remote locations.

Special table 2004-508914 gazette

http://niimi-ma.no-ip.com/~ishikai/enkaku/d_01.html

  In the existing system, when making a videophone call, when registering one of the terminals, a telephone number or the like must be entered, which is a burden on the user.

  Furthermore, when performing telemedicine, as described in Japanese Patent Application No. 2012-195056 by the present inventor, a smartphone or the like may be used as a device for photographing the affected area. Pairing becomes a problem.

  FIG. 1 is a diagram for explaining the use of a terminal such as a smartphone in addition to a videophone terminal and the necessity of pairing at that time in the remote medical care. That is, after constructing a normal videophone system by the videophone terminal 101 on the doctor side in (1), the relay server 102, and the videophone terminal 103 on the home patient side in (2), (2) The terminal 104 having a camera such as a smartphone is introduced on the home patient side.

  In this way, on the home patient side, in addition to the videophone terminal 103 that shoots in a fixed arrangement, the terminal 104 can be used as a small-handed camera that can be freely moved by holding it with the hand, and so on. Do. On the doctor's side, the videophone terminal 101 displays the affected area taken by the terminal 104 in a superimposed manner on the videophone screen, or by displaying it separately from the videophone screen. Increase.

  In order to enable such a service, the videophone terminal 103 and the terminal 104 need to be paired in advance. That is, it is necessary to incorporate the terminal 104 into a videophone system constructed in advance by the doctor's video terminal 101, the relay server 102, and the home patient's video terminal 103 by the pairing.

  When pairing, there is a method of completing pairing using near field communication such as NFC (Near Field Communication), but in recent years there are also terminals that are provided at low cost by eliminating devices such as NFC as much as possible. Since it is increasing, it cannot be applied to such terminals.

  In view of the above-described problems of the prior art, the present invention does not use a special module that leads to an increase in terminal costs such as NFC and does not increase the user burden, for example, a videophone in a telemedicine system It aims at providing the pairing system which implement | achieves pairing of terminals like the case of a terminal and a smart phone.

  Another object of the present invention is to provide an authentication server in the pairing system.

  Furthermore, an object of this invention is to provide the terminal in the said pairing system.

  In order to achieve the above object, the present invention includes an imaging unit that captures an image and obtains a captured image, and a display unit that displays the captured image as a display image. A display image is received from each of the two terminals, the first terminal and the second terminal, and the two terminals, and the two terminals are paired based on the comparison of the two received display images. An authentication server for determining whether or not the authentication server compares the received two display images so that the two display images are displayed on the first terminal. When the imaging unit images the display unit of the second terminal and simultaneously determines that the imaging unit of the second terminal is obtained by imaging the display unit of the first terminal, the pair Ring target Characterized in that to make the cross-sectional.

  In addition, the present invention provides a first terminal and a second terminal, which each include an imaging unit that captures an image and obtains a captured image, and a display unit that displays the captured image as a display image. An authentication server that receives a display image from each of two terminals including two terminals and determines whether the two terminals are to be paired based on a comparison of the two received display images. By comparing the two received display images, the two display images are captured by the imaging unit of the first terminal and the imaging unit of the second terminal at the same time. Is determined to be obtained by imaging the display unit of the first terminal, it is determined to be the pairing target.

  Furthermore, the present invention provides each of the terminals determined by the authentication server as a pairing target, an imaging unit that captures an image and obtains the captured image, and a display unit that displays the captured image as a display image. The terminal transmits the display image to the authentication server, and the authentication server receives and compares the display image from each of the two terminals including the first terminal and the second terminal, The two display images are obtained by the imaging unit of the first terminal imaging the display unit of the second terminal and simultaneously the imaging unit of the second terminal imaging the display unit of the first terminal. If it is determined that the two terminals are to be paired, the determination is made.

  According to the present invention, two display images are obtained when the imaging unit of the first terminal images the display unit of the second terminal, and at the same time, the imaging unit of the second terminal images the display unit of the first terminal. When it is determined that it has been received, the pairing is determined. Accordingly, since pairing is performed when it is determined that the two terminals are in the arrangement of the mirrors, appropriate pairing can be realized. Furthermore, since only the image is used for the determination, pairing can be realized without increasing the burden on the user and without requiring a special module in the terminal.

It is a figure for demonstrating use of terminals, such as a smart phone, in addition to a video telephone terminal in telemedicine, and the necessity of the pairing in that case. It is a functional block diagram of the pairing system which concerns on one Embodiment. It is a flowchart which shows the procedure of this invention which concerns on one Embodiment. It is a figure which shows the example which illustrates the state of mirror alignment typically. It is auxiliary explanatory drawing of FIG. It is a figure for demonstrating the marker to superimpose and its series of images, and is a figure which shows the example of arrangement | positioning of the mirror of the terminals in the said description, and the superimposed display of a marker. It is a figure for demonstrating the marker to superimpose and its series of images, and is a figure which shows typically the display image on which the marker of FIG. 6 was superimposed. It is a figure for demonstrating the marker to superimpose and its series of images, and is a figure which shows typically the example of a series of images produced | generated with respect to a marker in the display image of FIG. It is a figure which shows the marker of a "framework" as an example which shows that various aspects are possible for a marker.

  FIG. 2 is a functional block diagram of the pairing system according to the embodiment. The pairing system 1 includes a terminal 10A and a terminal 10B that are to be paired with each other, and an authentication server 20 that performs authentication at the time of pairing. The terminal 10A includes a display unit 11A, an imaging unit 12A, and a communication unit 13A. The terminal 10B includes a display unit 11B, an imaging unit 12B, and a communication unit 13B as the same configuration as the terminal 10A. The authentication server 20 includes a communication unit 21, an authentication unit 22, and an estimation unit 23. The outline of each part will be described below.

  As the terminal 10A and the terminal 10B, a mobile terminal, a smart phone, the videophone terminal 103 at the patient's house in FIG. When introduced into the system of FIG. 1, the authentication server 20 may be realized by a relay server 102 that relays a videophone. The terminal 10A and the terminal 10B copy each other at the time of pairing, and transmit the captured images and the like to the authentication server 20 to perform symmetric and common processing. This will be described on the terminal 10A side.

  The display unit 11A displays the captured image captured by the imaging unit 12A as a display image. At this time, the captured image may be directly displayed as a display image. Preferably, a superimposed image in which a predetermined marker is superimposed on the captured image is displayed as a display image. The imaging unit 12A captures the display unit 11B on the other side, obtains the captured image, and passes it to the display unit 11A.

  The communication unit 13A transmits the display image displayed on the display unit 11A to the communication unit 21 of the authentication server 20. When a predetermined marker is superimposed on the display image, information on the predetermined marker is also transmitted from the communication unit 13A to the communication unit 21 together with the display image.

  Further, when the authentication unit 20 succeeds in pairing authentication, the communication unit 13A receives address information (such as the IP address and mail address of the partner terminal 10B) from the communication unit 21 for pairing. Then, pairing is performed based on the pairing information, and communication is performed with the communication unit 13B on the other side.

  The terminal 10A side has been described above, but the terminal 10B side performs exactly the same processing by exchanging “A” and “B” at the end of the code used in the description. Note that the terminal 10A and the terminal 10B perform the same processing, but they are separate terminals, and therefore the performance does not have to be the same. For example, the display unit 11A and the display unit 11B may have different specifications as a display, and the imaging unit 12A and the imaging unit 12B may have different specifications as a camera.

  In the authentication server 20, the display unit transmitted from each of the terminal 10A and the terminal 10B is received by the communication unit 21, and when the authentication unit 22 can compare both the display images and determine that they are similar to each other, Terminal 10A and terminal 10B are determined to be pairing targets. When it is determined that it is a pairing target, the communication unit 21 transmits address information necessary for pairing to each terminal. That is, the address information of the terminal 10B is transmitted to the terminal 10A, and the address information of the terminal 10A is transmitted to the terminal 10B.

  Note that the content of the determination by the authentication unit 22 is specifically a determination as to whether a process of copying between the terminal 10A and the terminal 10B has been performed, and the determination is similar to both display images based on a predetermined criterion. It is made based on whether or not it can be determined.

  Here, in the present invention, in particular, when the terminal 10A and the terminal 10B are imprinted with each other, a technique is adopted in which they are imprinted in a “matching mirror” situation as described in detail later. When the method is appropriately implemented and succeeds in creating a situation where the images are mutually reflected, a common display image can be obtained by capturing and displaying a common image. On the other hand, if the method is not implemented or improperly implemented and fails to create a situation in which each other is copied, they are not copied from each other. Different display images will be obtained (unless they occur). Therefore, it is possible to determine whether or not the two display images are similar to each other depending on whether or not the two display images are similar.

  In addition, the terminals that can be photographed with each other in the situation of “matching mirror” are limited to terminals that are physically approached by taking a predetermined arrangement facing each other by the user's hand. Images similar to each other can be obtained only between terminals. Accordingly, the authentication server 20 side can make a pairing determination based on the similarity of images.

  Further, in the authentication server 20, when the pairing is determined by the authentication unit 22, the estimation unit 23 superimposes the two display images compared by the authentication unit 22 when displayed on the display units 11A and 11B. The pixel change based on various performances of the display units 11A and 11B and the imaging units 12A and 12B is estimated by analyzing the marker and a series of images of the marker generated by the arrangement of the “matching mirror”. The performance includes the resolution, brightness, screen size, white balance, camera performance, etc. of both terminals 11A and 11B, and how the pixel changes (deteriorates) based on these performances. It is estimated by comparison.

  FIG. 3 is a flowchart showing a procedure of the present invention according to an embodiment. With reference to FIG. 3, the detail of the process of each part of FIG. 2 is demonstrated.

  In step S1, the user places the terminal 10A and the terminal 10B facing each other so that they are placed in a state of a mirror that reflects each other. FIG. 4 is a diagram illustrating an example for schematically explaining the state of the mirror. In FIG. 4, casings 15A and 15B each having a rectangular parallelepiped shape are shown as examples of the casings of the terminals 10A and 10B, and the casings 15A and 15B face each other with their surfaces 16A and 16B parallel to each other. They are arranged so as to stand upright on the plane 100. The surfaces 16A and 16B may be substantially parallel to each other.

  On the surface 16A of the casing 15A of the terminal 10A, the display surface 17A of the display unit 11A and the imaging surface 18A of the imaging unit 12A are arranged. FIG. 5 is an auxiliary explanatory diagram of FIG. 4 and shows that the display surface 17A and the imaging surface 18A are arranged on the surface 16A, as viewed from the front. In the same manner for the terminal 10B, the display surface 17B of the display unit 11B and the imaging surface 18B of the imaging unit 12B are arranged on the surface 16B of the housing 15B. Although FIG. 5 shows the terminal 10A, the arrangement of the terminal 10B is the same. In FIG. 5, “A” at the end of the code is replaced with “B”, and the shape of the face 16B is viewed from the front. Yes.

  With this arrangement, as indicated by the arrow (1) in FIG. 4, the imaging unit 12A of the terminal 10A images the display surface 17B of the display unit 11B of the terminal 10B with the imaging surface 18A, and the captured image Is displayed on the display surface 17A of the terminal 10A. Similarly, as indicated by the arrow (2), the imaging unit 12B of the terminal 10B images the display surface 17A of the display unit 11A of the terminal 10A by the imaging surface 18B, and the captured image is displayed on the display surface of the terminal 10B. Displayed on 17B.

  Therefore, as shown by the bidirectional arrow in (3), in the terminal 10A, the imaging unit 12A displays the image captured by the imaging surface 18A on the display surface 11A of the display unit 11A, and in the terminal 10B, the imaging unit 12B Both the display surface 17B of the display unit 11B that displays the image captured on the imaging surface 18B behave as if they form a “matching mirror” and display is performed.

  Note that the arrangement of the “mating mirror” arranged by the user in step S1 is not limited to the example as shown in FIG. 4, and may generally be an arrangement satisfying the following relationship. That is, the terminal 10A including the display unit 11A that displays the captured image of the imaging unit 12A (or the superimposed image in which the marker is superimposed on the captured image) and the captured image of the imaging unit 12B (or the superimposed image in which the marker is superimposed on the captured image) As shown also as an arrow between the functional blocks in FIG. 2, the imaging unit 12A captures the display unit 11B and simultaneously the imaging unit 12B. The relationship is such that the display unit 11A is imaged.

  Further, as shown in FIG. 5, the terminals 10A and 10B have a configuration in which the imaging units 12A and 12B (imaging surfaces thereof) are on the same plane as the display units 11A and 11B (display surfaces thereof) (such as portable terminals) In general, since the display unit is inside the terminal, it is preferably manufactured from an in-camera (internal camera) from the following viewpoint. That is, if it is manufactured with the configuration of the in-camera, the user can easily make the mirror arrangement by simply facing both terminals as shown in FIG.

  Depending on the mode of manufacture of the terminal 10A and / or the terminal 10B, for example, both of them do not have an in-camera and only have an out-camera (rear camera) installed on the opposite side of the display unit. I can't make an arrangement. In such a case, one or more actual “mirrors” (actual mirrors) may be used to make an arrangement of “mating mirrors”.

  Returning to FIG. 3, in step S2, the terminal 10A and the terminal 10B simultaneously capture images of each other in the arrangement of the mirrors arranged by the user in step S1. That is, the imaging unit 12A and the imaging unit 12B capture images at the same time to obtain captured images. In conjunction with this, the display unit 11A and the display unit 11B obtain display images.

  Here, when performing imaging (and display) at the same time, it is only necessary to perform imaging at the same time with a certain accuracy. For example, the image capturing unit 12A and the image capturing unit 12B may perform image capturing simultaneously by a user's manual operation. Alternatively, the clocks of the terminal 10A and the terminal 10B are set in advance by the authentication server 20 and the like, and the imaging unit 12A and the imaging unit 12B simultaneously perform imaging at a predetermined time instructed from the authentication server 20 and the like. May be. Note that the clock adjustment and the instruction of the predetermined time may be individually performed between the communication units 13A and 21 and between the communication units 13B and 21.

  The captured image obtained in step S2 is obtained in detail through the following process. That is, the display unit 11A and the display unit 11B perform display as a real-time superimposed image in a state where a predetermined marker is superimposed on a real-time captured image captured by the imaging unit 12A and the imaging unit 12B, respectively. That is, at each time t, a superimposed image at the same time t is immediately generated and displayed from the captured image at the time t. Then, in step S2, the imaging unit 12A and the imaging unit 12B obtain captured images, assuming that the state in which the superimposed image is displayed is captured.

  Here, in the display units 11A and 11B, as a process additionally performed by the display unit 11A and 11B, only a predetermined marker is superimposed once on the captured images of the imaging units 12A and 12B. On the other hand, since the alignment mirrors are arranged in step S1, a series of images of the markers superimposed once are generated and obtained in the captured images of the imaging units 12A and 12B. The marker and its series of images initially superimposed once on the captured image are used in steps S4 and S5 described later.

  6-8 is a figure for demonstrating the said marker to superimpose and the series of images produced | generated. FIG. 6 is an example in which the terminals 10A and 10B are arranged with a pair of mirrors. The parts denoted by the same reference numerals are the same as in the example of FIG. 4, but in FIG. 6, the terminal 10B has the arrangement of FIG. In contrast, it is “inverted” on the plane 100. That is, in both FIG. 4 and FIG. 6, the terminals 10A and 10B are of the same type in terms of the arrangement of the imaging unit and the display unit, whereas both terminals stand in the same direction in FIG. In FIG. 6, both terminals stand in an inverted relationship.

  The reason why the terminals 10A and 10B are arranged on the mirror in the relationship of “inverted” in FIG. 6 is that the series of images is more reliably generated because of the relationship with the markers described in FIGS. This is an example of a preferable arrangement in that such a positional relationship is generated. Other arrangements may be used as long as markers as a series of images are generated without overlapping each other.

  In the arranged FIG. 6, in the terminal 10A, the display unit 11A displays the marker MA0 superimposed on the captured image and is displayed on the display surface 17A. Similarly, on the terminal 10B, the display unit 11B displays the marker MB0 superimposed on the captured image and is displayed on the display surface 17B.

  FIG. 7 shows an image PA displayed on the display unit 11A by the terminal 10A and an image PB displayed on the display unit 11B by the terminal 10A in the form of viewing the superimposed images in the example of FIG. FIG. In the image PA, a captured image of the imaging unit 12A is displayed in a region other than the superimposed rectangular marker MA0, that is, a white region in the image PA. (In FIG. 7, a specific example of the captured image is not shown and is shown as a white region.)

  Therefore, in the portion of the marker MA0 in the captured image in FIG. 7, the image PA is displayed in such a manner that the overwritten portion is hidden by the marker MA0 overwriting the captured image. Similarly, in the image PB, the captured image of the imaging unit 12B is displayed in a region other than the superimposed rectangular marker MB0, and the portion of the captured image that overlaps the marker MB0 is hidden.

  For the markers MA0 and MB0 to be superimposed, the contents (shape, pattern, etc.) of the images and the positions to be superimposed in the images PA and PB are set in advance as predetermined in each terminal. Several kinds of predetermined markers are prepared, and which one is used may be determined randomly at each terminal each time pairing according to the present invention is performed.

  FIG. 8 is a diagram schematically showing a series of images for the markers MA0 and MB0 that are first superimposed only once by the display units 11A and 11B in the example of FIG.

  That is, as indicated by the arrow (A1), the image MA 12 in the image PA of the display unit 11A is captured by the image capturing unit 12B on the other side, so that the first image MA1 is in the image PB of the display unit 11B. Generated. Subsequently, as indicated by an arrow (A2), the image MA1 in the image PB of the display unit 11B is further captured by the image capturing unit 12A on the other side, so that the second image MA2 is the image of the display unit 11A. Generated in the PA. Further, as indicated by an arrow (A3), an image MA3 of the image MA2 is generated in the image PB as the third image. Similarly, as long as the specifications of the imaging unit and display unit of both terminals 10A and 10B allow, the (n + 1) th image relative to the nth image is recursively generated alternately between the images PA and PB. It becomes.

  Similarly, as shown by the arrow (B1), the image capturing unit 12A on the other side captures the marker MB0 in the image PB of the display unit 11B, and the first image MB1 is generated in the image PA, and the arrow (B2) As shown in the figure, the image MB1 is captured by the counterpart imaging unit 12B, the second image MB2 is generated in the image PB, and the third image MB3 is included in the image PA as indicated by the arrow (B3). The image is recursively generated as long as the spec allows.

  As described above, when a series of images of the first superimposed markers MA0 and MB0 is generated, the number of image generation iterations increases from the nth image to the n + 1th image. Will deteriorate. That is, the blur increases, the size decreases, and the color tone changes. The deterioration depends on the distance and orientation of the terminals 10A and 10B when they are placed on the mirror and external conditions such as the light source, but basically the specifications of the display units 11A and 11B and the imaging units 12A and 12B. Depending on the pixel, the degree of deterioration is determined and causes a pixel change. In step S5 described later, pixel changes in both terminals 10A and 10B determined based on the spec dependence are estimated.

  Further, as in the schematic example of FIG. 8, as an example of the mirror alignment arrangement of the terminals 10A and 10B so that a series of images appear in order, the arrangement of FIG. In particular, the display surface 17A, the marker MA0, and the imaging surface 18A, and the display surface 17B, the marker MB0, and the imaging surface 18B are point-symmetric with each other after the surfaces 16A and 16B are parallel to each other. be able to. When arranging the points symmetrically, the markers MA0 and the markers MB are not the entirety of them, and only predetermined points need to constitute a point-symmetrical arrangement.

  Further, when the terminals 10A and 10B are manufactured as different types and there is a difference in size or the like between the display surfaces 17A and 17B and / or between the imaging surfaces 18A and 18B, it follows the point symmetry. As the arrangement, the predetermined points (centers of gravity, etc.) may be arranged symmetrically. The point-symmetric arrangement may be any arrangement that can be determined to be point-symmetric with a predetermined accuracy.

  In general, it is not necessarily arranged point-symmetrically. However, from the viewpoint of obtaining abundant information used for estimation in step S5, a series of images (at least the first predetermined number of them) are images PA, PB. Needs to be generated inside. For this reason, at the time of arrangement, the position where the series of images is generated becomes a position overlapping the markers MA0 and MB0 that are superimposed first (the captured image is overwritten and disappears as described above). It is necessary to avoid deviating from the range of the images PA and PB.

  Returning to FIG. 3, in step S3, the communication unit 13A and the communication unit 13B transmit the display images acquired in step S2 (the display image PA of the terminal 10A and the display image PB of the terminal 10B) to the communication unit 21, respectively. Thus, each display image is uploaded to the authentication server 20.

  In step S4, the authentication unit 22 of the authentication server 20 compares both images PA and PB acquired in step S2. If it is determined by the comparison that the terminals 10A and 10B are similar to each other based on a predetermined criterion, the process proceeds to step S5 and is determined not to be similar. If it is determined that it is not a pairing target, pairing is not performed, and the flow is terminated.

  Note that if it is determined that it is not similar and is not a pairing target, a message indicating an error is sent to both terminals 10A and 10B, and then a step is performed to prompt the user to acquire an image again. You may make it return to S1. Thus, the terminals 10A and 10B are actually close to each other, and the mirror alignment is improperly set even though the user faces the mirror alignment in step S1. When the images in both the images PA and PB are inappropriate, it is possible to prompt the user to perform the work again.

  As a first embodiment, the method for calculating the similarity between various images is used for the comparison method of the images PA and PB in step S4, and threshold determination is performed on the calculated similarity using a predetermined threshold. Just do it. For example, similarity may be calculated by sum of absolute differences (SAD) between pixels, or a known feature amount such as SIFT is calculated, and the series of feature points are associated with each other by a known method. The degree of similarity may be calculated using

  Further, in the similarity determination by comparing both images PA and PB, as a second embodiment, not only the similarity between images and the threshold determination for the similarity as in the first embodiment, but also an additional condition May be imposed. As an additional condition, it can be used that both images are determined to have a predetermined characteristic based on being a mirror-aligned arrangement and / or overlapping a predetermined marker. .

  In the example of the images PA and PB in FIG. 8, an image MA1 corresponding to the image of the marker MA0 superimposed on the terminal 10A is detected in the image PB of the terminal 10B, and the terminal When an image MB1 corresponding to the image of the marker MB0 superimposed on 10B is detected, it may be determined that the image has predetermined characteristics.

  When detecting the image MA1 and the image MB1, information on the markers MA0 and MB0 to be superimposed (information as the marker image) is combined in advance when the images PA and PB are received from the terminals 10A and 10B in step S3. Therefore, the regions similar to the marker images MA0 and MB0 may be detected from the images PB and PA, respectively. For the detection of the similar region, it may be determined that the detection is performed when the threshold determination is satisfied by using the above SAD, SIFT or the like.

  Similarly, in addition to the first images MA1 and MB1, the second and subsequent k-th images MA2, MB2,... MAk, MBk are detected from each image. May be imposed as

  Further, in the case of additionally detecting an image as described above, it may be further imposed on whether or not the positional relationship between the images in relation to the marker to be superimposed first matches the situation of the mirror. Good. For example, it may be determined whether or not it is possible to determine that the arrangement relationship between the superimposed marker MA0 and the image MB1 in the image PA and the arrangement relationship between the image MA1 and the superimposed marker MB0 in the image PB match.

  Returning to FIG. 3, in step S5, the estimation unit 23 further advances the comparison of both images by the authentication unit 22 and analyzes the second and subsequent images, thereby changing the pixels based on the specifications of the terminals 10A and 10B. presume. Specifically, it is as follows.

  The original image data of the original marker, that is, the marker to be superimposed first, is x (x is a vector in which the pixels of the image data are arranged), and the screens of the terminals 10A and 10B (display units 11A and 11B), camera ( Assuming ad, ac, bd, and bc as matrices representing pixel changes by the imaging units 12A and 12B), displayed images are as follows.

  The original marker generated by the terminal 10A, that is, the pixel of the marker MA0 in the image PA of FIG. 8 is ad · x (formula 1). Further, the second marker generated by the terminal 10A, that is, the pixel of the marker MB1 in the image PA of FIG. 8 is ac · ad · bd · x (formula 2).

  Similarly, the original marker generated by the terminal 10B, that is, the pixel of the marker MB0 in the image PB of FIG. 8 is bd · x (formula 3). Further, the second marker generated by the terminal 10B, that is, the pixel of the marker MA1 in the image PB of FIG. 8 is bc · bd · ad · x (formula 4).

  Since a plurality of equations can be derived by reading the marker as described above, it is possible to estimate a matrix representing these pixel changes. That is, first, the pixel change by the display unit can be estimated by comparing the original marker itself with the original marker displayed on the display unit. From (Equation 1), the pixel change ad caused by the display unit 11A is estimated, and from (Equation 3), the pixel change bd caused by the display unit 11B is estimated.

  Furthermore, the image change by the imaging unit can be estimated by considering the second marker. That is, ac · ad is calculated from (Equation 2) and (Equation 3), and since ad is already estimated from (Equation 1), the pixel change ac by the imaging unit 12A is estimated. Similarly, bc · bd is calculated from (Equation 4) and (Equation 1), and bd has already been estimated from (Equation 3), so the pixel change bc by the imaging unit 12B is estimated.

  When calculating each matrix ad, ac, bd, bc representing pixel change, a matrix that minimizes the error may be calculated by a known numerical calculation method. The closer each matrix is to the unit matrix, the higher the performance of the corresponding display unit or imaging unit. Each matrix may incorporate a predetermined model of pixel change, that is, deterioration. In each model, a unit matrix is formed when the degradation is zero. For example, a matrix may be defined by giving a parameter of a Gaussian function that models blur, and the parameter that minimizes the error may be obtained by matrix calculation to obtain a corresponding matrix.

  In calculating each matrix, it is necessary to calculate in advance which area on the image PA, PB each marker MA0, MA1, MB0, MB1, etc. corresponds to, but this has already been done in step S4. Since it is calculated, the result may be used. When the terminal 10B uses a marker y different from x, (Equation 3) becomes bd · y, and (Equation 2) becomes ac · ad · bd · y. Can be calculated.

  Returning to FIG. 3, in step S6, the authentication server 23 notifies each other of the address information of the other party in order to perform pairing between the terminals 10A and 10B. At the time of the notification, the pixel change estimated in step S5 is also notified. For example, when a videophone call is performed after pairing between the terminals 10A and 10B, the optimum data transfer amount in both terminals is determined. It can also be used as auxiliary information when doing so.

  Hereinafter, supplementary items (1) to (6) of the present invention will be described.

  (1) Various modes are possible for the predetermined marker to be superimposed on both terminals 10A and 10B. For example, instead of the examples of FIGS. 7 and 8 (rectangular markers), a “framework” marker as shown in FIG. 9 may be adopted. In FIG. 9, the same reference numerals as those in FIGS. 7 and 8 are given, but this is given by borrowing the same reference signs in order to show that the relationship between the images to be generated is the same.

  In FIG. 9, as shown in (1), a marker MA0 is superimposed as a “framework” of the outer peripheral portion of the display image PA in the terminal 10A, and similarly, as shown in (2), the display image is displayed in the terminal 10B. Marker MB0 is superimposed as the “framework” of the outer periphery of PB. Thus, as shown in (3) and (4), respectively, a series of images of the marker is obtained as a “framework” that recursively in a nested structure, for example.

  (2) The predetermined markers may not be superimposed on both terminals 10A and 10B. In this case, an image in which the screens of each other are simply mirrored as “mirrors” is obtained, but since they are similar to each other, pairing is possible.

  (3) In both terminals 10A and 10B, a predetermined marker may not be superimposed, and an article serving as a substitute for the marker may be arranged and used. For example, as the “framework” described in FIG. 9, instead of the marker to be superimposed, the “framework” formed as a part of the casing as an actual object around the display portions of both terminals 10A and 10B. May be used. Further, for example, instead of the rectangular marker as described with reference to FIGS. 7 and 8, a rectangular sticker may be attached to the surfaces of the display portions of both terminals 10A and 10B. When the article is substituted, it is preferable to notify the authentication server 20 of information such as color information so that the article can be identified on the image.

  (4) When placing the mirrors in step S1 in FIG. 3, the following may be performed to assist the user in placing them manually. In each terminal 10A, 10B, is it possible to detect from the display images PA, PB of their own display images (MA2, MB2 in the example of FIG. 7 etc.) that correspond to the images of the markers MA0, MB0 superimposed by themselves? Is constantly monitored, and when the detection is not made, the user is notified by the sound alarm or the like that the arrangement is not appropriate.

  Even if the images MA2 and MB2 are detected, if the distance from the original markers MA0 and MB0 is more than a predetermined distance, the direction of the mirror is still determined to be inappropriate and the audio alarm is continued. It may be.

  In addition, if the type of terminal is fixed, prepare equipment such as a stand or holder that can be placed on the mirror by installing both terminals, and use the equipment to arrange the mirror. It may be realized.

  Further, the authentication server 20 side compares the original marker image sent from both terminals 10A and 10B with the marker image in the images sent from both terminals 10A and 10B, and determines the deterioration degree. When the deterioration is within a certain range, the terminal 10A and 10B may be notified by determining that they face each other. Here, the value of the deterioration degree may be acquired as a parameter of the applied deterioration model in ad, ac, bd, and bc, which is a matrix representing the pixel change in the above (Expression 1) to (Expression 4).

  (5) In the present invention, uploading to the authentication server 20 has been described as an image captured and displayed at the same time. However, a moving image displayed simultaneously over a predetermined period is uploaded, and a predetermined ratio of the entire period is uploaded. When the pairing determination described in the image is made, the pairing may be performed based on the moving image. In this case, the superimposed marker may be configured to change over time.

  (6) When each terminal 10A, 10B uploads an image (or a moving image, the same applies hereinafter) to the authentication server 20, it also uploads an image time stamp (image acquisition time information). In addition to the similarity of images, it may be determined as a pairing condition that the time stamps are determined to match with a predetermined accuracy.

  1 ... Pairing system, 10A, 10B ... Terminal, 20 ... Authentication server

Claims (8)

An imaging unit that performs imaging and obtains a captured image, and a display unit that displays the captured image as a display image, each of which includes two first terminal and second terminal that are pairing determination targets. A terminal,
An authentication server that receives display images from each of the two terminals and determines whether or not the two terminals are to be paired based on a comparison of the two received display images. A ring system,
The authentication server compares the received two display images so that the two display images are captured by the imaging unit of the first terminal and the display unit of the second terminal at the same time. A pairing system, characterized in that, when an imaging unit of a terminal determines that the image is obtained by imaging a display unit of the first terminal, a determination is made as the pairing target.   The pairing system according to claim 1, wherein the authentication server performs the determination based on similarity between the two display images. When the display units of the two terminals each display a display image, a predetermined marker is superimposed on the captured image and displayed.
The authentication server, the pairing system according to claim 1 or 2 images of the predetermined markers, respectively based on detected Te odor of the two display images, and performs the determination.   The authentication server further calculates a pixel change of the image with respect to the superimposed predetermined marker in each of the two display images when it is determined to be the pairing target. The pairing system according to claim 3. The authentication server calculates a pixel change caused by the display unit of each terminal by comparing the original image data of the predetermined marker and the data when superimposed on the captured image, and further, the superposition and data definitive to one of the terminals at the time of the the data of the image in the other of the terminals of the mating terminal, by comparison, calculating a pixel change caused by the imaging unit of the terminal The pairing system according to claim 4. An imaging unit that performs imaging and obtains a captured image, and a display unit that displays the captured image as a display image, each of which includes two first terminal and second terminal that are pairing determination targets. An authentication server that receives a display image from each of the terminals and determines whether the two terminals are to be paired based on a comparison of the two received display images,
By comparing the two received display images, the two display images are captured by the imaging unit of the first terminal and the imaging unit of the second terminal at the same time. An authentication server, characterized in that when it is determined that the display is obtained by imaging the display unit of the first terminal, the determination is made as the pairing target. Each of the terminals determined by the authentication server as a pairing target,
An imaging unit that performs imaging to obtain a captured image;
A display unit that displays the captured image as a display image,
The terminal transmits the display image to the authentication server,
The authentication server receives and compares the display images from the two terminals, the first terminal and the second terminal, so that the two display images are displayed on the second terminal by the imaging unit of the first terminal. If the image capturing unit of the second terminal is determined to be obtained by capturing the display unit of the first terminal, the two terminals are determined to be paired. A terminal characterized by that. The display unit displays the display image by superimposing a predetermined marker in said captured image,
The terminal transmits information of the predetermined marker to the authentication server,
The authentication server terminal according to claim 7 in which the image of the predetermined markers, respectively based on detected Te odor of the two display images, and performs the determination.