JP6033431B2 - Information processing method, information processing apparatus, and information processing program - Google Patents

Description

  The present invention relates to a technical field such as a method for detecting a specific portion based on a moving image of an object imaged by an imaging means.

  Conventionally, a method is known in which the position, distance, and moving direction of a person approaching the camera is detected, and information is changed based on the position, distance, and moving direction and presented on a display unit (Patent Document 1). In the method described in Patent Document 1, a distance is estimated from, for example, a skin color portion of a face or the width of an eye from an image captured by a camera, and the position and size of an image to be displayed in an overlapping manner based on the estimated value. Is to change.

JP 2006-330011 A

  By the way, recent portable terminals can perform short-range wireless communication with a card incorporating a non-contact type IC chip. When establishing short-range wireless communication between a portable terminal and a non-contact type IC chip, for example, it is necessary to bring the back surface of the portable terminal and the plane of the card substantially parallel to bring them close to each other. Thus, it is convenient if the portable terminal can execute processing such as data input or output by a change in the relative positional relationship between the portable terminal and the card in a situation where the card is brought close to the portable terminal. On the other hand, many portable terminals have a function of displaying an image captured by a camera disposed on the back side in real time on a display formed on the front side. Therefore, for example, when establishing short-distance wireless communication between the mobile terminal and the card, the plane of the card can be imaged with the camera of the mobile terminal. In order to recognize the relative positional relationship (for example, distance) between the mobile terminal and the card from the image of the card imaged in this way, it is conceivable to use, for example, the technique disclosed in Japanese Patent Application Laid-Open No. 2005-151867. However, in the method described in Patent Document 1, the distance is estimated using a feature amount that is commonly present on the surface of the subject. Therefore, when the feature amount cannot be acquired from the surface of the subject, the method described in Patent Document 1 cannot be used.

  The present invention relates to an information processing apparatus capable of detecting a change in the relative positional relationship between an information processing apparatus such as a portable terminal and an object such as a card without depending on information appearing on the surface of the object. An object is to provide an information processing method, an information processing program, and a recording medium on which the information processing program is recorded.

In order to solve the above-described problem, an invention according to claim 1 is an information processing method that an information processing program causes an information processing apparatus to perform, and the information processing method is a series of images continuously captured by an imaging unit. A determination step of determining a target portion that can be specified with reference to an edge of an object appearing in the reference image in a reference image that constitutes the image, and any one of a plurality of comparison images in which the determined target portion follows the reference image A detection step for detecting an event indicating that the object has changed, and an output step for outputting a signal corresponding to the detected event or another event related to the detected event, wherein the object stores data. And an information processing method comprising: an IC module having a storage unit that performs processing and a processing unit that accesses the data in response to a request from a device capable of communicating with the object Is characterized by performing a predetermined process based a signal output by said output step, the data of the storage unit. The invention according to claim 2 is the information processing method according to claim 1, wherein the object is a card including an IC module having the storage unit and the processing unit, and the storage unit is an electronic value Balance data indicating a balance is stored, and the processing unit executes balance change processing for changing the stored balance data using balance change information input from a device within a short range wireless communication range. The information processing method is configured to set a charge amount indicating an amount to be added to the balance according to the output signal, and the amount corresponding to the charge amount An input step of acquiring balance change information for changing the stored balance data from an external balance change device and inputting the balance change information to the IC module. To. According to the present invention, the balance data stored in the storage unit of the IC module provided in the card can be changed to change the relative positional relationship between the card and the information processing apparatus. According to a third aspect of the present invention, in the information processing method according to the first aspect, the processing unit is configured to perform predetermined processing on the data in response to a request from a device within a short-range wireless communication range. A process is executed and a process result is returned, and the information processing method further includes a setting step of setting a parameter used for the predetermined process in accordance with the signal output by the output step. Features. According to this invention, it is possible to set parameters used for processing on data stored in the storage unit of an object by changing the relative positional relationship between the object and the information processing apparatus. According to a fourth aspect of the present invention, in the information processing method according to any one of the first to third aspects, the determining step includes a reference image that constitutes a series of images that are continuously captured by the imaging unit. In Step 1, the linear first edge of the object appearing in the reference image is determined as the first target portion, and the linear second edge orthogonal to the first edge is determined as the second target portion. The detection step is a detection step of detecting an event indicating that the determined first target portion and second target portion have changed in any of a plurality of comparison images subsequent to the reference image. The first target portion has been displaced from the first region including at least a part of the determined first target portion to the second region located in the perpendicular direction of the first target portion. A first event indicating The second region indicating that the second region of interest is displaced from the third region including at least a part of the determined second region of interest to the fourth region located in the perpendicular direction of the second region of interest. Each of the two events is detected, and the output step detects the first event and the second event when the first event and the second event are detected with a time difference within a predetermined threshold. A signal corresponding to a composite event with an event is output .

According to the present invention, the attention portion that can specify the change in the relative positional relationship between the information processing apparatus and the object in the space on the basis of the edge of the object without depending on the information appearing on the surface of the object. And a signal corresponding to the change can be output. In addition, the change in the relative positional relationship between the information processing device and the object in the space can be detected by the displacement of the target part in two orthogonal directions, and the target part is displaced almost simultaneously in two orthogonal directions. It is possible to output a signal according to.

According to a fifth aspect of the present invention, in the information processing method according to the fourth aspect , the second area includes two areas positioned in opposite directions with respect to the first area, and The output step outputs the signal that differs depending on a previous region in which the target portion is displaced among the two regions included in the second region.

  According to the present invention, it is possible to output different signals depending on the direction in which the target portion is displaced from the first region.

According to a sixth aspect of the present invention, in the information processing method according to the fourth or fifth aspect , at least a part of the first edge fits in the first region, and the first region and the first region The boundary of the second region is parallel to the first edge, and at least a part of the second edge is within the third region, and the third region and the fourth region A notification step of notifying a user of information prompting adjustment of a relative positional relationship between the information processing apparatus and the object so that a boundary is parallel to the second edge; In the reference image in which at least a part of the first edge fits in the first area and at least a part of the second edge fits in the third area, the first edge is changed to the first area. And the second edge is determined as the second note. And determining a portion.

  According to the present invention, the relative positional relationship between the information processing apparatus and the object can be adjusted by the user so that the displacement of the target portion in the two orthogonal directions can be reliably detected.

According to a seventh aspect of the present invention, in the information processing method according to the fourth or fifth aspect , at least a part of the first edge fits in the first region, and the first region and the first region The boundary of the second region is parallel to the first edge, and at least a part of the second edge is within the third region, and the third region and the fourth region The method further includes an arranging step of arranging the first to fourth regions so that the boundary is parallel to the second edge, and the determining step includes at least a part of the first edge being the first edge. In the reference image in which at least a part of the second edge falls within the third area, the first edge is determined as the first attention part, and the second Is determined as the second target portion.

  According to the present invention, the information processing program is provided so that the user can reliably detect the displacement of the target portion in two orthogonal directions without adjusting the relative positional relationship between the information processing apparatus and the object. First to fourth regions can be arranged.

According to an eighth aspect of the present invention, in the information processing method according to any one of the fourth to seventh aspects, the output step corresponds to a detection time difference between the first event and the second event. A first signal corresponding only to the first event, a second signal corresponding only to the second event, and a combined event of the first event and the second event Any one of the third signals is selectively output.

  According to this invention, three signals can be used properly according to the time difference between the detection of the first event and the second event.

According to a ninth aspect of the present invention, in the information processing method according to the eighth aspect , the output step outputs the third signal that differs according to the detection order of the first event and the second event. It is characterized by outputting.

  According to this invention, two signals can be used properly according to the detection order of the first event and the second event.

A tenth aspect of the present invention is the information processing method according to any one of the fourth to ninth aspects, wherein the region including at least a part of the target portion and the perpendicular direction of the target portion are located. The distance between the region and the perpendicular direction is set to a threshold value or more.

  According to the present invention, it is possible to prevent the detection of the displacement due to the hand shake of the user who has at least one of the information processing apparatus and the object, and improve the detection accuracy.

An eleventh aspect of the present invention is the information processing method according to any one of the fourth to tenth aspects, wherein the imaging unit continuously captures images at a predetermined imaging interval, and the detection step includes the predetermined step. The event indicating that the target portion is displaced is detected so that the acceleration based on the imaging interval and the displacement amount of the target portion from the position in the reference image changes in a predetermined pattern.

  According to the present invention, it is possible to prevent the detection of the displacement due to the hand shake of the user who has at least one of the information processing apparatus and the object, and improve the detection accuracy.

The invention according to claim 12 is a determining unit that determines a target portion that can be specified with reference to an edge of an object appearing in the reference image in a reference image that constitutes a series of images continuously captured by the imaging unit; Detecting means for detecting an event indicating that the determined portion of interest has changed in any of a plurality of comparison images subsequent to the reference image; and corresponding to the detected event or another event related to the event Output means for outputting a signal to be output, wherein the object is a storage unit that stores data, and a processing unit that accesses the data in response to a request from a device that can communicate with the object And the information processing device performs predetermined processing based on the signal output by the output unit and data in the storage unit. To. The invention according to claim 13 is a determination step of determining a target portion that can be specified with reference to an edge of an object appearing in the reference image in a reference image that constitutes a series of images continuously captured by the imaging unit; A detection step for detecting an event indicating that the determined portion of interest has changed in any of a plurality of comparison images subsequent to the reference image; and corresponding to the detected event or another event related to the event An information processing program for causing a computer to execute an output step of outputting a signal to be executed, wherein the object accesses the data in response to a request from a storage unit that stores data and a device that can communicate with the object And the information processing program is output to the computer through the output step. A signal, characterized in that to perform a predetermined process based on the data of the storage unit.

  According to the present invention, it is possible to specify a change in the relative positional relationship between the information processing apparatus and the object in the space on the basis of the edge of the object without depending on the information appearing on the surface of the object. And a signal corresponding to the change can be output.

It is a figure which shows the board card 1 and the portable terminal 2 which are used in embodiment of this invention. (A) is a figure which shows the example of a schematic structure of the portable terminal 2, (B) is a figure which shows an example of the functional block in the control part 28. FIG. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. It is a figure which shows the example of the displacement event of an attention part. (A) is a figure which shows the attention part in case the board card 1 is imaged in the state in which the edge of the board card 1 turned sideways and the edge of the portable terminal 2 made vertical are parallel, (B) is a diagram showing an example in which the distance in the perpendicular direction between the region including the target portion and the region located in the normal direction of the target portion is set to be equal to or greater than a threshold value. It is a flowchart which shows an example of the displacement event detection and signal output process of an attention part. It is a figure showing an example of outline composition of electronic money system S to which the present invention is applied. It is a sequence diagram which shows an example of the charge operation of electronic value. It is a figure which shows a mode that charge amount is set by the change of the relative positional relationship of the electronic money card 1 and the portable terminal 2. FIG. It is a figure which shows a mode that charge amount is set by the change of the relative positional relationship of the electronic money card 1 and the portable terminal 2. FIG. It is a sequence diagram which shows an example of the information acquisition operation by the portable terminal. It is a figure which shows a mode that an information type is set by the change of the relative positional relationship of the electronic money card 1 and the portable terminal 2. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[1. Embodiment of the Invention]
First, an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing a board card 1 and a portable terminal 2 used in the embodiment of the present invention. FIG. 2A is a diagram illustrating a schematic configuration example of the mobile terminal 2. The board card 1 is an example of an object in the present invention, and the mobile terminal 2 is an example of an information processing apparatus in the present invention. Although the present invention is preferably applied to the board card 1, it can also be applied to, for example, a key holder, a wallet, a periodic case, a wristwatch, and the like other than the board card 1. Further, the present invention is preferably applied to the mobile terminal 2, but can also be applied to a stationary personal computer or the like.

  The board card 1 is a flat card whose dimensions are determined by, for example, international standards. For the board card 1, for example, an electronic money card, a credit card, a membership card, or the like can be applied. The present invention is applicable to both the board card 1 on which a non-contact type IC chip is mounted and the board card 1 on which a non-contact type IC chip is not mounted. The non-contact type IC chip mounted on the board card 1 is an IC module that employs Near Field Communication (NFC) technology using, for example, a 13.56 MHz band frequency. The non-contact type IC chip mounted on the board card 1 responds to a request from an antenna, a storage unit (for example, a nonvolatile memory) that stores data, and a portable terminal 2 that is within a distance range capable of short-range wireless communication. Accordingly, a processing unit (for example, a CPU) that performs a predetermined process on the data stored in the storage unit and responds with a processing result is provided. The shape of the board card 1 shown in FIG. 1 is a rectangle having rounded corners, but may be a square, a rhombus, a circle, or the like.

  The portable terminal 2 can be carried by a user, and for example, a smartphone, a mobile phone, a game machine, a tablet computer, etc. can be applied. As shown in FIG. 2, the mobile terminal 2 includes a camera 21 (an example of an imaging unit), a display unit 22, a speaker 23, an operation unit 24, a storage unit 25, a short-range wireless communication unit 26, a mobile wireless communication unit 27, And a control unit 28 and the like. The camera 21, display unit 22, speaker 23, operation unit 24, storage unit 25, short-range wireless communication unit 26, and mobile wireless communication unit 27 are controlled via an input / output interface unit (not shown) and the bus 29. Connected to the unit 28.

  The camera 21 is installed on the back side of the mobile terminal 2, for example, and images (photographs) a subject according to a user instruction. As an imaging mode by the camera 21, a moving image imaging mode and a still image imaging mode can be selected. For example, the image captured by the camera 21 is stored in an image storage area allocated to the storage unit 25 and displayed on the display D in the display unit 22. The display unit 22 includes a touch panel display D, for example. The display D is installed, for example, on the front side of the mobile terminal 2, and a screen for arranging user-designable icons according to a control signal from the control unit 28, an image of the board card 1 captured by the camera 21, and the like. Is displayed. The speaker 23 outputs the audio signal from the control unit 28 to the outside as audio. The operation unit 24 includes an input button for inputting an instruction from the user. The operation unit 24 can input an instruction via an icon displayed on the touch panel display D.

  The storage unit 25 is configured by a nonvolatile memory such as a flash memory or an EEPROM, for example. The storage unit 25 stores an OS (Operating System), various application programs, and the like. The application program includes the information processing program of the present invention. The application program can be downloaded from a predetermined server connected to the Internet, for example. Alternatively, the application program may be recorded on a recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc) and read from the recording medium into the storage unit 25 via a drive.

  The near field communication unit 26 includes, for example, an IC module that employs near field communication (NFC) technology using a 13.56 MHz band frequency, an antenna, and the like. The short-range wireless communication unit 26 communicates with the non-contact type IC chip when the board card 1 containing the non-contact type IC chip comes close to within a distance range where short-range wireless communication is possible. It has become. The mobile wireless communication unit 27 performs wireless communication with a base station in the mobile communication network. Thereby, the control part 28 can access a predetermined server via the internet, for example, and can communicate with the said server.

  The control unit 28 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. FIG. 2B is a diagram illustrating an example of functional blocks in the control unit 28. For example, the control unit 28 executes processing to be described later according to an information processing program stored in the storage unit 25. In such processing, as shown in FIG. 2 (B), the control unit 28 includes a target portion determination unit 28a, a change event detection unit 28b, an information notification unit 28c, a region arrangement unit 28d, a signal output unit 28e, and a process. It functions as the part 28f. The attention portion determination unit 28a is an example of a determination unit in the present invention. The change event detection unit 28b is an example of detection means in the present invention. The signal output unit 28e is an example of output means in the present invention.

  For example, in the state where the board card 1 and the portable terminal 2 are close to each other, the attention portion determination unit 28a captures a series of images continuously captured at a predetermined imaging interval by the camera 21 and configures the series of images. In the reference image, a target portion that can be specified with reference to the edge of the board card 1 appearing in the reference image is determined. Here, the series of images corresponds to a moving image captured in the moving image capturing mode and a series of still images continuously captured (continuous shooting) in the still image capturing mode. For example, the attention portion determination unit 28a detects the straight edge of the board card 1 appearing in the reference image, and determines the detected straight edge as the attention portion. One or more attention portions are determined. For example, when there are two attention portions, the attention portion determination unit 28a determines the first linear edge of the board card 1 appearing in the reference image as the first attention portion, and the first edge Is determined as a second target portion.

  The change event detection unit 28b detects an event indicating that the target portion determined by the target portion determination unit 28a has changed in any of a plurality of comparison images subsequent to the reference image. Here, the “comparison image” needs to be close in time (for example, about 1 to 2 seconds from the imaging time of the reference image), but does not have to be the next image immediately after the reference image. For example, the change event detection unit 28b detects that the target portion is located in a second region located in a perpendicular direction of a straight line passing through any two points on the target portion from the first region including at least a part of the target portion. An event indicating that the displacement (that is, the position of the target portion has changed) is detected. The attention portion in the comparison image is specified by detecting the straight edge of the board card 1 appearing in the comparison image, as in the reference image. In the present embodiment, the edge of the board card 1 is detected in each of the reference image and the comparison image without depending on the information appearing on the surface of the board card 1. In addition, when there are two attention parts determined as described above, the change event detection unit 28b determines the first attention from the first region including at least a part of the determined first attention part. A first event indicating that the first part of interest is displaced in a second region located in the direction of the normal of the part (that is, the direction of the perpendicular of a straight line passing through any two points on the first part of interest); , A perpendicular direction of the second target portion from the third region including at least a part of the determined second target portion (that is, a normal direction of a straight line passing through two arbitrary points on the second target portion) And a second event indicating that the second portion of interest has been displaced in the fourth region located at). The “second region” includes two regions located in opposite directions with respect to a straight line passing through two arbitrary points on the first target portion. The “fourth region” includes two regions located in opposite directions with respect to a straight line passing through two arbitrary points on the second target portion.

  In the following description, an event indicating that the attention portion has been displaced is referred to as a “displacement event of the attention portion”. Among these, in particular, the first event indicating that the first attention portion has been displaced is referred to as the The “first displacement event” and the second event indicating that the second portion of interest has been displaced are referred to as the “second displacement event of the portion of interest”, respectively.

  Here, an example of the displacement event of the target portion will be described with reference to FIGS. 3-9 is a figure which shows the example of the displacement event of an attention part. In the example of FIGS. 3 to 9, the board card 1 having a substantially rectangular edge appears in the reference image and the comparative image, respectively. The area A1a shown in the examples of FIGS. 3 and 4 is an example of the “first area”, and the area A1b and the area A1c are examples of the “second area”. 5 to 9 are examples of the “third area”, and the areas A2b and A2c are examples of the “fourth area”. The positions of these regions (that is, the coordinates in the image frame corresponding to the reference image and the comparison image) are set in advance, for example. Note that the positions and numbers of the areas illustrated in FIGS. 3 to 9 are merely examples, and a larger number of areas than the areas illustrated in FIGS. 3 to 9 may be set.

  In the example of FIG. 3A, a straight edge included in the area A1a is determined as the target portion P1 among the edges of the board card 1 appearing in the reference image. Then, from the state shown in FIG. 3A, the comparison image (for example, after the user has translated the board card 1 or the portable terminal 2 in the horizontal direction (for example, the board card 1 is translated in the right direction in the drawing)). After several frames), as shown in FIG. 3B, the target portion P1 is displaced from the region A1a to the region A1b. On the other hand, from the state shown in FIG. 3A, the user translates the board card 1 or the portable terminal 2 in the direction opposite to that in FIG. 3B (for example, the board card 1 is translated in the left direction of the drawing). )), The target portion P1 is displaced from the region A1a to the region A1c as shown in FIG. 3C. The displacement event of the target portion P1 shown in FIG. 3 (B) and the displacement event of the target portion P1 shown in FIG. 3 (C) are detected separately by the change event detection unit 28b. Thereby, the change of the relative positional relationship of the portable terminal 2 and the board card 1 in space can be detected by the displacement event of the attention part in one direction.

  Next, also in the example of FIG. 4A, the straight edge included in the region A1a is determined as the attention portion P1, as in FIG. 3A. And in the comparative image after the user moved away the board card 1 and the portable terminal 2 from the state shown in FIG. 4 (A), as shown in FIG. 4 (B), The attention portion P1 is displaced from the region A1a to the region A1b. On the other hand, in the comparative image after the user brings the board card 1 and the portable terminal 2 closer (closer to the vertical direction of the drawing) from the state shown in FIG. 4A, as shown in FIG. The attention portion P1 is displaced from the region A1a to the region A1c. Also in this case, the displacement event of the target portion P1 shown in FIG. 4B and the displacement event of the target portion P1 shown in FIG.

  Next, in the example of FIG. 5A, among the edges of the board card 1 appearing in the reference image, the linear edge included in the area A1a is determined as the first target portion P1, and the area A2a A straight edge including a part is determined as the second target portion P2. Then, the comparison image after the user has translated the board card 1 or the portable terminal 2 in the oblique direction from the state shown in FIG. 5A (for example, the board card 1 is translated in the diagonally lower right direction in the drawing). Then, as shown in FIG. 5B, the target portion P1 is displaced from the region A1a to the region A1b, and the target portion P2 is displaced from the region A2a to the region A2b. On the other hand, from the state shown in FIG. 5A, the user translates the board card 1 or the portable terminal 2 in the direction opposite to that in FIG. 5B (for example, the board card 1 is parallel to the diagonally upper left direction in the drawing). In the comparative image after the movement), as shown in FIG. 5C, the target portion P1 is displaced from the region A1a to the region A1c, and the target portion P2 is displaced from the region A2a to the region A2c. The first displacement event of the target portion P1 shown in FIG. 5 (B), the second displacement event of the target portion P2 shown in FIG. 5 (B), and the first displacement event of the target portion P1 shown in FIG. 5 (C). The second displacement event of the target portion P2 shown in FIG. 5C is detected by being distinguished by the change event detection unit 28b. As described above, in the example shown in FIG. 5, it is possible to detect displacement events in two orthogonal directions. Thereby, the change in the relative positional relationship between the mobile terminal 2 and the board card 1 in the space can be detected by the displacement event of the target portion in two orthogonal directions.

  Next, in the example of FIG. 6A, as in FIG. 5A, the straight edge included in the area A1a among the edges of the board card 1 appearing in the reference image is the first target portion P1. A straight edge that is determined and partly included in the region A2a is determined as the second target portion P2. Then, in the comparative image after the user moves the board card 1 and the portable terminal 2 away from the state shown in FIG. 6A, as shown in FIG. 6B, the attention portion P1 is changed from the area A1a to the area A1b. And the target portion P2 is displaced from the region A2a to the region A2b. On the other hand, in the comparative image after the user brings the board card 1 and the portable terminal 2 closer to each other from the state shown in FIG. 6A, as shown in FIG. And the target portion P2 is displaced from the region A2a to the region A2c. The first displacement event of the target portion P1 shown in FIG. 6 (B), the second displacement event of the target portion P2 shown in FIG. 6 (B), and the first displacement event of the target portion P1 shown in FIG. 6 (C). The second displacement event of the target portion P2 shown in FIG. 6C is detected by being distinguished by the change event detecting unit 28b. Thus, in the example shown in FIG. 6, it becomes possible to detect displacement events in two orthogonal directions. In the example shown in FIG. 6, the first displacement event of the target portion P1 shown in FIG. 6B and the second displacement event of the target portion P2 shown in FIG. 6B are almost simultaneously (within a predetermined threshold). Is detected). Similarly, the first displacement event of the target portion P1 shown in FIG. 6C and the second displacement event of the target portion P2 shown in FIG. 6C are detected almost simultaneously (with a time difference within a predetermined threshold). The

  In the example of FIGS. 3 to 6, it is assumed that the board card 1 is imaged in a state where the edge of the board card 1 turned sideways and the edge of the portable terminal 2 turned sideways are parallel. However, the case where the board card 1 is imaged in the state where the edge of the board card 1 and the edge of the portable terminal 2 are non-parallel is assumed. In this case, the information notification unit 28c has at least a part of the straight edge of the board card 1 within the area A1a, and the boundary between the area A1a and the area A1b (or the area A1c) is parallel to the straight edge. As such, the user may be notified of information that prompts the user to adjust the relative positional relationship between the mobile terminal 2 and the board card 1. Thereby, a user can adjust the relative positional relationship of the portable terminal 2 and the board card 1 so that the change event detection part 28b can detect the displacement event of the attention part in one direction reliably. . Then, after the user adjusts the relative positional relationship between the portable terminal 2 and the board card 1, for example, the state shown in FIG. In the reference image in which at least a part of the edge is within the area A1a, the straight edge is determined as the target portion P1. For example, the information notification unit 28c notifies the user by displaying information for prompting the adjustment on the display D or outputting the information from the speaker 23 as sound.

  Further, when the board card 1 is imaged in a state where the edge of the board card 1 and the edge of the mobile terminal 2 are non-parallel, the information notification unit 28c is configured to at least part of the linear first edge of the board card 1. Is within the region A1a, and the boundary between the region A1a and the region A1b (or the region A1c) is parallel to the first edge, and at least a second linear edge perpendicular to the first edge The relative positional relationship between the mobile terminal 2 and the board card 1 is such that a part of the mobile terminal 2 is within the area A2a and the boundary between the area A2a and the area A2b (or the area A2c) is parallel to the second edge. Information that prompts adjustment may be notified to the user. This allows the user to adjust the relative positional relationship between the portable terminal 2 and the board card 1 so that the change event detection unit 28b can reliably detect the displacement event of the target portion in two orthogonal directions. Can do. Then, after the user adjusts the relative positional relationship between the mobile terminal 2 and the board card 1, for example, the state shown in FIG. In the reference image in which at least a part of the second edge falls within the area A1a and at least a part of the second edge falls within the area A2a, the first edge is determined as the first target part P1, and the first The second edge is determined as the second target portion P2.

  On the other hand, when the board card 1 is imaged in a state in which the edge of the board card 1 and the edge of the mobile terminal 2 are non-parallel, the region placement unit 28d has at least a part of the straight edge of the object of the board card 1. Arrange the first region and the second region so that they fit in the first region and the boundary between the first region and the second region is parallel to the linear edge (that is, You may comprise so that the position of the said area | region may be set on the image frame corresponding to a reference | standard image and a comparison image. Thus, the region placement unit 28d can change the position of the target portion in one direction without detecting the relative positional relationship between the portable terminal 2 and the board card 1 so that the user can reliably detect the displacement event in the one direction. The first and second regions can be arranged. In addition, the area arranging portion 28d has at least a part of the linear first edge of the board card 1 within the first area, and the boundary between the first area and the second area is the first edge. At least a portion of the linear second edge perpendicular to the first edge is within the third region, and the boundary between the third region and the fourth region is the first region. You may comprise so that the said 1st-4th area | region may each be arrange | positioned so that it may become parallel to 2 edges. Thus, the region placement unit 28d can reliably detect the displacement event of the target portion in the two orthogonal directions without adjusting the relative positional relationship between the mobile terminal 2 and the board card 1. Can arrange | position the 1st-4th area | region.

  FIG. 7 is a diagram illustrating an example of the displacement event of the target portion when the regions A1a, A1b, A1c, A2a, A2b, and A2c arranged by the region arranging unit 28d are used. As shown in FIG. 7A, after the region is arranged, the attention portion determination unit 28a has at least a part of the first edge within the region A1a and at least one of the second edge. In the reference image in which the portion falls within the region A2a, the first edge is determined as the first target portion P1, and the second edge is determined as the second target portion P2. Also in this case, the first displacement event of the target portion P1 shown in FIG. 7B, the second displacement event of the target portion P2 shown in FIG. 7B, and the target portion P1 shown in FIG. 7C. The first displacement event and the second displacement event of the target portion P2 shown in FIG. 7C are detected by being distinguished by the change event detection unit 28b.

  Incidentally, for example, the first displacement event of the target portion P1 and the second displacement event of the target portion P2 shown in FIG. 5 may occur after a certain period of time. The example of FIG. 8 shows a case where the user translates the board card 1 or the portable terminal 2 in the horizontal direction (FIG. 8B) and then translates in the vertical direction (FIG. 8C). In this case, the detection order of the first displacement event of the target portion P1 is earlier than the second displacement event of the target portion P2. On the other hand, in the example of FIG. 2 is translated in the vertical direction (FIG. 9B) and then translated in the horizontal direction (FIG. 9C). In this case, the second displacement event of the target portion P2 is shown. However, the detection order of the first displacement event and the second displacement event is temporarily associated with each displacement event in the RAM or the like. Thereby, for example, the signal output unit 28c detects the first displacement event and the second displacement event. Time difference, and the detection order of the first displacement event and the second displacement event can be specified.

  FIG. 10A shows the attention portion P1 and the attention when the board card 1 is imaged in a state in which the edge of the board card 1 turned sideways is parallel to the edge of the portable terminal 2 turned vertically. Part P2 is shown. Even in this case, it is possible to detect a displacement event in two orthogonal directions by the change event detection unit 28b.

  FIG. 10B shows the perpendicular direction distance d1 between the region A1a including the target portion P1 and the regions A1b and A1c located in the vertical direction of the target portion P1, and a part of the target portion P2. The vertical direction distance d2 between the region A2a including the region A2a and the regions A2b and A2c located in the vertical direction of the target portion P2 is set to be equal to or more than a threshold value. Note that either one of the distance d1 and the distance d2 may be set to a threshold value or more. For example, a region located between the region A1b including the attention portion P1 and the region A1b whose distance from the region A1a is equal to or greater than a threshold is referred to as a buffer region BA. Even if the target portion is displaced to the buffer area BA in the comparison image, the change event detecting unit 28b does not detect the displacement event of the target portion. Accordingly, even if the attention portion is displaced to the buffer area BA due to the hand shake of the user who has at least one of the board card 1 and the portable terminal 2, the displacement event can be prevented from being detected, and the detection accuracy can be improved. .

  As another example, the change event detection unit 28b displaces the target portion so that the acceleration based on the predetermined imaging interval by the camera 21 and the amount of displacement of the target portion from the position in the reference image changes in a predetermined pattern. You may comprise so that the displacement event which shows having performed may be detected. For example, the acceleration is calculated by the control unit 28 in time series based on a predetermined imaging interval by the camera 21 and the amount of displacement from the position of the target portion in the reference image. When the moving direction of the target portion is set to the positive direction, the change event detection unit 28b has an initial acceleration (> 0) at which the target portion is displaced being equal to or greater than the first threshold value, and the target portion is displaced. When the final acceleration (<0) is less than or equal to the second threshold value, the acceleration is detected to change in a predetermined pattern, and the displacement event of the target portion is detected. Here, the acceleration is an acceleration in the same direction as the moving direction of the target portion. When the initial acceleration (> 0) is smaller than the first threshold value, or when the final acceleration (<0) is larger than the second threshold value, it is regarded as a camera shake, and the displacement event of the target portion is not detected. Thus, even without using an acceleration sensor, it is possible to prevent the displacement event of the attention portion from being detected due to the hand shake of the user who has at least one of the board card 1 and the portable terminal 2 only by image processing. Can be improved.

  Next, the signal output unit 28e outputs, for example, a signal corresponding to the displacement event detected by the change event detection unit 28b or another event related to the detected displacement event (hereinafter referred to as “related event”). Output to 28f. Here, as an example of the related event, a combined event of the first displacement event and the second displacement event can be cited. The correspondence relationship between each displacement event or related event described above and an output signal may be defined on the information processing program of the present invention, or may be defined on a table stored in the storage unit 25. Also good. For example, in the example of FIGS. 3 to 7, when a displacement event (also a first displacement event) of the target portion P <b> 1 is detected, the signal SG <b> 1 is output. In the example of FIGS. 5 to 7, when the second displacement event of the target portion P2 is detected, a signal SG2 different from the signal SG1 is output. Further, the signal output unit 28e may be configured to output different signals depending on the previous region where the target portion is displaced among the two regions included in the second or fourth region. Thereby, a different signal can be output according to the direction in which the target portion is displaced from the first or third region. For example, in the example of FIGS. 3 to 7, the signal SG <b> 11 is output when the area where the target portion P <b> 1 is displaced is the area A <b> 1 b. On the other hand, when the region where the target portion P1 is displaced is the region A1c, a signal SG12 different from the signal SG11 is output. In the example of FIGS. 5 to 7, the signal SG <b> 21 is output when the region where the target portion P <b> 2 is displaced is the region A <b> 2 b. On the other hand, when the region where the target portion P2 is displaced is the region A3c, a signal SG22 different from the signal SG21 is output.

  Further, the signal output unit 28e detects the first displacement event of the target portion P1 and the second displacement event of the target portion P2 with a time difference within a predetermined threshold (for example, in the examples of FIGS. If detected, a signal SG3 (different from signal SG1 and signal SG2) corresponding to a composite event (an example of a related event) of the first displacement event and the second displacement event is output. May be. As a result, it is possible to output a signal corresponding to the fact that the target portions P1 and P2 are displaced substantially simultaneously in two orthogonal directions. In particular, in the example of FIGS. 5 and 7, when the first displacement event and the second displacement event are detected with a time difference exceeding a predetermined threshold value, the signal SG1 corresponding to the first displacement event (or The signal SG11 or the signal SG12) is output, and the signal SG2 (or the signal SG21 or the signal SG22) corresponding to the second displacement event is output. That is, the signal output unit 28e has a first signal (SG1, SG11, or SG12) corresponding to only the first displacement event according to the detection time difference between the first displacement event and the second displacement event. , A second signal (SG2, SG21, or SG22) corresponding only to the second displacement event, and a third signal SG3 corresponding to a composite event of the first displacement event and the second displacement event, One of them is selectively output. As a result, the three signals can be used properly according to the detection time difference between the first displacement event and the second displacement event. In addition, when the first displacement event and the second displacement event are detected with a time difference exceeding a predetermined threshold, the signal output unit 28e determines the detection order of the first displacement event and the second displacement event. A different third signal may be output accordingly. For example, in the case of the detection order shown in the example of FIG. 8, the signal SG31 is output. On the other hand, in the detection order shown in FIG. 9, a signal SG32 different from the signal SG31 is output. Thereby, two signals can be properly used according to the detection order of the first displacement event and the second displacement event. In the example of FIG. 6, the first displacement event of the target portion P1 and the second displacement event of the target portion P2 are set so as not to be detected with a time difference exceeding a predetermined threshold (detected almost simultaneously). Is done. Therefore, the output is limited to the output of the third signal SG3 corresponding to the combined event of the first displacement event and the second displacement event.

  The processing unit 28f performs a predetermined process according to the signal output from the signal output unit 28e. Examples of predetermined processing include processing for inputting predetermined information (including parameters) from the outside, processing for outputting predetermined information (including parameters) to the outside, processing for setting predetermined parameters, and displaying predetermined information For example, a process of displaying on D. The process to be executed is determined from these processes according to the signal output from the signal output unit 28e. Further, as an example of processing for inputting predetermined information from the outside, for example, processing for inputting (acquiring) information by a “read command” from a non-contact IC chip built in the board card 1, or the mobile wireless communication unit 27 For example, a process of acquiring the information by requesting the predetermined information to the predetermined server through the server. As an example of processing for outputting predetermined information to the outside, for example, processing for outputting predetermined information by a “write command” to a non-contact type IC chip built in the board card 1, or predetermined processing through the mobile radio communication unit 27 For example, a process of outputting (transmitting) predetermined information to a device (for example, a server). As an example of processing for setting the predetermined parameter, data (stored in the storage unit) is received in response to a request from the portable terminal 2 within a distance range in which the non-contact type IC chip built in the board card 1 is capable of short-range wireless communication. There is a process for setting parameters used for a predetermined process to be executed on the data). Thereby, the parameter used for the said process can be set by changing the relative positional relationship of the board card 1 and the portable terminal 2. FIG.

  FIG. 11 is a flowchart illustrating an example of the displacement event detection and signal output processing of the target portion. The process illustrated in FIG. 11 is started when a start instruction is input from the user via the operation unit 24, for example. When the process shown in FIG. 9 is started, the control unit 28 starts capturing a series of images continuously captured by the camera 21 (step S1). Next, the control unit 28 performs a target part determination process (step S2). The attention part determination process is a process in which the attention part determination unit 28a determines an attention part that can be specified with reference to the edge of the board card 1 appearing in the reference image in the reference image constituting the series of images. Next, the control unit 28 determines whether or not the target portion has been determined in the reference image (step S3). If it is determined that the target portion cannot be determined in the reference image (step S3: NO), the process proceeds to step S4. In step S4 here, the information notification unit 28c notifies the user of information for urging adjustment of the relative positional relationship between the board card 1 and the portable terminal 2 so that the target portion can be determined. Here, the area arranging portion 28d has at least a part of the linear edge of the object of the board card 1 within the first area, and the boundary between the first area and the second area is the linear edge. The first region and the second region may be arranged so as to be parallel to each other.

  On the other hand, when it is determined that the target portion can be determined in the reference image (step S3: YES), the process proceeds to step S5. In step S <b> 5, the control unit 28 executes a detection process for the displacement event of the target portion. The attention part displacement event detection process is a process in which the change event detection unit 28b detects an event indicating that the attention part is displaced in any of a plurality of comparison images subsequent to the reference image. Next, the control unit 28 determines whether or not the displacement event of the target portion has been detected (step S6). And when it determines with the displacement event of an attention part not being detectable (step S6: NO), it progresses to step S4. In step S4 here, the information notification unit 28c notifies the user of information for urging adjustment of the relative positional relationship between the board card 1 and the portable terminal 2 so that the displacement event of the target portion can be detected.

  On the other hand, when it is determined that the displacement event of the target portion has been detected (step S6: YES), the process proceeds to step S7. In step S7, the signal output unit 28e outputs a signal corresponding to the detected displacement event or related event to the processing unit 28f. Thus, the processing unit 28f performs a predetermined process according to the output signal. Next, the control unit 28 determines whether or not there is an end instruction (step S8). For example, when an end instruction is input from the user via the operation unit 24, it is determined that there is an end instruction (step S8: YES), and the process illustrated in FIG. 11 ends. On the other hand, when it is determined that there is no termination instruction (step S8: NO), the process returns to step S2 and the above process is repeated.

  As described above, according to the embodiment of the present invention, the change in the relative positional relationship between the board card 1 and the mobile terminal 2 in the space does not depend on the information appearing on the surface of the board card 1. In addition, it can be detected by a change in the portion of interest that can be specified with the edge of the board card 1 as a reference, and a signal corresponding to the change can be output.

In the above-described embodiment, the target portion determination unit 28a is configured to determine the straight edge of the object as the target portion. ), A surface surrounded by the edge, a tangent line that touches the edge, or a point on the tangent line may be determined as a target portion. Further, in the above embodiment, the change event detection unit 28b is configured to detect an event indicating that the target portion has been displaced (that is, the position of the target portion has changed). An event indicating that the area of the surface surrounded by the edge has changed may be detected. A change in the area of a surface means a change in the occupation ratio of the surface in an image frame or a specific region in the image frame.
[2. Example]
Next, an embodiment in which the present invention is applied to the electronic money system S will be described with reference to FIGS. FIG. 12 is a diagram showing a schematic configuration example of an electronic money system S to which the present invention is applied. As shown in FIG. 12, the electronic money system S is configured using an electronic money card 1, a portable terminal 2, an electronic money server 3, a database DB, and the like. The electronic money card 1 is an example of the board card 1 described above. The electronic money server 3 is an example of a balance change device.

  The electronic money card 1 includes a non-contact type IC chip 1a (IC module) having a storage unit and a processing unit, and an antenna. In a storage unit (for example, a non-volatile memory) of the electronic money card 1, balance data indicating a balance of electronic value, log data, an electronic money number, and the like are stored. Electronic value is electronic information corresponding to monetary value. The electronic money number is, for example, identification information for identifying the balance of electronic value from the balance of electronic value of another electronic money card. Log data is stored in the non-contact IC chip 1a in real time at the time of transaction or charging. However, since the data storage capacity of the storage unit of the non-contact IC chip 1a is small, the number of log data that can be stored in the storage unit is limited to several. For this reason, when log data exceeding the limited number is stored, the processing unit deletes the oldest log data and stores the newest log data. Thereby, the latest log data for the limited number of cases is stored in the storage unit of the non-contact IC chip 1a. The log data corresponds to, for example, transaction history data and charge history data. The transaction history data is information generated with respect to a transaction in which an electronic money card is used for settlement (payment) at a store, for example. The transaction history data includes, for example, data such as an electronic money number, transaction date and time, transaction amount, and store ID of the store where the transaction was made. The store ID is identification information uniquely assigned to each store. The charge history data is information that is generated with respect to a charge in which the electronic money server 3 is used, for example. Here, charging means increasing the balance of the electronic value stored in the storage unit of the non-contact IC chip 1a. The charge history data includes data such as an electronic money number, a charge date and time, and a charge amount. In addition, the electronic money card can be charged from a payment terminal installed in the store. In this case, the charge history data includes, for example, data such as an electronic money number, a charge date and time, a charge amount, and a store ID of the charged store. The processing unit (for example, CPU) of the electronic money card 1 is stored in the storage unit using balance change information input from a device (for example, the portable terminal 2) within a distance range in which near field communication is possible. A balance change process for changing the balance data is executed, and the result of the process is returned.

  The electronic money server 3 is a server that manages the movement of monetary value by electronic value, and includes a communication unit, a storage unit, a control unit, and the like. The monetary value is transferred by changing (increasing or decreasing) the balance of electronic value. The communication unit of the electronic money server 3 is connected to a network NW composed of a mobile communication network and the Internet, and controls the communication state. The storage unit of the electronic money server 3 is configured by, for example, a hard disk drive, and stores an operating system and an electronic money management program for performing settlement processing using electronic value and electronic value charging processing. The control unit of the electronic money server 3 includes a CPU, a ROM, a RAM, and the like, and performs a payment process using an electronic value and a charge process using an electronic value according to an electronic money management program.

  The database DB is provided in a database server installed in the electronic money server 3 or outside the electronic money server 3. The database DB is provided with a user information database, a store information database, a log database, and an electronic value database. User information of users (members) of electronic money cards is registered for each user in the user information database. Here, the user information includes, for example, a user ID, a password, a user name, an address, a telephone number, an e-mail address, and an electronic money number. The user ID is identification information uniquely assigned to each user. Store information database and store information of stores (member stores) that can handle electronic money cards are registered for each store. The store information includes, for example, a store ID, a password, a store name, an address, a telephone number, and an e-mail address. In the log database, for example, log data transmitted from the payment terminal of the store to the electronic money server 3 via the network NW every predetermined time, for example, in batch processing is registered. In the electronic value database, the management value of the electronic value balance is registered in association with the electronic money number or the like. The management value of the electronic value balance indicates the balance of the electronic value specified by the electronic money number. The electronic money server 3 issues privilege information (for example, a coupon) to a user who has made a transaction or charge based on the log data registered in the log database by satisfying the privilege information issue condition. The privilege information is, for example, information indicating that a special benefit or treatment such as a discount can be received from a store when a payment is made for a product or service. As a condition for issuing privilege information, for example, “within the last x months (period), at store xx (or at more than x stores) (location), there were x transactions (or x charges). ) (Number of times) "or" within the last x months (period), at the store xx (or at more than x stores) (location), there was a transaction for x yen (or x The condition “(Yen charged) (amount)” is set.

  FIG. 13 is a sequence diagram illustrating an example of an electronic value charging operation. 14 and 15 are diagrams showing how the charge amount is set by the change in the relative positional relationship between the electronic money card 1 and the portable terminal 2. In the example of FIG. 13, first, the user brings the electronic money card 1 and the portable terminal 2 close to each other within a distance range in which short-range wireless communication is possible, and a predetermined instruction (for example, the balance of the electronic money card 1 is displayed). When the confirmation instruction is performed, the mobile terminal 2 activates the camera 21 and requests balance data from the electronic money card 1 via the short-range wireless communication unit 26 (hereinafter referred to as “balance request”). Then, by a read command) (step S11). In response to the balance request, the non-contact IC chip 1a of the electronic money card 1 reads the electronic money number together with the balance data stored in the storage unit and transmits it to the portable terminal 2 (step S12).

  The portable terminal 2 acquires the electronic money number together with the balance data from the electronic money card 1, and starts the process shown in FIG. 11 (step S13). Next, the portable terminal 2 displays information prompting the user to determine a desired charge amount on the display D (step S14). In the example of FIG. 14, in order to match the basic position shown in FIG. 14B, the user fits the edge of the electronic money card 1 in the area between the frame a and the frame b displayed on the display D, for example. As described above, the relative positional relationship between the electronic money card 1 and the portable terminal 2 is adjusted. Note that a partial region between the frame a and the frame b corresponds to the region A1a and the region A2a illustrated in FIG. Next, the user changes the relative positional relationship between the electronic money card 1 and the portable terminal 2 so as to correspond to a desired charge amount. For example, when the user wants to charge only ¥ 1,000, the electronic money card 1 and the portable terminal 2 are arranged so that the edge of the electronic money card 1 fits in the area within the frame a from the basic position (FIG. 14A). And keep them away vertically. In addition, the partial area | region in the frame a respond | corresponds to area | region A1b and area | region A2b shown in FIG. On the other hand, for example, when the user wants to charge for ¥ 5,000, the electronic money card 1 is placed in an area between the frame b and the frame c from the basic position (FIG. 14C). The money card 1 and the portable terminal 2 are brought close to each other in the vertical direction. A partial region between the frame b and the frame c corresponds to the region A1c and the region A2c illustrated in FIG. On the other hand, for example, when the user wants to charge for ¥ 3,000, for example, the electronic money card 1 is placed in an area between the frame a and the frame b from the state where the edge of the electronic money card 1 is in the frame a. The relative positional relationship between the money card 1 and the portable terminal 2 is adjusted. Thus, when the relative positional relationship between the electronic money card 1 and the portable terminal 2 is changed, the displacement detected by the change event detection unit 28b in the displacement event detection and signal output processing of the target portion shown in FIG. A signal corresponding to the event is output from the signal output unit 28e to the processing unit 28f. Thereby, the processing unit 28f of the portable terminal 2 displays the charge amount corresponding to the signal on the display D according to the output signal (step S15). For example, as shown in FIGS. 14A to 14C, “¥ 1,000 charge”, “¥ 3,000 charge”, and “¥ 5,000 charge” are switched and displayed. Further, the processing unit 28f of the portable terminal 2 sets a charge amount (amount to be added to the balance of the electronic value) corresponding to the output signal when the screen of the display D is tapped by the user, for example (Step S28). S16).

  On the other hand, in the example of FIG. 15, the user moves the electronic money card 1 in the right direction in the figure (or the mobile terminal 2 in the left direction in the figure), for example, so as to correspond to the desired charge amount. The relative positional relationship between the money card 1 and the portable terminal 2 is changed. In the example of FIG. 15, for example, a partial area between the amount line of ¥ 3,000 and the amount line of ¥ 5,000 corresponds to the area A1c shown in FIG. The partial area between the amount line and the amount line of ¥ 7,000 corresponds to the area A1a shown in FIG. 3, and the partial area between the amount line of ¥ 7,000 and the amount line of ¥ 10,000 is , Corresponding to the region A1b shown in FIG. The user adjusts the relative positional relationship between the electronic money card 1 and the portable terminal 2 to finally align the edge of the electronic money card 1 between the amount lines corresponding to the desired charge amount. In response to this, in the displacement event detection and signal output processing of the target portion shown in FIG. 11, a signal corresponding to the displacement event detected by the change event detection unit 28b is output from the signal output unit 28e to the processing unit 28f. Thereby, the processing unit 28f of the portable terminal 2 displays the charge amount corresponding to the signal on the display D according to the output signal (step S15). For example, as shown in FIGS. 15A to 15C, “¥ 3,000 charge”, “¥ 5,000 charge”, and “¥ 7,000 charge” are switched and displayed. Further, the processing unit 28f of the portable terminal 2 sets a charge amount (amount to be added to the balance of the electronic value) corresponding to the output signal when the screen of the display D is tapped by the user, for example (Step S28). S16).

  Next, the processing unit 28 f of the mobile terminal 2, for example, in accordance with a transmission instruction from the user, the set charge amount, balance data and electronic money number acquired from the non-contact IC chip 1 a of the electronic money card 1, Is transmitted to the electronic money server 3 via the mobile wireless communication unit 27 (step S17). The electronic money number may be acquired through an image from, for example, printing on the surface of the electronic money card 1. Next, when receiving the charge amount, balance data, and electronic money number, the electronic money server 3 searches the database DB using the electronic money number as a key (step S18). Next, the electronic money server 3 acquires necessary information by searching the database DB, and then performs user authentication processing (step S19). When the result of the authentication process is good, the electronic money server 3 refers to the management value of the balance of the electronic value associated with the electronic money number in the electronic value database and changes the balance data by the charge amount. Balance change information (write command in this embodiment) is generated, and the balance change information is transmitted to the portable terminal 2 (step S20). Here, for example, when charging ¥ 2,000 at a balance of ¥ 1,000, the write command is, for example, a command “overwrite the balance to ¥ 3,000”. In this case, the command is added to this command. The parameter to be set is “¥ 3,000”. As another example of the balance change information, an addition command may be applied. This addition command is, for example, a command “Add 2,000 to the balance”. In this case, the parameter added to this command is “¥ 2,000”.

  Next, the processing unit 28f of the portable terminal 2 acquires the balance change information transmitted from the electronic money server 3 and inputs (transfers) the balance change information to the non-contact type IC chip 1a of the electronic money card 1 (step S21). Next, the non-contact IC chip 1a of the electronic money card 1 executes balance change processing for changing balance data stored in the storage unit using the balance change information input from the mobile terminal 2 (step S22). The processing result is returned (step S23). Next, the portable terminal 2 transfers the processing result acquired from the non-contact type IC chip 1a to the electronic money server 3 (step S24). When the electronic money server 3 receives the processing result transmitted from the mobile terminal 2, the electronic money server 3 transmits a completion notification to the mobile terminal 2 (step S25).

  In the electronic value charging operation shown in FIG. 13, when the processing unit 28f of the mobile terminal 2 sets the charge amount according to the output signal, the charge amount and the non-contact type IC chip 1a are acquired. The electronic money number may be transmitted to the electronic money server 3 (that is, balance data is not transmitted at this time). In this case, the electronic money server 3 acquires balance data from the non-contact type IC chip 1a via the portable terminal 2 (by a read command). Next, the electronic money server 3 performs the processes of steps S18 and S19, and inputs balance change information to the non-contact type IC chip 1a via the portable terminal 2 (by a write command). Then, the contactless IC chip 1a of the electronic money card 1 executes balance change processing for changing balance data stored in the storage unit using the balance change information input from the mobile terminal 2, and obtains the processing result. It responds via the portable terminal 2. When the electronic money server 3 receives the processing result transmitted from the mobile terminal 2, the electronic money server 3 transmits a completion notification to the mobile terminal 2.

  As described above, according to the electronic value charging operation shown in FIG. 13, by changing the relative positional relationship between the electronic money card 1 and the portable terminal 2, the contactless IC chip 1 a of the electronic money card 1 is changed. The balance data stored in the storage unit can be changed.

  FIG. 16 is a sequence diagram illustrating an example of an information acquisition operation performed by the mobile terminal 2. FIG. 17 is a diagram illustrating a state in which the information type is set by a change in the relative positional relationship between the electronic money card 1 and the mobile terminal 2. This information type is the type of information requested to the electronic money server 3. This information type indicates, for example, privilege information such as a coupon or a log. In the example of FIG. 16, first, the user brings the electronic money card 1 and the portable terminal 2 close to each other within a distance range in which short-range wireless communication is possible, and a predetermined instruction (for example, the balance of the electronic money card 1 is displayed). When the confirmation instruction is performed, the portable terminal 2 activates the camera 21 and requests log data from the electronic money card 1 via the short-range wireless communication unit 26 (in this embodiment, by a read command) ( Step S31). In response to the log data request, the non-contact IC chip 1a of the electronic money card 1 reads the electronic money number together with the log data stored in the storage unit and transmits it to the mobile terminal 2 (step S32).

  The portable terminal 2 acquires the electronic money number together with the log data from the electronic money card 1 and starts the process shown in FIG. 11 (step S33). Next, the portable terminal 2 displays the balance included in the acquired log data and information prompting the user to determine a desired information type on the display D (step S34). The mobile terminal 2 may request balance data from the electronic money card 1 separately from the log data request, and acquire the balance data from the electronic money card 1. In this case, the balance indicated by the balance data may be displayed. In the example of FIG. 17, in order to match the basic position shown in FIG. 17B, the user fits the edge of the electronic money card 1 in an area between the frame a and the frame b displayed on the display D, for example. As described above, the relative positional relationship between the electronic money card 1 and the portable terminal 2 is adjusted. Next, the user changes the relative positional relationship between the electronic money card 1 and the portable terminal 2 so as to correspond to a desired information type. For example, when the user wants to display privilege information such as a coupon, the electronic money card 1 is arranged so that the edge of the electronic money card 1 fits in the area within the frame a displayed on the display D (FIG. 17A). 1 and the portable terminal 2 are moved away from each other in the vertical direction. On the other hand, for example, when the user wants to display the contents of the log data, the edge of the electronic money card 1 fits in an area between the frame b and the frame c displayed on the display D (FIG. 8C )) The electronic money card 1 and the portable terminal 2 are brought close to each other in the vertical direction. Thus, when the relative positional relationship between the electronic money card 1 and the portable terminal 2 is changed, the displacement detected by the change event detection unit 28b in the displacement event detection and signal output processing of the target portion shown in FIG. A signal corresponding to the event is output from the signal output unit 28e to the processing unit 28f. Thereby, the processing unit 28f of the mobile terminal 2 displays the information type corresponding to the signal on the display D in accordance with the output signal (step S35). For example, as shown in FIGS. 17A to 17C, “coupon”, “balance ¥ 3,000”, and “log” are switched and displayed. Furthermore, when the screen of the display D is tapped by the user, for example, the processing unit 28f of the mobile terminal 2 sets an information type corresponding to the output signal (step S36).

  Next, the processing unit 28f of the portable terminal 2 receives the information indicating the set information type and the electronic money number acquired from the non-contact type IC chip 1a of the electronic money card 1 in response to a transmission instruction from the user. It transmits to the electronic money server 3 via the mobile radio | wireless communication part 27 (step S37). Next, when receiving the information type and the electronic money number, the electronic money server 3 searches the database DB using the electronic money number as a key (step S38). Next, the electronic money server 3 obtains necessary information by searching the database DB, and then performs user authentication processing (step S39). If the result of the authentication process is good, the electronic money server 3 acquires information corresponding to the received information type and transmits it to the mobile terminal 2 (step S40). For example, when the information type indicates privilege information and the user satisfies the privilege information issuance condition, privilege information for the user is issued, and the privilege information is transmitted to the mobile terminal 2 as information corresponding to the information type. On the other hand, when the information type indicates a log, log data including the acquired electronic money number is acquired from the log database, and the log data is transmitted to the mobile terminal 2 as information corresponding to the information type.

  Next, the processing unit 28 f of the mobile terminal 2 acquires information (privilege information or log data) transmitted from the electronic money server 3, and uses the information corresponding to the information type set on the display D of the display unit 22. (Step S41). By the way, when the log data is acquired from the electronic money server 3, the log data and the log data acquired from the electronic money card 1 are integrated and displayed on the display D.

  As described above, according to the information acquisition operation illustrated in FIG. 16, the privilege information display and the log data display are switched by changing the relative positional relationship between the electronic money card 1 and the mobile terminal 2. be able to.

  In the above embodiment, an example in which the present invention is applied to the electronic money system S has been described. However, the present invention can be applied to other information processing systems. For example, the present invention can be applied to a payment processing system using a credit card or a payment processing system such as an online transaction.

DESCRIPTION OF SYMBOLS 1 Board card 2 Portable terminal 3 Electronic money server 21 Camera 22 Display part 23 Speaker 24 Operation part 25 Memory | storage part 26 Short-distance wireless communication part 27 Mobile radio | wireless communication part 28 Control part

Claims (13)

An information processing method that an information processing program causes an information processing apparatus to perform,
The information processing method includes:
A determination step of determining a target portion that can be identified with reference to an edge of an object appearing in the reference image in a reference image constituting a series of images continuously captured by the imaging means;
A detection step of detecting an event indicating that the determined portion of interest has changed in any of a plurality of comparison images subsequent to the reference image;
Outputting a signal corresponding to the detected event or other event related to the event;
Including
The object includes an IC module having a storage unit for storing data, and a processing unit for accessing the data in response to a request from a device capable of communicating with the object,
The information processing method performs predetermined processing based on the signal output in the output step and the data in the storage unit. The object is a card including an IC module having the storage unit and the processing unit,
The storage unit stores balance data indicating a balance of electronic value,
The processing unit executes balance change processing for changing the stored balance data using balance change information input from a device within a short range wireless communication range, and responds with a processing result. Yes,
The information processing method includes:
A setting step of setting a charge amount indicating an amount to be added to the balance according to the output signal;
An input step of acquiring balance change information for changing the stored balance data by the amount of the charge from an external balance change device and inputting the balance change information to the IC module;
Further including
The information processing method according to claim 1 . The processing unit performs a predetermined process on the data in response to a request from a device within a distance range in which short-range wireless communication is possible, and returns a processing result.
The information processing method further includes a setting step of setting a parameter used for the predetermined processing according to the signal output by the output step.
The information processing method according to claim 1 . The determining step determines, as a first portion of interest, a linear first edge of an object appearing in the reference image in a reference image constituting a series of images continuously captured by the imaging means; and Determining a linear second edge perpendicular to the first edge as a second portion of interest;
The detection step is a detection step of detecting an event indicating that the determined first target portion and second target portion have changed in any of a plurality of comparison images subsequent to the reference image, Indicates that the first target portion has been displaced from a first region including at least a part of the determined first target portion to a second region located in a direction perpendicular to the first target portion. The second target portion from the third region including at least a part of the first event and the determined second target portion to the fourth region located in the direction perpendicular to the second target portion. Each detecting a second event indicating that is displaced,
In the output step, when the first event and the second event are detected with a time difference within a predetermined threshold, a signal corresponding to a composite event of the first event and the second event is output. The information processing method according to any one of claims 1 to 3, wherein the information is output . The second region includes two regions located in opposite directions with respect to the first region,
The output step outputs the signal different depending on a previous region where the target portion is displaced among two regions included in the second region.
The information processing method according to claim 4 . At least a portion of the first edge fits in the first region, and a boundary between the first region and the second region is parallel to the first edge, and the second region The information processing apparatus, the object, and the object are arranged such that at least a part of an edge fits in the third area and a boundary between the third area and the fourth area is parallel to the second edge. A notification step of notifying the user of information prompting adjustment of the relative positional relationship of
In the reference image, at least a part of the first edge fits in the first area and at least a part of the second edge fits in the third area. And the second edge as the second portion of interest.
6. The information processing method according to claim 4 or 5 , wherein: At least a portion of the first edge fits in the first region, and a boundary between the first region and the second region is parallel to the first edge, and the second region The first to fourth regions so that at least a part of the edge fits in the third region and the boundary between the third region and the fourth region is parallel to the second edge. Further comprising an arrangement step of arranging each of
In the reference image, at least a part of the first edge fits in the first area and at least a part of the second edge fits in the third area. And the second edge as the second portion of interest.
6. The information processing method according to claim 4 or 5 , wherein: The output step includes a first signal corresponding only to the first event and a first signal corresponding only to the second event according to a time difference in detection between the first event and the second event. One of a second signal and a third signal corresponding to a composite event of the first event and the second event,
The information processing method according to any one of claims 4 to 7 . The output step outputs the third signal that differs depending on the detection order of the first event and the second event.
The information processing method according to claim 8 . It claims 4 to wherein the distance said vertical line direction between the region located perpendicular direction of the region and the portion of interest includes at least a portion of said target portion is set above the threshold the information processing method according to any one of 9. The imaging means continuously images at a predetermined imaging interval,
The detecting step detects the event indicating that the target portion is displaced such that an acceleration based on the predetermined imaging interval and a displacement amount of the target portion from a position in the reference image changes in a predetermined pattern. To
The information processing method according to any one of claims 4 to 10 , wherein: A determination unit that determines a target portion that can be identified with reference to an edge of an object appearing in the reference image in a reference image that constitutes a series of images continuously captured by the imaging unit;
Detecting means for detecting an event indicating that the determined portion of interest has changed in any of a plurality of comparison images subsequent to the reference image;
Output means for outputting a signal corresponding to the detected event or another event related to the event;
An information processing apparatus comprising:
The object includes an IC module having a storage unit for storing data, and a processing unit for accessing the data in response to a request from a device capable of communicating with the object,
The information processing apparatus performs predetermined processing based on a signal output from the output unit and data in the storage unit. A determination step of determining a target portion that can be identified with reference to an edge of an object appearing in the reference image in a reference image constituting a series of images continuously captured by the imaging means;
A detection step of detecting an event indicating that the determined portion of interest has changed in any of a plurality of comparison images subsequent to the reference image;
Outputting a signal corresponding to the detected event or other event related to the event;
An information processing program for causing a computer to execute
The object includes an IC module having a storage unit for storing data, and a processing unit for accessing the data in response to a request from a device capable of communicating with the object,
The information processing program causes the computer to perform predetermined processing based on the signal output in the output step and the data in the storage unit.

Images