JP2020091876A - Information processing device, information processing method, program, and information processing server - Google Patents

Abstract

To provide a program allowing ensuring that an image taken for identification is an image of an authorized identification card.SOLUTION: A program for making a computer function as each means of an information processing device. The information processing device includes: photographing means for taking an image using an imaging device of the information processing device; and control means for controlling the image acquisition for acquiring an image obtained by taking a first surface of a card, an image obtained by taking a side surface of the card, and an image obtained by taking a second surface of the card, from an image obtained by taking, with the photographing means, an operation in which a user turns the identification card over from the first surface to the second surface.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information processing device, an information processing method, a program, and an information processing server for acquiring information used for identity verification.

In recent years, it has become possible for service providers, etc. who need to verify their identities by taking an image of their ID, which shows the information displayed on their ID such as a driver's license, and a facial image of their face, using a smartphone or personal computer. The identity is confirmed by sending it.

In general, the identification image and face image provided are collated manually to confirm the identity. For this reason, for the purpose of reducing the work load for manually identifying the person, a technique has been proposed in which a robot is used to collate the image of the ID card with the face image of the owner of the ID card (Patent Reference 1).

JP, 2017-104947, A

By the way, when using an image of an identification card for identity verification, for example, the provided image of the identification image is not an image of different identifications on the front and back, or a photo of the identification image already imaged. It must be ensured that the image is not a taken image, but an authentic identification.

The present invention has been made in view of the above problems, and an object of the present invention is to realize a technique that more reliably ensures that an image captured for identity verification is an image of an authentic identification card.

In order to solve this problem, for example, the program of the present invention has the following configuration. That is,
A program for causing a computer to function as each unit of an information processing apparatus, the information processing apparatus comprising:
Photographing means for photographing an image using the image pickup device of the information processing device;
From the image obtained by the photographing means photographing the operation of the user turning over the identification card from the first side to the second side, the image of the first side of the card and the image of the card It is characterized by having a control means for controlling image acquisition so as to acquire an image in which a side surface is captured and an image in which the second surface of the card is captured.

According to the present invention, it is possible to more reliably ensure that the image captured for identity verification is an authentic identification image.

The figure which shows an example of the information processing system which concerns on embodiment of this invention. FIG. 3 is a block diagram showing a functional configuration example of a smartphone as an example of the information processing apparatus according to the first embodiment. Block diagram showing a functional configuration example of an information processing server according to the first embodiment The flowchart which shows operation|movement of the process for personal identification in the smart phone which concerns on Embodiment 1. The flowchart which shows operation|movement of the identification card photography process in the smart phone which concerns on Embodiment 1. FIG. 6 is a diagram illustrating an example of a user interface displayed when the front surface of the identification card is captured in the identification card capturing process. FIG. 4 is a diagram illustrating an example of a user interface displayed after an image of the front surface of the identification card is acquired in the identification image capturing process. FIG. 4 is a diagram illustrating an example of a user interface displayed after an image of the side surface of the identification card is acquired in the identification image capturing process. The flowchart which shows operation|movement of the confirmation process of the imaging|photography fact in the smart phone which concerns on Embodiment 1. The figure explaining the example of the user interface displayed in the confirmation process of a shooting fact. The flowchart which shows operation|movement of an identity verification process in the smart phone which concerns on Embodiment 2. The flowchart which shows the example of the user interface displayed in the face image photographing process which concerns on Embodiment 2. Flowchart showing the operation of face image shooting processing in the smartphone according to the second embodiment

(Embodiment 1)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the following description, the same or substantially the same configurations will be described with the same reference numerals. Further, as an example of the information processing apparatus, an example using a smartphone capable of capturing an image will be described. However, the present embodiment is applicable not only to smartphones but also to other devices capable of capturing images. These devices may include, for example, personal computers, game consoles, tablet terminals, and the like.

<Structure of information processing system>
The configuration of the information processing system 10 according to the present embodiment will be described with reference to FIG. The information processing system 10 is a system for identifying a person using a captured image. The image of the identification card and the face image taken by the smartphone 100 are transmitted to the information processing server 110 via the network 111. The ID card is a card that can be used for identity verification, and includes, for example, a license. In the example of the present embodiment, a case where a license is handled as an example of the ID card will be described, but the ID card may include a residence card or an insurance card.

In the information processing server 110, the identity verification result is set based on the image received from the smartphone 100 (by an operation of an operator or the like who performs identity verification). The information processing server 110 transmits the personal identification result to the smartphone 100.

The smartphone 100 includes a camera, and captures and acquires an image of a user's identification card and a user's face image on an application for personal identification. The smartphone 100 first captures a user's face and acquires a user's face image, for example. Then, in order to further ensure the authenticity of the ID card, an ID image capturing process described below is performed to acquire an image of the ID card. In addition, the smartphone 100 further performs a process of confirming a shooting fact, which will be described later, to ensure that the image of the identification card is actually taken. The smartphone 100 transmits the face image of the user and the images acquired by these processes to the information processing server 110, and as a result, receives the identity verification result from the information processing server 110. In the present embodiment, acquisition of the user's face image is not essential. When the smartphone 100 does not capture the face image of the user, the smartphone 100 may transmit the image acquired by the identification image capturing process or the capturing fact confirmation process to the information processing server 110.

The network 111 includes a communication network such as the Internet or a mobile phone network, for example, and transmits information between the smartphone 100 and the information processing server 110.

<Structure of smartphone>
Next, a configuration example of the smartphone 100 will be described with reference to FIG. The configuration illustrated in FIG. 2 illustrates functional blocks that configure a smartphone as an example of the information processing device according to the present embodiment. Note that each of the functional blocks described may be integrated or separated, and the functions described may be implemented by another block. Also, what is described as hardware may be realized by software, and vice versa.

The communication unit 201 includes, for example, a communication circuit, and transmits/receives necessary data by connecting to the Internet via mobile communication such as LTE or connecting to a network via wireless LAN communication.

The control unit 202 includes a CPU 210 and a RAM 211. For example, the program recorded in the recording unit 207 is expanded in the RAM 211, and the CPU 210 executes the program to control the operation of each unit in the smartphone 100. Further, the control unit 202 executes an identification card photographing process, a photographing fact confirmation process, and a face image photographing which will be described later. At this time, the control unit 202 functions as an image capturing unit that captures an image using an image capturing device, and a control unit that controls image acquisition that acquires a specific image to be described later from the image obtained by the image capturing unit. It also functions as each unit such as a display control unit for controlling the display on the display unit 206.

The operation unit 203 includes buttons and a touch panel included in the smartphone 100, and can perform operations on the GUI for various operations displayed on the display unit 206. The sensor unit 204 includes a sensor for biometric authentication (fingerprint, face, etc.) for determining that the user of the smartphone 100 is a genuine user. The image pickup device 205 is, for example, a camera mechanism including an image pickup element, and picks up an image of a subject according to an instruction from the control unit 202. The imaging device 205 can continuously capture images at a predetermined rate and output them to the control unit 202. The imaging device 205 is separately provided on the surface (front side) on which the display unit 206 of the smartphone 100 is provided and the opposite surface (back side). For example, in the identification card photographing process and the photographing fact confirmation process, which will be described later, the back side image pickup device 205 is used to photograph an image. On the other hand, when capturing a face image of a user, which will be described later, the front-side image capturing device 205 is used to capture the face image. Within a predetermined time after shooting (for example, immediately after shooting), the smartphone 100 sends the hash value of the image/moving image in the background to the information processing server 110, and the hash value of the sent image/moving image is sent on the server side. By collating, it is guaranteed that the image was taken with the smartphone at that time. In addition, it is possible to guarantee that the image/moving image sent has not been processed by the user.

The display unit 206 includes a display device such as an LCD or an OLED. In response to an instruction from the control unit 202, the display unit 206 displays a GUI relating to an ID photographing process and a photographing fact confirmation process, which will be described later, GUIs of various applications, and the like.

The recording unit 207 includes, for example, a non-volatile memory such as a semiconductor memory, and holds an image photographed/acquired for identity verification, or holds a program executed by the control unit 202 or a set value. The program stored in the recording unit 207 includes an operating system and various applications for realizing various functions of the smartphone 100, and the applications include an application for performing personal identification processing described later. The recording unit 207 may record the image photographed/acquired for personal identification and the information on the personal identification result in, for example, a distributed ledger used in the block chain technology. In this case, the smartphone 100 may function as one node that manages the distributed ledger, or may only transmit data to the node that manages the distributed ledger.

The voice input device 208 includes a microphone, for example, and inputs a voice uttered by a user who uses the smartphone 100. The voice input device 208 may have a function of not only inputting the voice uttered by the user but also recognizing the voice uttered by the user and outputting numerical values and text information. Of course, the voice input by the voice input device 208 may be recognized by the control unit 202 and treated as numerical values or text information.

<Configuration of information processing server>
Next, a functional configuration example of the information processing server 110 will be described with reference to FIG. Note that each of the functional blocks described may be integrated or separated, and the functions described may be implemented by another block. Also, what is described as hardware may be realized by software, and vice versa.

The communication unit 301 includes a communication circuit that communicates with the plurality of smartphones 100 via the network 111. The information processed by the control unit 302 is transmitted to the smartphone 100 or the like, or the information processed by the control unit 302 is received from the smartphone 100 or the like.

The control unit 302 includes a CPU 310, which is a central processing unit, and a RAM 311. The control unit 302 controls the operation of each unit of the information processing server 110 by expanding and executing the program stored in the recording unit 304 in the RAM 311.

The RAM 311 includes a volatile storage medium such as a DRAM, and temporarily stores parameters for the control unit 302 to execute the program, processing results, and the like. The recording unit 304 includes a non-volatile recording medium such as a semiconductor memory, and stores setting values and programs necessary for the operation of the information processing server 110. In addition, the recording unit 304 holds the user's face image or identification image received from each of the smartphones 100 in the database for each identifier that is given each time personal identification is performed. This database holds the identity verification result for each user. When the personal identification result for each user is set, the control unit 302 transmits the personal identification result to the smartphone 100 of the corresponding user. Further, when the control unit 302 receives an inquiry about the identification result of a specific user from a device of a financial institution or a service providing company (not shown), the control unit 302 may return the stored identification result.

The operation unit 303 is an operation mechanism capable of operating the information processing server 110 directly or remotely, and includes a configuration in which an operation instruction is received from an external device via communication in addition to a mouse and a keyboard. For example, an operator who identifies himself using an image or an administrator of the information processing server performs a necessary operation using the operation unit 303. The display unit 305 is a display mechanism that can directly or remotely display information that the information processing server 110 has and information that is output using a user interface. The display unit 305 may be a display device connected to the information processing server 110, and also includes a configuration for outputting display information when it is displayed by an external device via communication.

<Operation of identity verification processing on smartphone>
Next, the operation of the personal identification processing on the smartphone 100 will be described with reference to FIG. It should be noted that this processing is realized by the CPU 210 of the control unit 202 executing a program (including a personal identification application) recorded in the recording unit 207. For example, this processing is performed when the control unit 202 activates the identification application by a user operation on the operation unit 203 and further determines that the user is valid by biometric authentication and digital signature using the sensor unit 204. Be started. That is, the personal identification processing is started in a state in which it is confirmed that the user who uses the personal identification application is the genuine user of the smartphone 100.

In S401, the control unit 202 executes face image shooting processing. For example, the control unit 202 continuously captures the face image of the user using the front-side imaging device 205, and displays the image output from the imaging device 205 on the display unit 206. The control unit 202 acquires a face image and stores it in the recording unit 207 in response to an operation instruction from the user or when the control unit 202 determines that the user's face is located at a specific position. Record. The image to be recorded may be one still image or may be composed of a plurality of images.

In S402, the control unit 202 executes an identification card photographing process. The ID card capturing process is performed by capturing a series of operations in which the user turns the ID card in his/her hand from inside to outside (or vice versa), and then the images of the front and back of the ID card are captured. An image (side image) showing the thickness of the identification card is acquired together with the image. Details of the ID photographing process will be described later.

By acquiring images of each side of the ID card from the images of actions that rotate the ID card, it is possible to prove that the acquired image is an image of one authentic ID card. can do. By using this ID photographing process, for example, when images of different IDs are provided on the front and the back, or an image of the ID that is already imaged without the ID is captured and provided. Such cases can be excluded. That is, it is possible to confirm that the "entity" of the identification card is there, and it is possible to further ensure that the image taken for identity verification is the authentic identification image.

In the ID shooting process, a series of actions were taken in addition to the image of the front side of the ID, the side of the ID, and the back of the ID. A moving image may be acquired. At this time, the compressed moving image may be obtained by taking the difference data of the adjacent images. By doing so, it is possible to reduce the amount of data held or transmitted in the recording medium for the identification of the person.

In S403, the control unit 202 determines whether the identification card photographing process has been completed successfully. For example, the control unit 202 successfully ends the ID image capturing process when the ID image capturing process is stopped in the middle of image capturing (for example, the process end flag is not set to a value indicating that the process is successful). If it is determined that there is not, this processing ends. If not, the process proceeds to S404.

In step S404, the control unit 202 executes a process of confirming the shooting fact. According to the instruction displayed on the display unit 206, the execution process of the shooting facts is performed by taking images of a user continuously moving the ID card up, down, left, and right, from a predetermined portion of the ID card (for example, a face photograph or an address field). , Etc.) to acquire multiple images taken at different positions. The details of the process of confirming the shooting fact will be described later. Instructions are changed so that the order or position of movement is different each time processing is performed, so if the user can move the ID card according to the instruction, confirm that the user is actually taking the ID card. You can That is, the authenticity of the ID card can be ensured by performing the ID card image capturing process and the image capturing fact confirmation process executed in S402.

In the process of confirming the shooting facts, a moving image obtained by shooting a series of actions may be acquired in addition to the image taken when the position of the identification card is moved to a predetermined position. Also in this case, a compressed moving image obtained by taking difference data between adjacent images may be used as in the identification image capturing process. By doing so, it is possible to reduce the amount of data held or transmitted in the recording medium for the identification of the person. In the present embodiment, the case where an image is acquired in the process of confirming the shooting fact will be described as an example, but the operation may be performed only by confirming the position of the ID card moved by the user. In this case, in S406 to be described later, the information processing server may be notified that the confirmation has been completed successfully.

In step S405, the control unit 202 determines whether or not the shooting fact confirmation process has been completed successfully. For example, the control unit 202 succeeds in the process of confirming the shooting facts when the process of confirming the shooting facts is stopped during the shooting (for example, the processing end flag is not set to a value indicating processing success). It is determined that the processing has not ended, and this processing ends. If not, the process proceeds to S406.

In step S<b>406, the control unit 202 has successfully completed both the identification card photographing process and the photographing fact confirmation process. Send to. When the compressed moving image is acquired in addition to the plurality of images in the ID photographing process and the photographing fact confirmation process, each compressed moving image is also transmitted to the information processing server 110. In addition, since biometric authentication and a digital signature are used to confirm that the user is an authentic user of the smartphone at the start of this process, image provision (spoofing) by another person is prevented. That is, by the identity verification process of the present embodiment, the authentic user transmits the information of the identification card having high authenticity.

In step S407, the control unit 202 receives the identity verification result from the information processing server 110. In step S<b>408, the control unit 202 causes the display unit 206 to display the personal identification result in response to receiving the personal identification result from the information processing server 110. As a result, the user can confirm the result of identity verification authenticated on the server side. The control unit 202 thereafter ends the operation of the identity verification processing.

<Behavior of photo ID processing on smartphone>
Next, the operation of the identification card photographing process in the smartphone 100 will be described with reference to FIG. In step S501, the control unit 202 displays a shooting start screen. The shooting start screen is displayed, for example, as a GUI as shown in FIG. In FIG. 6A, the display unit 206 of the smartphone 100 displays some guides for photographing the front surface of the identification card. The card frame 601 is a guide that indicates the position where the front surface of the identification card should be placed in the captured image. The card frame 601 indicates the orientation (vertical orientation) of the ID card when the user takes the ID card. By setting the orientation of the ID card to portrait orientation (vertical orientation), it is possible to reduce the possibility of hiding the ID card information with your finger when shooting a series of actions that rotate the ID card in your hand. .. The information frame 602 represents the arrangement of information on the front side of the identification card (license in the example of this embodiment), and the user can identify which side of the identification card is to be photographed. Is composed of. The text guide 604 is, for example, a text indicating how to hold the ID card and the side to be photographed, “Let's put the license on the palm and copy the front side”. The figure guide 603 is a guide for intuitively understanding the state of holding the identification card. The start button 605 is an instruction member that starts shooting when pressed. The uncaptured thumbnail 607 indicates that the images captured on each side of the identification card have not been acquired yet. As will be described later, when the image of each face of the identification card is acquired, the uncaptured thumbnail 607 is replaced with the thumbnail of the acquired image.

In step S502, the control unit 202 determines whether a shooting start instruction has been received. For example, when the start button 605 on the shooting start screen is pressed by the user, the control unit 202 determines that the shooting start instruction has been received, and proceeds to step S503. At this time, the control unit 202 instructs the imaging device 205 to continuously output captured images at a predetermined frame rate (for example, 30 frames per second or lower and the number of frames that can be determined in S503). In addition, the processing end flag is initialized to a value indicating processing failure. When the control unit 202 does not determine that the instruction to start shooting is received, the control unit 202 returns to S502 again. When advancing to S503, the control unit 202 instructs the imaging device 205 to continuously output captured images at predetermined time intervals. In addition, the processing end flag is initialized to a value indicating processing failure.

In step S<b>503, the control unit 202 determines whether the captured image includes an image that does not include at least a part of the outline of the identification card. Specifically, the control unit 202 determines whether the ID card in the photographed image is outside the predetermined area (for example, outside the frame). If the identification card goes out of the frame of the image being captured while capturing each surface, it cannot be guaranteed that the same identification document is captured. That is, when it is determined that the captured image includes an image that does not include at least a part of the outline of the ID card, the image acquisition is stopped, so that the ID card can be prevented from being replaced. If the control unit 202 determines that there is an image that does not include at least a part of the outline of the identification card, the control unit 202 ends this processing, and if not, the process proceeds to S504.

In step S<b>504, the control unit 202 detects that the face to be processed of the ID (either the front face, the side face, or the back face) is detected, and the image captured at that timing is changed to the process target face. Acquire as a captured image. The control unit 202 detects and determines the presence of characteristic information (for example, a face photograph) on each side of the identification card, the arrangement of the information, the shape of the identification card (for example, the shape of the side surface), and the like in the captured image. The surface to be processed is detected.

FIG. 6B shows an example of a user interface whose display is controlled by the control unit 202 when an image of the front surface of the identification card is acquired. On the display unit 206, a guide display of the card frame 601 and the like is displayed together with an image of the hand 611 of the user and the front surface 612 of the identification card taken. Also, a shooting button 613 indicating that shooting is in progress is displayed. Since the front surface of the identification card is detected (at the position of the card frame 601) in the image being captured, a check mark 614 indicating that the identification surface has been detected is displayed. The thumbnail 615 is a thumbnail of the image of the front side of the identification card, and is replaced with the uncaptured thumbnail 607.

In step S505, the control unit 202 determines whether to acquire an image of the next surface. For example, when there is another surface for which an image should be acquired, the process proceeds to S506. Note that the control unit 202 changes the processing end flag to a value indicating that the processing is successful when ending the ID photographing processing.

In step S506, the control unit 202 causes the display unit 206 to display a guide for photographing the next face of the identification card. When the guide for imaging the next surface is displayed on the display unit 206, the control unit 202 proceeds to S503 again to shoot the next surface.

In FIG. 7A, the control unit 202 captures a guide display for capturing the side surface of the identification card in response to the image of the front surface of the identification card being acquired (S504). The image is displayed together with the displayed image (S506). For example, a side guide 701 of the identification card is displayed together with an image of the front surface 612 of the identification card being photographed. The text guide can be switched to the text guide 702 indicating the face (that is, the side face) of the next identification card or the timing of the action, such as “Let's clearly copy the side face and stop for about 1 second”. By displaying a guide display or a text guide for photographing the next face of the identification card, the user can perform an operation of turning over the face of the next identification card without making a mistake. At the same time, it is possible to properly grasp the timing and speed of turning over the ID card.

Further, FIG. 7B shows an example of a user interface whose display is controlled by the control unit 202 when the image of the side surface of the identification card is acquired (S504). A display guide such as a side guide 701 is displayed on the display unit 206 together with an image of the side 703 of the identification card. Since the side face of the ID card is detected in the image being taken, a check mark 704 indicating that the ID is detected is displayed. The thumbnail 705 is a thumbnail of an image of the side surface of the identification card, and is replaced with the uncaptured thumbnail 607.

In addition, in FIG. 8A, in response to the acquisition of the image of the side surface of the identification card (S504), the control unit 202 captures a guide display for capturing the back surface of the identification card. The image is displayed together with the displayed image (S506). In this example, the backside guide 801 of the identification card is displayed together with the image of the side surface 703 of the identification card. The text guide can be switched to the text guide 802 indicating the back side to be photographed next, for example, “Let's rotate and copy the back side”.

Further, FIG. 8B shows an example of a user interface whose display is controlled by the control unit 202 when an image of the back surface of the identification card is acquired. A display guide such as the back guide 801 is displayed on the display unit 206 together with an image of the back 803 of the identification card. Since the back side of the ID card is detected in the image being taken, a check mark 805 indicating that the ID has been detected is displayed. The thumbnail 804 is a thumbnail of the image of the back side of the identification card, and is replaced with the uncaptured thumbnail 607.

In S505, when the images of all the surfaces of the identification card have been acquired, it is not necessary to acquire the images of the next surface. Therefore, the control unit 202 thereafter ends the image capturing by the imaging device 205, and the identification card. The shooting process ends.

<Operation of confirmation process of shooting fact on smartphone>
Next, the operation of the confirmation process of the shooting fact in the smartphone 100 will be described with reference to FIG. In step S<b>901, the control unit 202 determines a moving order for moving a predetermined part (an image of a face in this embodiment) of the identification card up, down, left and right. For example, a combination of a plurality of movement orders recorded in advance in the recording unit 207 is read out and the randomly selected movement order is determined. Alternatively, the moving order may be acquired from the information processing server 110. In step S902, the control unit 202 determines whether there is a shooting start instruction. For example, after the movement confirmation screen shown in FIG. 10A is displayed on the display unit 206, it is determined whether a start button (not shown) has been pressed by a user operation. If it is determined that there is an instruction to start shooting, the process proceeds to S903, and if not, the process returns to S902.

In step S903, the control unit 202 determines whether a face photograph has been detected at the target position. For example, the control unit 202 displays the order of movement 1 to 4 in the card part detection frame of the movement confirmation screen shown in FIG. Further, the control unit 202 displays the active card part detection frame 1001 in which the card part detection frame having the movement order 1 is active-displayed. In the example of FIG. 10A, the user moves the identification card 1004 so that a predetermined part (for example, a face photograph) of the identification card 1004 overlaps with the active card part detection frame 1001. Further, the control unit 202 displays on the display unit 206 an image obtained by photographing the operation of moving the identification card by the user. The text guide 1003 indicates that the user moves the identification card between the card part detection frame 1001 and the card part detection frame 1002 in numerical order. The control unit 202 proceeds to S904 when a face photograph is detected at the position of the active card part detection frame 1001 in the image to be captured, and otherwise continues the processing of S903.

In step S904, the control unit 202 acquires an image of the action of the user moving the identification card to the active card part detection frame 1001 and displays a check mark 1005 shown in FIG. 10B. When the user moves the identification card and a face photograph is detected in the plurality of card part detection frames, the control unit 202 further displays a line connecting the check marks on the display unit 206.

In step S905, the control unit 202 determines whether the current target position is the last target position. For example, if all the card part detection frames have been detected (that is, a check mark is displayed), the process proceeds to S907. If not, the process advances to step S906 to change the processing target to the next target position. At this time, the next target position is changed according to the moving order determined in S901, and the display of the active card part detection frame is changed according to the moving order. The control unit 202 then returns to the process of S903.

In step S907, the control unit 202 determines that the shooting fact has been confirmed. Specifically, the processing end flag in the processing of confirming the shooting fact is changed to a value indicating the processing success. After that, the control unit 202 ends the shooting by the image pickup device 205 and ends the series of operations.

In the above example, the case where the front side of the identification card is first captured and then the side surface and the back surface are sequentially captured in the identification image capturing process has been described. However, the back side of the ID card may be photographed first, and then the side surface and the front surface may be photographed in order. Further, in the identification card photographing processing, if there is an image in which at least a part of the outer shape of the identification card is not included in the images obtained by photographing the turning over operation, the processing is stopped. However, the processing may be stopped when a predetermined proportion of the outer shape of the identification card is not included.

As described above, in the present embodiment, an image obtained by photographing the front side of the identification card and the identification card from the image obtained by photographing the user's operation of turning over the identification card (for example, from the front side to the back side). The image acquisition was controlled so as to acquire the image in which the side surface was captured and the image in which the back surface of the identification card was captured. By doing so, it is possible to prove that the acquired image is an image of one authentic identification card, and the image captured for identity verification is the image of the authentic identification card. Can be more certain.

(Embodiment 2)
Next, a second embodiment will be described. In the second embodiment, the face image capturing process in the personal identification process is different from that in the first embodiment. In the face image capturing process of the present embodiment, the control unit 202 causes the display unit 206 to display an instruction that the user should respond while the face of the user is being captured. Then, the control unit 202 generates the user's face image from the plurality of captured images of the user's face in response to the user's response to the instruction to respond corresponding to the response corresponding to the instruction to respond. get. Although part of the personal identification processing in the second embodiment is different from that in the first embodiment, the configurations of the smartphone 100 and the information processing server 110 of the second embodiment are substantially the same as the configurations of the first embodiment. For this reason, substantially the same configurations are denoted by the same reference numerals, duplicate description is omitted, and different points are mainly described.

<Identity confirmation processing on a smartphone>
The operation of the personal identification processing on the smartphone 100 will be described with reference to FIG. Note that this processing is realized by the CPU 210 of the control unit 202 executing a program (including a personal identification application) recorded in the recording unit 207, as in the first embodiment. For example, this processing is started when the control unit 202 activates the identification application by a user operation on the operation unit 203 and further determines that the user is valid by biometric authentication using the sensor unit 204. .. That is, the control unit 202 is started after confirming that the user who uses the personal identification application is the genuine user of the smartphone 100.

In step S1101, the control unit 202 executes the face image capturing process of this embodiment. Details of the face image capturing process will be described with reference to FIG. In step S1102, the control unit 202 determines whether the face image capturing process has been successfully completed. The control unit 202 proceeds to S402 if the processing end flag in the face image capturing processing has a value indicating processing success, and ends the person confirmation processing if not. After that, the control unit 202 executes S402 to 408 and ends the series of operations as in the first embodiment.

<Operation of face image shooting process on smartphone>
Next, the operation of the face image capturing process will be described with reference to FIG. In step S1301, the control unit 202 displays a guide for face position on the display unit 206. For example, the control unit 202 displays the face image shooting screen shown in FIG. 12A on the display unit 206, and then displays the face position guide 1201 on the face image shooting screen. On the face image capturing screen, an image in which the user's face is captured is displayed together with the guide 1201. Therefore, the face position guide 1201 indicates the position where the user's face should be arranged in the captured image.

In step S1302, the control unit 202 determines whether there is an instruction to start shooting. For example, it is determined whether or not a start button (not shown) is pressed by a user operation on the face image capturing screen shown in FIG. If it is determined that there is an instruction to start shooting, the process proceeds to S1303, and if not, the process returns to S1302.

In step S<b>1303, the control unit 202 acquires information on an instruction that the user should respond to and causes the display unit 206 to display the information. For example, the combination of the instruction information recorded in the recording unit 207 in advance is acquired, and the randomly selected instruction information is determined. Alternatively, information on an instruction that the user should respond may be acquired from the information processing server 110. The control unit 202 causes the display unit 206 to display information on the determined instruction and the text guide 1203, for example, the instruction 1202 illustrated in FIG. The text guide 1203 is a text that tells the user that he or she should respond to the instruction, such as "Let's read the short sentence aloud." The instruction to be responded by the user is not limited to the short text and may be a numerical value with a predetermined number of digits. Alternatively, for example, like the instruction 1204 in FIG. 12B, it may indicate the orientation of the face that the user should change in response to the instruction. In this case, a text guide 1205 such as "Let's face in the direction of the arrow" is displayed. Further, the control unit 202 may perform both the reading and the change of the face direction. In this way, by confirming the user's reading of the random display, it is possible to ensure that the user is actually shooting. On the other hand, by photographing the orientation of the user's face, it is possible to ensure that the subject being photographed (the user's face) is alive (also referred to as Liveness Detection). By combining both reading and changing the orientation of the face, it is possible to confirm the characteristics of both and to generate a more reliable face image.

In S1304, the control unit 202 determines whether or not there is a correct response from the user. For example, the control unit 202 recognizes the user's voice input via the voice input device 208, and determines whether or not the (user's reading) instruction corresponds to the corresponding response. Alternatively, the control unit 202 recognizes the orientation of the user's face in the captured image, and determines whether the recognized orientation of the face matches the response corresponding to the instruction to respond. If the instruction to respond and the answer match, it is determined that there is a correct response, and the process proceeds to S1305. If not, the process returns to S1303, for example.

In step S1305, the control unit 202 displays a check mark or the like and returns feedback to the user's response. In step S1306, the control unit 202 determines whether the displayed instruction is the last instruction. If the instruction is the last instruction, the process proceeds to step S1307, and if not, the process returns to step S1303. That is, in the case of further confirming the reading of the user after confirming the change of the face direction, or in the case of further confirming the second reading after confirming the first reading, the process returns to S1303.

In step S<b>1307, the control unit 202 determines that one or more instructions to be responded and the response of the user corresponding to the instructions match, and extracts the user's face image from the captured image of the user's face. get. The control unit 202 records the image of the user's face in the recording unit 207, and ends the face image capturing process.

As described above, in this embodiment, the instruction to respond should be displayed while the user's face image is captured by the imaging device 205. Then, when the response of the user to the instruction matches the answer corresponding to the instruction to be responded, the face image of the user is acquired from the image of the face of the user captured by the imaging device 205. .. By doing so, the user can confirm that the face image is actually acquired on the spot. In addition, it is possible to guarantee that the photographed object is a living person. That is, the authenticity of the transmitted face image can be guaranteed.

Claims (1)

A program for causing a computer to function as each unit of an information processing apparatus, the information processing apparatus comprising:
Photographing means for photographing an image using the image pickup device of the information processing device;
From the image obtained by the photographing means photographing the operation of the user turning over the identification card from the first side to the second side, the image of the first side of the card and the image of the card A program, comprising: a control unit that controls image acquisition so as to acquire an image of a side surface of the card and an image of the second surface of the card.

Images