JP2020095728A - Portable terminal, identification system, and program - Google Patents

Abstract

To provide a portable terminal, an identification system, and a program for strictly performing identification in on-line transaction.SOLUTION: A portable terminal 1 has a camera 33 and performs identification of a user at the start of a service by using only a portable terminal 1 owned by the user, and comprises: a guide screen output unit 14 that outputs a guide screen for designating the arrangement position of a coin 9 that is photographed together with a driver's license 3 with a photograph of the face printed thereon; an identification image acquisition unit 15 that acquires, from the camera 33, an identification image that is an image including the coin 9 and the driver's license 3 according to the output guide screen; a collation image acquisition unit 19 that acquires, from the camera, a collation image that is an image including the face of an owner of the driver's license 3; and an image transmission unit 20 that transmits the identification image acquired from the identification image acquisition unit 15 and the collation image acquired from the collation image acquisition unit 19, to an identification server 4 that performs identification by using the images.SELECTED DRAWING: Figure 1

Description

The present invention relates to a mobile terminal, a personal identification system, and a program used for personal identification processing.

Conventionally, for example, when a credit card is delivered from a credit company to a sending customer, it is necessary to strictly confirm the identity. As a system for that purpose, a system for efficiently performing personal identification and collection/delivery work has been disclosed (for example, Patent Document 1). According to Japanese Patent Laid-Open No. 2004-242242, when delivering a packaged product in which a credit card is enclosed to a recipient, the package is delivered after confirming that the recipient is the recipient. Specifically, as a procedure for identity verification, first, the driver in charge of delivery and delivery is face-to-face with the recipient and confirms the identity of the items on the label of the delivery and delivery and the license etc. presented by the recipient. Visually check the name and address of the document. Then, enter the recipient's date of birth, the document type data, and the symbol number data described in the personal identification document in the personal identification document into the terminal possessed by the driver in charge of delivery, and read the label. Match with the obtained data. If the verification result is a match, the package is delivered to the recipient.

JP, 2009-237823, A

For example, as means for confirming the identity in opening a bank account, etc., conventionally, there is no need for forwarding mail or limited to the person who performs face-to-face identity verification as described in Patent Document 1 at the home of the applicant applying for the account opening. The identity is confirmed by sending mail. Recently, financial institutions are considering ways to complete online identity verification. Under such circumstances, non-face-to-face personal identification is required as a method of personal identification. However, even if it is a non-face-to-face type, it is necessary to strictly verify the identity, as in the face-to-face type.

Therefore, it is an object of the present invention to provide a mobile terminal, an identity verification system, and a program for strictly performing identity verification in online transactions.

The present invention solves the above problems by the following means. It should be noted that, for ease of understanding, reference numerals corresponding to the embodiments of the present invention will be given and described, but the present invention is not limited thereto. Further, the configurations described with reference numerals may be appropriately improved, or at least a part of them may be replaced with other components.

A first aspect of the present invention is a mobile terminal (1) having a photographing unit (33), wherein the identity of the user at the start of service is confirmed only by the mobile terminal possessed by the user. A guide screen output means (14) for outputting a guide screen designating the arrangement position of the object (9) to be photographed together with the posted identity verification document (3) and the guide screen output by the guide screen output means. Therefore, the personal identification image acquisition means (15) for acquiring the personal identification image, which is an image including the object and the personal identification document, from the photographing unit, and the face of the holder who possesses the personal identification document. A collation image acquisition unit (19) that acquires a collation image, which is an image, from the photographing unit, the personal identification image acquired by the personal identification image acquisition unit, and the collation image acquired by the collation image acquisition unit. And a collation image transmitting means (20) for transmitting to an identity verification server (4) for performing identity verification using an image.
A second aspect of the present invention is, in the portable terminal (1) of the first aspect, a repetitive processing means (the processing by the guide screen output means (14) and the processing by the personal identification image acquisition means (15) are repeated a plurality of times ( 13), the guide screen output means outputs the guide screen in which the arrangement position of the target object is different each time, and the personal identification image acquisition means includes a plurality of the personal identification documents obtained by photographing the personal identification document from different photographing directions. The personal identification image is acquired from the photographing unit (33), which is a mobile terminal.
A third invention is the portable terminal (1) according to the first invention or the second invention, wherein the guide screen output means (14) randomly determines the target object from a plurality of predetermined positions. The mobile terminal is characterized in that the guide screen including the arrangement position of (9) is output.
A fourth invention is the portable terminal (1) according to the first invention or the second invention, wherein the guide screen output means (14) randomly determines in a predetermined designated area of the guide screen. The mobile terminal is characterized in that the guide screen including the arrangement position of the object (9) is output.
A fifth invention is the portable terminal (1) according to any one of the first invention to the fourth invention, wherein the guide screen output means (14) confirms the identity together with the arrangement position of the object (9). The mobile terminal is characterized in that the guide screen designating the arrangement position of the document (3) is output.
A sixth invention is the portable terminal (1) according to any one of the first to fifth inventions, wherein the guide screen output means (14) displays the guide screen showing the arrangement position as a frame line. The mobile terminal is characterized by outputting.
A seventh invention is the portable terminal (1) according to any one of the first invention to the sixth invention, wherein the guide screen output means (14) sets the arrangement position of the object (9) to a circular shape. The mobile terminal is characterized in that the displayed guidance screen is output and the object includes a coin.
An eighth invention is the portable terminal (1) according to any one of the first invention to the seventh invention, wherein the object is based on the identity verification image acquired by the identity verification image acquisition means (15). (9) and the legitimacy confirmation means (16) for confirming the legitimacy of the personal identification document (3), and the collation image acquisition means (19) can confirm the legitimacy by the legitimacy confirmation means. In this case, the collation image is acquired, which is a portable terminal.
A ninth invention is, in the portable terminal (1) according to any one of the first invention to the eighth invention, blink detection for detecting a blink from an image of the face of the holder acquired from the photographing section (33). A portable terminal comprising means (18), wherein the collation image acquisition means (19) acquires the collation image photographed by the photographing unit after the blink detection means detects a blink.
A tenth aspect of the present invention comprises a personal identification (1) including a mobile terminal (1) having a photographing unit (33), and a personal identification server (4) communicably connected to the mobile terminal and performing an identification process. In the system (100), the mobile terminal confirms the identity of the user at the start of the service only with the mobile terminal possessed by the user, and an identity verification document (3) in which a face photograph is posted. According to the guide screen output means (14) for outputting a guide screen designating the arrangement position of the object (9) to be photographed together, and the guide screen output by the guide screen output means, the object and the personal identification are confirmed. The personal identification image acquiring means (15) for acquiring the personal identification image, which is an image including a document, from the photographing unit, and the collation image, which is an image including the face of the holder who possesses the personal identification document, are photographed. Collation image acquisition means (19) acquired from the department, the identity verification image acquired by the identity verification image acquisition means, and the verification image acquired by the verification image acquisition means are transmitted to the identity verification server. Image sending means (20), the personal identification server collating the collation image received from the mobile terminal with a facial photograph image included in the personal identification image, and a face collating means (43). Result transmitting means (44) for transmitting matching result information by the face matching means to the mobile terminal, the mobile terminal based on the received matching result information, the mobile terminal and the identity verification server. A service start determination means (22) for determining whether to start service processing with a service providing server (7) that is communicably connected to the server and performs service processing. System.
An eleventh aspect of the present invention includes a computer (1) possessed by the user, which has a photographing unit (33) and is used to confirm the identity of the user at the time of service start, together with an identity verification document (3) in which a facial photograph is posted. Includes a guide screen output means for outputting a guide screen designating the arrangement position of the object (9) to be photographed, and the object and the personal identification document according to the guide screen output by the guide screen output means. Identity verification image acquisition means for acquiring the identity verification image, which is an image, from the imaging unit, and verification image acquisition that acquires a verification image, which is an image including the face of the holder who possesses the identity verification document, from the imaging unit Collation for transmitting a personal identification means, the personal identification image acquired by the personal identification image acquisition means, and the collation image acquired by the collation image acquisition means to a personal identification server (4) for performing personal identification using images. It is a program (31b) for functioning as an image transmission means.

According to the present invention, it is possible to provide a mobile terminal, a personal identification system and a program for strictly performing personal identification in online transactions.

It is a figure showing the whole personal identification system composition concerning this embodiment. It is a functional block diagram of the portable terminal which concerns on this embodiment. It is a functional block diagram of an identity verification server concerning this embodiment. It is a flow chart which shows service start processing in the personal digital assistant concerning this embodiment. It is a flow chart which shows the identity verification processing in the personal digital assistant concerning this embodiment. It is a flow chart which shows personal identification image processing in a personal digital assistant concerning this embodiment. It is a figure which shows the example of a pattern of the guidance screen which concerns on this embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on this embodiment. It is a flow chart which shows legitimacy confirmation processing in a personal digital assistant concerning this embodiment. It is a flowchart which shows the image analysis process in the identity verification system which concerns on this embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on this embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on this embodiment. It is a flow chart which shows the collation image processing in the personal identification system concerning this embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on this embodiment. It is a figure which shows the example of a pattern of the guidance screen which concerns on a modification. It is a figure which shows the example of a display mode of the guidance screen which concerns on a modification.

Hereinafter, modes for carrying out the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited to this.
(Embodiment)
FIG. 1 is a diagram showing an overall configuration of an identity verification system 100 according to the present embodiment.
FIG. 2 is a functional block diagram of the mobile terminal 1 according to the present embodiment.
FIG. 3 is a functional block diagram of the identity verification server 4 according to the present embodiment.
The personal identification system 100 shown in FIG. 1 is a system for performing personal identification required only when the first service of various services is started, using only the mobile terminal 1. The personal identification system 100 is a system that performs personal authentication required only when the service is started for the second time and thereafter only by the mobile terminal 1.

Here, various services are, for example, services related to opening an account in a bank, various banking services, car sharing services, renting real estate, and the like, and refer to various services that require identity verification. Here, in the present embodiment, the case of starting a transaction with a financial institution such as opening an account will be described as an example, and the identity verification in the invention is a confirmation for enabling an online procedure using a mobile terminal or the like. Refers to processing. On the other hand, the personal authentication means a state in which the personal identification has already been confirmed, and for preventing impersonation etc. at the time of starting the service for the second time and thereafter.
In the following description, a case will be described as an example where the mobile terminal 1 is used to start a service that requires identification and authentication.

The identity verification system 100 includes a mobile terminal 1 and an identity verification server 4. The mobile terminal 1 can be connected to the communication network N via a base station R for wireless communication, for example. The identity verification server 4 is communicatively connected via the communication network N.
The personal identification system 100 is connected to the service providing server 7 via the communication network N. The service providing server 7 is a server for performing service processing with the mobile terminal 1.

The mobile terminal 1 is, for example, a terminal carried by a person who wants to receive the service provided by the service providing server 7. Hereinafter, a person who intends to receive the service is referred to as a user (owner). The mobile terminal 1 is, for example, a mobile device having a computer function represented by a smartphone or a tablet.
As shown in FIG. 2, the mobile terminal 1 includes a control unit 10, a storage unit 30, a camera 33 (imaging unit), a touch panel display 35, and a communication interface unit 39.

The control unit 10 is a CPU (central processing unit) that controls the entire mobile terminal 1. The control unit 10 cooperates with the above-described hardware and executes various functions by appropriately reading and executing an operating system (OS) and various application programs stored in the storage unit 30.
The control unit 10 includes an application calling unit 11, an identity verification unit 12, an identity authentication unit 21, and a service start determination unit 22 (service start determination means).

When executing the transaction application 31a, the application calling unit 11 receives a request for identity verification or identity authentication from the service providing server 7 that performs processing corresponding to the transaction application 31a, and executes the identity verification application 31b (program). call.
The identity verification unit 12 is a control unit that performs identity verification processing. The identity verification unit 12 includes an identity verification image processing unit 13 (repetition processing means) and a collation image processing unit 17.

The personal identification image processing unit 13 includes a guide screen output unit 14 (guidance screen output unit), a personal identification image acquisition unit 15 (person identification image acquisition unit), and a validity confirmation unit 16 (validity confirmation unit). ..
The identity verification image processing unit 13 acquires an image required for identity verification by repeating the processing by the guidance screen output unit 14, the identity verification image acquisition unit 15, and the validity verification unit 16 multiple times.

The guidance screen output unit 14 outputs, on the touch panel display 35, a guidance screen that specifies the document position that is the arrangement position of the identification document and the coin position that is the arrangement position of the coin (object) to be photographed together with the identification document. To do.
Here, the personal identification document is a publicly recognized identification card in which a facial photograph of the user is posted. A driver's license 3 is shown in FIG. 1 as an example of an identification document. In addition, the personal identification document may be a my number card or the like on which a user's face photo is posted.
Moreover, the type of coins is not limited.
Further, the guide screen is displayed together with the through image in a state where the through image by the camera 33 is displayed on the touch panel display 35. Here, the through image is an image in which the shooting range of the camera 33 is displayed on the touch panel display 35. Also, as will be described later, the coin positions when the guide screen is output multiple times are different positions.
The identity verification image acquisition unit 15 acquires the identity verification image, which is an image including the identity verification document and the coin, by the user operating the camera 33 and integrally photographing the identity verification document and the coin. ..

The legitimacy confirmation unit 16 confirms the legitimacy of the personal identification image. For example, when nothing is reflected in at least one of the document position and the coin position, the validity checking unit 16 determines that the validity cannot be checked and gives an error. In addition, even if the identity verification document is reflected in the document position, the validity checking unit 16 validates the identity verification document when the identity verification document cannot be recognized, that is, when the character cannot be recognized from the image of the identity verification document. It is an error because the sex cannot be confirmed. In addition, even if the validity confirmation unit 16 can recognize the identity verification document, if the validity period of the identity verification document has expired, the validity confirmation unit 16 determines that the validity cannot be verified and makes an error.

Then, the identity verification image processing unit 13 stores the acquired identity verification image in the acquired image storage unit 32 when the validity can be confirmed. Further, the personal identification image processing unit 13 extracts a facial photograph image, which is a facial photograph portion of the personal identification document, from the acquired personal identification image. Then, the personal identification image processing unit 13 stores the extracted face photograph image in the acquired image storage unit 32.

The collation image processing unit 17 acquires the face image of the user and collates it with the face photograph of the identity verification document. The collation image processing unit 17 includes a blink detection unit 18 (blink detection unit), a collation image acquisition unit 19 (collation image acquisition unit), and an image transmission unit 20 (collation image transmission unit).
First, in order to acquire the face image of the user, the user keeps the camera 33 in the position where the face of the user is captured and continues to photograph the face of the user. Then, the blink detection unit 18 detects the blink from the acquired captured image.
The collation image acquisition unit 19 acquires the face image of the user captured by the camera 33 as the collation image used for collation after the blink detection unit 18 detects the blink.
The image transmission unit 20 transmits the acquired collation image and the facial photograph image stored in the acquired image storage unit 32 to the identity verification server 4 for collation.

The personal authentication unit 21 is a control unit that performs personal authentication. The personal authentication unit 21 includes a collation image processing unit 17. That is, the personal authentication unit 21 acquires a collation image that is the face image of the user after the blink detection by the camera 33, and acquires the collation image and the face photo image stored in the acquired image storage unit 32. It is sent to the identity verification server 4.
The service start determination unit 22 receives the matching result (matching result information) from the identity verification server 4 and determines whether or not the service can be started based on the received matching result.

The storage unit 30 is a storage area such as a semiconductor memory device for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
The storage unit 30 includes a program storage unit 31 and an acquired image storage unit 32.
The program storage unit 31 is a storage area that stores various application programs (hereinafter, application programs are referred to as applications, applications, programs, or the like). The program storage unit 31 stores a transaction application 31a.

The transaction application 31a is installed in the mobile terminal 1 in advance, or downloaded to the mobile terminal 1 by communicating with an application distribution server (not shown) via the communication network N.
The transaction application 31a is a program for performing service processing with the service providing server 7. The transaction application 31a is, for example, an application for opening a bank account. The transaction application 31a includes an identity verification application 31b.
The identity verification application 31b is a program for performing various functions of identity verification and identity authentication executed by the control unit 10 of the mobile terminal 1.
Although the transaction application 31a is described as including the identity verification application 31b in the present embodiment, the transaction application 31a and the identity verification application 31b may be different applications.

The acquired image storage unit 32 is a storage area that stores the image acquired by the camera 33. The acquired image storage unit 32 stores the personal identification image. Further, the acquired image storage unit 32 stores the face photograph image extracted from the personal identification image.

The camera 33 is a photographing device. The camera 33 has an in-camera 33a and an out-camera 33b. The in-camera 33a is a camera provided on the side of the touch panel display 35 of the mobile terminal 1. The out-camera 33b is a camera provided on the back side of the mobile terminal 1.
The touch panel display 35 has a function as a display unit including a liquid crystal panel and the like, and a function as an input unit for performing various operation inputs by the user's finger.
The communication interface unit 39 is an interface for communicating with various servers via the base station R and the communication network N, and serves as a transmission unit and a reception unit.

Returning to FIG. 1, the driver's license 3 is in the shape of a card, and is a public identification card as described above. On the front surface of the driver's license 3, personal information of the user including the user's address, name, date of birth, and user's face photograph is described. Further, on the surface of the driver's license 3, an expiration date, a license number, a grant date, a color type (whether gold or not, etc.) are described. In the personal identification system 100, the user of the mobile terminal 1 takes a picture of his/her driver's license 3 or the like using the mobile terminal 1.

The identity verification server 4 is a server that performs identity verification by online procedure with the mobile terminal 1 and performs identity verification by face authentication. The identity verification server 4 is assumed to be owned by a company different from the service providing company, but is not limited to this.
As shown in FIG. 3, the identity verification server 4 includes a control unit 40, a storage unit 50, and a communication interface unit 59.

The control unit 40 is a CPU that controls the entire identity verification server 4. The control unit 40 executes various functions in cooperation with the above-described hardware by appropriately reading and executing the OS and application programs stored in the storage unit 50.
The control unit 40 includes an identity verification image analysis unit 41, a face matching unit 43 (face matching unit), and a matching result transmitting unit 44 (result transmitting unit).

The personal identification image analysis unit 41 receives from the mobile terminal 1 a personal identification image including an image of the front surface of the personal identification document. Then, the personal identification image analysis unit 41 converts the characters included in the personal identification document of the personal identification image into text by OCR (optical character recognition). After that, the personal identification image analysis unit 41 transmits the text-formed character string to the mobile terminal 1.
The face matching unit 43 receives the matching image transmitted by the mobile terminal 1 and the face photograph image. Then, the face collation unit 43 collates the received collation image with the face photograph image. Here, the face matching unit 43 may show the matching result between the matching image and the face photograph image by a score. For example, the face matching unit 43 performs image matching processing and calculates the degree of matching between the face photograph images as a score. Here, the score is represented by a numerical value from 0 to 100, for example, and the higher the degree of coincidence, the higher the numerical value.
The collation result transmitting unit 44 transmits the collation result to the mobile terminal 1.

The storage unit 50 is a storage area such as a hard disk or a semiconductor memory device for storing programs, data, and the like required by the control unit 40 to execute various processes.
The storage unit 50 includes a program storage unit 51. The program storage unit 51 is a storage area that stores various programs. The program storage unit 51 stores a personal identification program 51a.
The identity verification program 51a is a program for performing various functions executed by the control unit 40 of the identity verification server 4.
The communication interface unit 59 is an interface for performing communication with various servers such as the service providing server 7 and the mobile terminal 1 via the communication network N.

Here, the computer refers to an information processing device including a control unit, a storage device, and the like, and the mobile terminal 1 and the identity verification server 4 are information processing devices including a control unit and a storage unit, respectively. Included in the concept.

The service providing server 7 shown in FIG. 1 is a server for performing service processing. Although only one service providing server 7 is shown in the personal identification system 100 in FIG. 1, a plurality of servers may be provided for each service.
Although not shown, the service providing server 7 includes a control unit, a storage unit, a communication interface unit, and the like.

The base station R shown in FIG. 1 is a base station for wireless communication, and relays for the mobile terminal 1 to communicate with various servers. The base station R is, for example, a base station of a wireless LAN (Local Area Network) or a base station for a mobile terminal communication network of a communication carrier.
The communication network N is a network between various servers or between various servers and the base station R, and is an internet line, a mobile terminal communication network, or the like.

Next, processing of the personal identification system 100 will be described.
FIG. 4 is a flowchart showing a service start process in the mobile terminal 1 according to this embodiment.
First, a user who wants to start a service, for example, if the transaction application 31a is installed in the mobile terminal 1 in advance, the icon of the transaction application 31a displayed on the touch panel display 35 of his/her mobile terminal 1 (not shown). Select operation such as tap). Further, for example, when the transaction application 31a is downloaded to the mobile terminal 1 for the first time, the user performs a selection operation such as tapping the icon of the transaction application 31a provided on the touch panel display 35 of the mobile terminal 1 of the user. do. By doing so, the control unit 10 of the mobile terminal 1 executes the transaction application 31a.

When the transaction application 31a is executed, the main screen (not shown) of the transaction application 31a is displayed on the touch panel display 35 of the mobile terminal 1. Therefore, when the user performs an operation for starting a service (for example, an operation for selecting a menu) from the main screen, the mobile terminal 1 starts the service in step S10 (hereinafter, simply referred to as “S”) in FIG. The operation is accepted and, for example, the process for starting the service is executed. After that, the control unit 10 (application calling unit 11) calls the personal identification application 31b.

In S11, the control unit 10 determines whether or not the identity verification processing has been completed. The case where the identity verification processing is completed means that the identity verification processing has already been performed because the service using the transaction application 31a has already been performed once. Therefore, the case where the personal identification processing has been completed corresponds to the case where the second or later service start is accepted. Further, the case where the identity verification processing is not completed corresponds to the case where the service start using the transaction application 31a is accepted for the first time.
Note that the control unit 10 may determine whether or not the identity confirmation processing has been completed, for example, by storing that the identity verification has been completed in the identity verification application 31b, and make a determination based on that. .. Further, when the identity verification processing is completed, the identity verification image and the facial photograph image are stored in the acquired image storage unit 32, so the control unit 10 stores the facial photograph image in the acquired image storage unit 32. You may judge by whether or not it is done.
When the identity verification process has been completed (S11: YES), the control unit 10 moves the process to S13. On the other hand, when the identity verification processing is not completed (S11: NO), the control unit 10 shifts the processing to S12.
In S12, the control unit 10 (identity confirmation unit 12) performs an identification process.

Here, the identification process performed by the mobile terminal 1 will be described.
FIG. 5 is a flowchart showing the identity confirmation processing in the mobile terminal 1 according to the present embodiment.
FIG. 6 is a flowchart showing the identity verification image processing in the mobile terminal 1 according to the present embodiment.
FIG. 7 is a diagram showing a pattern example of the guide screen according to the present embodiment.
FIG. 8 is a diagram showing a display example on the mobile terminal 1 according to the present embodiment.
FIG. 9 is a flowchart showing the validity confirmation process in the mobile terminal 1 according to the present embodiment.
FIG. 10 is a flowchart showing image analysis processing in the personal identification system 100 according to this embodiment.
11 and 12 are diagrams showing display examples on the mobile terminal 1 according to the present embodiment.
FIG. 13 is a flowchart showing the collation image processing in the personal identification system 100 according to this embodiment.
FIG. 14 is a diagram showing a display example on the mobile terminal 1 according to the present embodiment.

In S20 of FIG. 5, the control unit 10 (identity confirmation image processing unit 13) of the mobile terminal 1 performs identity verification image processing on the front side of the identity verification document.
Here, the personal identification image processing will be described with reference to FIG.
In S30 of FIG. 6, the control unit 10 (guidance screen output unit 14) performs coin position determination processing. The personal identification application 31b is prepared in advance with six patterns of guidance screens having different coin positions, such as guidance screens 61 to 66 shown in FIG. The guide screen 61 has a coin position 61b provided on the upper right side of the document position 61a. The guide screen 62 is provided with a coin position 62b at the right center of the document position 62a. The guide screen 63 is provided with a coin position 63b on the lower right side of the document position 63a. Similarly, on the guide screens 64 to 66, coin positions 64b to 66b are respectively provided on the left side of the document positions 64a to 66a in order from the upper side.
The control unit 10 randomly determines one guide screen from the guide screens 61 to 66, for example. Here, to determine at random means to display all the guide screens in random order. However, not all guide screens but only some guide screens may be designated.

In S31 of FIG. 6, the control unit 10 (guidance screen output unit 14) outputs the guidance screen determined by the coin position determination processing to the touch panel display 35. At that time, the control unit 10 causes the out-camera 33b to be executed and causes the touch panel display 35 to display the determined guide screen in a state where the through-image by the out-camera 33b is output.
As shown in FIG. 8(A), the user looks at the displayed guidance screen 64 and puts the driving license 3 and the coins 9 into the frame of the guidance screen so that the driving license 3 and the coins 9 can be accommodated. And the position of the mobile terminal 1 is adjusted.
In a state in which the driver's license 3 and the coin 9 are accommodated in the frame of the guide screen 64, the user operates the photographing button 64c displayed on the guide screen 64. Accept a shooting operation.

In S33, the control unit 10 (identity confirmation image acquisition unit 15) acquires the identity verification image 68a shown in FIG. 8(B). Then, the control unit 10 outputs the confirmation screen 68 including the personal identification image 68a as shown in FIG. 8B to the touch panel display 35. When the user wants to take a picture again, he/she can touch the button “retake” to return to the guide screen 64 shown in FIG. 8A and take the person confirmation image again.
For example, when the user touches the “next” button shown in FIG. 8B, the control unit 10 (validity confirmation unit 16) causes the validity confirmation shown in FIG. 9 in S34 of FIG. Perform processing.

In S40 of FIG. 9, the control unit 10 determines whether or not an object can be recognized at the coin position by performing a known image recognition process on the personal identification image. When the object can be recognized (S40: YES), the control unit 10 shifts the processing to S41. On the other hand, when the object cannot be recognized (S40: NO), the control unit 10 shifts the processing to S49.
In S41, the control unit 10 determines whether or not an object can be recognized at the document position. When the object can be recognized (S41: YES), the control unit 10 shifts the processing to S41. On the other hand, when the object cannot be recognized (S41: NO), the control unit 10 shifts the processing to S49.

In S42, the control unit 10 determines whether or not this process is a process relating to the surface. For example, the control unit 10 determines that the process is a process related to the surface when the process is a transition process from S20 in FIG. 5, but the process is a surface process when the process is a transition process from S23 or S24 in FIG. It is determined that the process is not related to. When the process is related to the surface (S42: YES), the control unit 10 shifts the process to S43. On the other hand, if the process is not related to the surface (S42: NO), the controller 10 shifts the process to S45.
In S43, the control unit 10 performs the image analysis process shown in FIG.

In S50 of FIG. 10, the control unit 10 transmits the acquired identity verification image to the identity verification server 4.
In S51, the control unit 40 of the identity verification server 4 receives the identity verification image.
In S52, the control unit 40 (personal identification image analysis unit 41) acquires character information in which the personal identification document included in the received personal identification image is converted into text.
In S53, the control unit 40 transmits the acquired character information to the mobile terminal 1. In addition, for example, when the character information cannot be obtained from the personal identification document because the image of the personal identification document is small or unclear, the control unit 40 indicates that the character information cannot be obtained. , May be transmitted to the mobile terminal 1. After that, the control unit 40 ends this processing.

In S54, the control unit 10 of the mobile terminal 1 receives the character information.
In S55, the control unit 10 confirms the validity based on the received character information. For example, when the control unit 10 receives from the identity verification server 4 that the character information could not be obtained, the control unit 10 cannot verify the validity. Further, the control unit 10 cannot confirm the validity when the validity period of the personal identification document has expired. After that, the control unit 10 shifts the processing to S44 of FIG.

In S44 of FIG. 9, the control unit 10 (validity confirmation unit 16) determines whether or not the validity of the character information has been confirmed. When the validity can be confirmed (S44: YES), the control unit 10 shifts the processing to S45. On the other hand, if the validity cannot be confirmed (S44: NO), the control unit 10 moves the process to S49.
In S45, the control unit 10 (validity confirming unit 16) moves the process to S35 in FIG. 6 assuming that the authenticity of the personal identification image can be confirmed.
On the other hand, in S49, the control unit 10 (validity confirmation unit 16) determines that the validity of the personal identification image could not be confirmed, and moves the process to S35 in FIG.

In S35 of FIG. 6, it is determined whether the legitimacy has been confirmed. When the validity can be confirmed (S35: YES), the control unit 10 moves the process to S36. On the other hand, when the validity cannot be confirmed (S35: NO), the control unit 10 moves the process to S39.
In S36, the control unit 10 stores the acquired identity verification image in the acquired image storage unit 32. After that, the control unit 10 shifts the processing to S21 of FIG.
On the other hand, in S39, the control unit 10 outputs, for example, the fact that the validity is an error to the touch panel display 35, and ends this process without proceeding to the next.

In S21 of FIG. 5, the control unit 10 extracts a facial photograph image from the acquired personal identification image.
In S22, the control unit 10 stores the extracted face photograph image in the acquired image storage unit 32.
The processing of S23 and S24 is the same as the processing of FIG. 6 described above.

FIG. 11A shows a guide screen 81 for acquiring a personal identification image including the image on the back surface of the driver's license 3. FIG. 11B shows a confirmation screen 82.
FIG. 12A shows a guide screen 83 for obliquely obtaining a personal identification image including an image of the surface of the driver's license 3. On the guide screen 83, a frame indicating the document position 83a has a trapezoidal shape in order to encourage photographing from an oblique direction. The coin position 83b is located at the lower right of the document position 83a. In this way, on the guidance screen for obtaining the personal identification image including the image of the surface of the driver's license 3 from an oblique direction, for example, the coin position is in the lower right, center lower, and lower left directions of the document position. An image may be prepared in advance, and the control unit 10 may randomly determine the guide screen of one pattern from the images. When photographing from an oblique direction, a space is formed in the vertical direction of the document position, and therefore the coin position can be arranged in such a labor direction. FIG. 12B shows the confirmation screen 84.

As described above, the guide screens 81 and 83 are screens in which the words 81e and 83e are different depending on the object to be imaged, and the coin positions 81b and 83b are different.
As described above, when photographing the driving license 3, the control unit 10 first photographs the front surface of the driving license 3, then photographs the back surface of the driving license 3, and finally, the driving license. The driver's license 3 is controlled so as to be photographed from an oblique direction with the surface visible.
By doing so, when the driver's license 3 is photographed a plurality of times, it is possible to output a guide screen in which coin positions are arranged at various positions.

In S25 of FIG. 5, the control unit 10 (collation image processing unit 17) performs the collation image processing shown in FIG.
In S60 of FIG. 13, the control unit 10 activates the in-camera 33a and outputs the face shooting screen 85 shown in FIG. 14 including the through image of the in-camera 33a to the touch panel display 35. The user adjusts the position of the mobile terminal 1 so that the face of the user can enter the face frame 85a of the face shooting screen 85.
In S61 of FIG. 13, the control unit 10 performs a known face recognition process on the captured image acquired from the in-camera 33a to detect eyes.
In S62, the control unit 10 (blink detection unit 18) determines whether or not a blink has been detected. When the blink is detected (S62: YES), the control unit 10 shifts the processing to S63. On the other hand, when the blink is not detected (S62: NO), the control unit 10 stays in this process until the blink is detected.

In S63, the control unit 10 (matching image acquisition unit 19) sets the face image captured after the blink is detected as the matching image.
In S<b>64, the control unit 10 (image transmission unit 20) transmits the acquired collation image together with the facial photograph image stored in the acquired image storage unit 32 to the identity verification server 4. Here, the facial photograph image stored in the acquired image storage unit 32 is an image captured and stored by the mobile terminal 1 in the personal identification image processing.
In S65, the control unit 40 (face matching unit 43) of the identity verification server 4 receives the matching image and the face photograph image from the mobile terminal 1.

In S66, the control unit 40 (face matching unit 43) performs face matching processing for matching the matching image with the face photograph image. The control unit 40 performs the image matching process, and calculates, for example, the degree of matching between the matching image and the face photograph image as a score.
In S67, the control unit 40 (matching result sending unit 44) sends the matching result to the mobile terminal 1. After that, the control unit 40 ends this processing.
In S68, the control unit 10 (matching image processing unit 17) of the mobile terminal 1 receives the matching result. After that, the control unit 10 shifts the processing to S14 of FIG.

In S13 of FIG. 4, the control unit 10 (personal authentication unit 21) performs a person authentication process. The control unit 10 performs the same processing as the collation image processing described in FIG. 13 as the personal authentication processing. The control unit 10 acquires the collation image used in the collation image processing, and transmits it to the identity verification server 4 together with the facial photograph image already stored in the acquired image storage unit 32. In the personal identification server 4, the control unit 40 (face matching unit 43, matching result transmitting unit 44) matches the matching image with the face photograph image, and transmits the matching result to the mobile terminal 1.

In S14, the control unit 10 (service start determination unit 22) determines whether or not the authentication is successful, based on the collation result. For example, when the score obtained as the matching result is equal to or more than the threshold value, the control unit 10 determines that the authentication is successful. When it is determined that the authentication is successful (S14: YES), the control unit 10 shifts the processing to S15. On the other hand, when it is not determined that the authentication is successful (S14: NO), the control unit 10 moves the process to S16.
In S15, the control unit 10 performs service processing with the service providing server 7 corresponding to the transaction application 31a. After that, the control unit 10 ends this processing.

On the other hand, in S16, the control unit 10 outputs a message of the authentication error to the touch panel display 35. After that, the control unit 10 ends this processing. That is, if the result of the verification of the face image and the face photograph image of the personal identification document cannot be verified by the process of the personal identification or the personal authentication, the service process is not started. In the case of an authentication error, the control unit 10 preferably deletes the image stored in the acquired image storage unit 32.

Thus, according to the present embodiment, the personal identification system 100 has the following effects.
(1) When providing a service using the mobile terminal 1, if the mobile terminal 1 has the camera 33, it is possible to confirm the identity using only the mobile terminal 1, which is convenient. In other words, the user can provide the user with an environment for verifying the identity anytime, anywhere without the hassle of going to a window or the like. In addition, security is high because the identification is performed using biometric information. Further, it is not necessary to register biometric information in advance such as a fingerprint, which is highly convenient.
(2) Since the process of acquiring the personal identification image and the process of collating the facial photograph image acquired from the personal identification image with the own facial image taken by the user can be performed in a series of flows, It is convenient.

(3) Since the guidance screen designating the arrangement positions of the driving license 3 and the coins 9 is output, the user places the driving license 3 and the coins 9 at the designated positions according to the instructions on the guidance screen. . Then, since the mobile terminal 1 acquires the personal identification image including the driver's license 3 and the coin 9 arranged by the user in accordance with the instruction on the guidance screen, the mobile terminal 1 instructs the user each time the personal identification image is acquired. Can be something that complies. Therefore, the randomness can be secured by the arrangement positions of the driver's license 3 and the coin 9 on the guide screen.

(4) As necessary for identity verification, the guidance screen is output multiple times to obtain identity verification images of the driver's license 3 and the coin 9 taken from different shooting directions. Since the guide screen changes the arrangement position of the coins 9 and arranges the coins at different positions each time, it is possible to require the user to perform an operation when the user takes an image. Therefore, it becomes a mechanism in which an image taken in advance cannot be used, and it is possible to make it difficult for fraud. Further, since the driver's license 3 is photographed from an oblique direction, the thickness of the driver's license 3 can be confirmed, and it can be confirmed whether it is a fake or not.

(5) Since the guide screen is randomly determined from a plurality of predetermined coin positions, randomness can be ensured and processing can be simplified.
(6) Since the guide screen has the arrangement position indicated by a frame line and the effect position is indicated by the same circular shape as the coin, the guide screen can be easily understood by the user.

(7) The legitimacy of the personal identification image is confirmed, and if the legitimacy is confirmed, the collation image is acquired. Therefore, in the case of fraud, it is possible to stop the process at the previous stage without collating. it can.
(8) Since the face image of the user is the image after detecting the blink of the user, an image suitable for the collation image can be acquired.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. In addition, the effects described in the embodiments are merely enumeration of the most suitable effects generated by the present invention, and the effects according to the present invention are not limited to those described in the embodiments. It should be noted that the above-described embodiment and the modification described later can be appropriately combined and used, but detailed description thereof will be omitted.

(Variation)
(1) In the present embodiment, the guide screen is prepared in advance with six patterns having different coin positions, and a guide screen randomly selected from the patterns is output, but this is taken as an example. Not limited. For example, the coin position may be set at an arbitrary position within a predetermined designated area.
FIG. 15 is a diagram showing a pattern example of a guide screen according to the modification.
The guide screen 291 shown in FIG. 15A includes a document position 291a and a coin area 291b. The guide screen 291 shows the coin position to the right, and the document position 291a is arranged to the left. In this case, the control unit of the mobile terminal 1 randomly determines the coin position in the range of the coin area 291b and outputs the coin position.

The guide screen 292 illustrated in FIG. 15B includes a document position 292a and a coin area 292b. The guide screen 292 shows the coin position on the left side and the document position 292a on the right side. In this case, the control unit of the mobile terminal 1 randomly determines the coin position in the range of the coin area 292b and outputs the coin position.
By doing so, the control unit of the mobile terminal 1 randomly determines the coin position in the coin area after determining whether the coin position is on the right side or the left side of the document position. Can be rich in.

(2) In the present embodiment, the case where the document position and the coin position output on the guide screen are indicated by the frame lines has been described as an example, but the present invention is not limited to this. The guide screen may be displayed by another method as long as it shows the document position and the coin position in a manner easily understood by the user.
FIG. 16 is a diagram showing a display mode example of a guide screen according to the modification.
The guide screen 295 shown in FIG. 16(A) shows the document position 295a and the coin position 295b by broken lines.
The guide screen 296 shown in FIG. 16B shows the document position 296a in a hook shape.
Alternatively, the document position and the coin position may be displayed in a clear state so as to have transparency, and the range excluding the document position and the coin position may be displayed in a blurred state, for example. Further, a frame line or the like indicating the document position and the coin position may be displayed such that the display and the non-display are repeated with the passage of time. Furthermore, although the coin position is shown as a circular shape, it may be an elliptical shape or a polygonal shape.

(3) In the present embodiment, as the validity confirmation, an object is reflected in the document position and the coin position, the personal identification document can be read, and the validity period of the personal identification document has not expired. Although the example of performing the confirmation processing has been described, the present invention is not limited to this. Only a part of the above-mentioned confirmations may be confirmed, or another confirmation may be performed. For example, the identity of the personal identification document captured on the surface of the personal identification document and the personal identification image captured from the diagonal direction of the personal identification document may be confirmed.
(4) In the present embodiment, the personal identification image has been described as a still image, but the present invention is not limited to this. In order to require the user to operate, a mechanism for showing the proof of the identity verification document by taking a driving license in a moving image or the like may be used.

(5) In the present embodiment, an example in which a driver's license is used as the personal identification document has been described, but the present invention is not limited to this. If it is a public identification card with a photograph of the person's face, it may be, for example, a my number card. When the personal identification card is used as the personal identification document, the back side does not include information necessary for acquisition, and thus the front side and the diagonal direction may be taken.
(6) In the present embodiment, an example in which a coin is used as an object to be photographed together with the personal identification document has been described, but the object is not limited to this. For example, a button or the like may be used as long as it has a circular shape. In addition, when the user does not have a circular object, a paper with a circle or the like may be placed on the paper according to the position of the coin.
Further, the person is not limited to the circular shape, and the other person that the user may have when performing the person confirmation may be used as an object to perform the person confirmation.

(7) In the present embodiment, the case where the personal identification server performs the image matching process and the character recognition process has been described as an example, but the present invention is not limited to this. For example, an image matching server communicatively connected to a personal identification server and an OCR server are provided, face images are matched using the image matching server, and character recognition processing is performed using the OCR server. It may be.
(8) In the present embodiment, the processing using the identity verification server and the mobile terminal has been described, but the content of processing between the identity verification server and the mobile terminal is not limited to this. The process other than the image acquisition performed by the mobile terminal may be performed by the identity verification server, or the process performed by the identity verification server may be performed by the mobile terminal.

(9) Various services are applicable to the service for performing personal identification and personal authentication in the present embodiment. For example, in the case of banking business, it can be used for account opening and account linking processing, cash withdrawal processing using ATM, and the like.
As for using the ATM, personal identification is performed by the mobile terminal, and when authenticated, a QR code (registered trademark) is displayed on the mobile terminal, and the ATM is read, whereby the mobile terminal is replaced by a cash card. Can be
For other ATMs, a QR code is output to the ATM, the mobile terminal reads the QR code, authenticates the identity after associating the account information, and replaces the mobile terminal with a cash card. Can be
Also, in the case of an agent acting on behalf of the person at the counter, the agent receives information proving that he/she has been authenticated, so that he/she does not need to prepare a power of attorney in advance. I'm done.

1 mobile terminal 3 driver's license 4 identity verification server 7 service providing server 10, 40 control unit 12 identity verification unit 13 identity verification image processing unit 14 guidance screen output unit 15 identity verification image acquisition unit 16 validity verification unit 17 collation image processing Part 18 Blink detection part 19 Matching image acquisition part 20 Image transmission part 21 Personal authentication part 22 Service start determination part 30,50 Storage part 31a Transaction application 31b Personal identification application 32 Acquired image storage part 33 Camera 35 Touch panel display 41 Personal identification image analysis Part 43 Face matching part 44 Matching result sending part 100 Identity verification system

Claims (11)

A mobile terminal having a photographing unit,
The data necessary to confirm the identity of the user at the time of starting the service is performed using only the mobile terminal possessed by the user,
A guide screen output means for outputting a guide screen specifying the arrangement position of the object to be photographed together with the personal identification document in which the facial photograph is posted,
According to the guide screen output by the guide screen output unit, a personal identification image acquisition unit that acquires a personal identification image that is an image including the object and the personal identification document from the photographing unit,
A collation image acquisition unit that acquires a collation image that is an image including the face of the holder who possesses the identity verification document from the photographing unit,
The personal identification image acquired by the personal identification image acquisition unit and the collation image acquired by the collation image acquisition unit, a collation image transmission unit that transmits to the personal identification server that performs personal identification using an image,
Be equipped with,
A mobile terminal characterized by. The mobile terminal according to claim 1,
And a repeating processing means for repeating the processing by the guide screen output means and the processing by the personal identification image acquisition means a plurality of times,
The guide screen output means outputs the guide screen in which the arrangement position of the object is different each time,
The personal identification image acquisition means acquires a plurality of the personal identification images obtained by photographing the personal identification document from different photographing directions from the photographing unit,
A mobile terminal characterized by. In the mobile terminal according to claim 1 or 2,
The guide screen output means outputs the guide screen including the arrangement position of the object randomly determined from a plurality of predetermined positions,
A mobile terminal characterized by. In the mobile terminal according to claim 1 or 2,
The guide screen output means outputs the guide screen including the arrangement position of the object randomly determined within a predetermined area of the guide screen determined in advance,
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 4,
The guide screen output means outputs the guide screen specifying the arrangement position of the personal identification document together with the arrangement position of the object,
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 5,
The guide screen output means outputs the guide screen showing the arrangement position as a frame line,
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 6,
The guide screen output means outputs the guide screen showing the arrangement position of the object as a circular shape,
The object includes a coin,
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 7,
Based on the personal identification image acquired by the personal identification image acquisition means, a legitimacy confirmation means for confirming the legitimacy of the object and the personal identification document,
The collation image acquisition unit acquires the collation image when the legitimacy can be confirmed by the legitimacy confirmation unit.
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 8,
A blink detecting means for detecting a blink from the image of the face of the holder acquired from the photographing unit,
The collation image acquisition means acquires the collation image photographed by the photographing unit after the blink detection means detects a blink,
A mobile terminal characterized by. A mobile terminal having a photographing unit,
An identity verification server that is communicably connected to the mobile terminal and performs identity verification processing;
An identity verification system including
The mobile terminal is
The data necessary to confirm the identity of the user at the time of starting the service is performed using only the mobile terminal possessed by the user,
A guide screen output means for outputting a guide screen specifying the arrangement position of the object to be photographed together with the personal identification document in which the facial photograph is posted,
According to the guide screen output by the guide screen output unit, a personal identification image acquisition unit that acquires a personal identification image that is an image including the object and the personal identification document from the photographing unit,
A collation image acquisition unit that acquires a collation image that is an image including the face of the holder who possesses the identity verification document from the photographing unit,
A matching image transmission unit that transmits the personal identification image acquired by the personal identification image acquisition unit and the collation image acquired by the collation image acquisition unit to the personal identification server.
Equipped with
The identification server is
Face collating means for collating the collation image received from the mobile terminal with the face photograph image included in the personal identification image,
Collating result information by the face collating means, result transmitting means for transmitting to the mobile terminal,
Equipped with
Whether the mobile terminal starts service processing between the mobile terminal and a service providing server which is communicably connected to the personal identification server and performs service processing based on the received verification result information. A service start determination means for determining
A personal identification system characterized by. A computer possessed by the user, which has an image capturing unit and is used to obtain data necessary for identifying the user when the service starts,
A guide screen output means for outputting a guide screen specifying the arrangement position of the object to be photographed together with the personal identification document in which the facial photograph is posted,
According to the guide screen output by the guide screen output unit, a personal identification image acquisition unit that acquires a personal identification image that is an image including the object and the personal identification document from the photographing unit,
A collation image acquisition unit that acquires a collation image that is an image including the face of the holder who possesses the identity verification document from the photographing unit,
The personal identification image acquired by the personal identification image acquisition unit and the collation image acquired by the collation image acquisition unit, a collation image transmission unit that transmits to the personal identification server that performs personal identification using an image,
And a program to make it work.

Images