JP2020077421A - Portable terminal, identity verification server, identity verification system, and program - Google Patents

Abstract

To provide a portable terminal, an identity verification server, an identity verification system and a program for identity verification using electronic data obtained by a method other than reading an IC card.SOLUTION: A portable terminal 1 with a camera 23 is such that identity verification of a user at the start of a service can be performed by the portable terminal 1 alone carried by the user. The portable terminal comprises: an identity verification image processing unit 13 to acquire an identity verification image from the camera 23, where the identity verification image is an image of an identity verification document having a photograph of the face thereon; a storage image processing unit 15 to acquire a storage image from the camera 23, where the storage image is an image including both the identity verification document and the face of the user carrying the identity verification document; and a collation image processing unit 16 to acquire a collation image from the camera 23, where the collation image is an image including the face of the user, and transmit the identity verification image and the collation image to an identity verification server 4, which performs identity verification using the images.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a mobile terminal, an identity verification server, an identity verification system and a program used for identity verification processing.

  In the past, so-called internet banking services have been provided to customers through communication networks such as the internet in banks and the like. In the Internet banking service, for example, the one using biometric information is considered in order to confirm the identity of the customer by using a terminal owned by the customer such as a mobile terminal. As an example, there is disclosed a method of performing personal identification by face authentication using a face image stored in an IC card such as a license and a face image of a customer who is an operator of the terminal (for example, Patent Document 1). 1).

JP, 2005-88080, A

Since the device described in Patent Document 1 uses face image data stored in the chip of the IC card, the information terminal needs to be equipped with a reader for reading the IC card.
Not only for services such as banks, but for applying to start using transactions for services of various financial institutions, in order to use it with a mobile terminal, it is necessary to be able to verify the person's identity with electronic data. There is a demand for a mechanism that can be used by other methods.

  Therefore, it is an object of the present invention to provide a mobile terminal, an identification server, an identification system, and a program for performing identification using electronic data obtained by a method other than reading an IC card.

  The present invention solves the above problems by the following means. It should be noted that, for ease of understanding, reference numerals corresponding to the embodiments of the present invention will be given and described, but the present invention is not limited thereto. Further, the configurations described with reference numerals may be appropriately improved, or at least a part of them may be replaced with other components.

A first aspect of the present invention is a mobile terminal (1) having a photographing unit (23), and obtains data necessary for confirming a user's identity at the time of starting a service, using only the mobile terminal possessed by the user. A personal identification image acquisition unit (13) that acquires an personal identification image, which is an image of the personal identification document (3) on which a facial photograph is posted, from the photographing unit, the personal identification document, and the personal identification. The stored image acquisition means (15) for acquiring a stored image, which is an image including both the face of the holder who holds the document from the photographing unit, and a collation image which is an image including the face of the holder, A verification image acquisition unit (16) acquired from the image capturing unit, the verification image acquired by the verification image acquisition unit, and the verification image acquired by the verification image acquisition unit are used to verify verification of the identity using images. The mobile terminal is provided with a collation image transmission means (16) for transmission to the identity verification server (4).
2nd invention is a portable terminal (1) of 1st invention, The display part (25) which displays the image of the imaging | photography range by the said imaging | photography part (23), the said personal identification document (3), and the said holder. And a guide image output means (14) for outputting a guide image showing the arrangement range of the face to the display unit.
3rd invention is the portable terminal (1) of 2nd invention, The said guide image output means (14) WHEREIN: The arrangement | positioning range of the said personal identification document (3) is a random position which does not overlap with the position of the said face. And outputting the guide image shown in a range of a predetermined size or more to the display unit (25).
A fourth invention is the portable terminal (1) of the second invention or the third invention, wherein an image of the personal identification document (3) included in the saved image acquired by the saved image acquisition means (15) is included. And a stored image confirmation means (15) for confirming the position of the face image of the holder based on the guide image output by the guide image output means (14). ..
A fifth aspect of the present invention is the portable terminal (1) according to any one of the first to fourth aspects, wherein the saved image acquisition means (16) has a plurality of different arrangement positions of the personal identification document (3). Is acquired from the photographing unit (23).
A sixth aspect of the invention is to extract a face photograph image from the personal identification image acquired by the personal identification image acquisition means (13) in the mobile terminal (1) according to any one of the first to fifth aspects of the invention. The face photograph image extracting means (13) is provided, and the collation image transmitting means (16) sends the face photograph image and the collation image extracted by the face photograph image extracting means to the identity verification server (4). The mobile terminal is characterized by transmitting and.
A seventh invention is an identity verification server (4) that is communicably connected to the mobile terminal (1) according to any one of the first to sixth inventions and performs identity verification processing. Face collating means (43) for collating the collation image received from the mobile terminal with the face photograph image included in the personal identification image, and result transmitting means for transmitting collation result information by the face collating means to the mobile terminal. (44), and the identity verification server.
An eighth invention is an identity verification system (100) including the mobile terminal (1) according to any one of the first to sixth inventions and the identity verification server (4) of the seventh invention. If the identity of the person is not confirmed, the portable terminal transmits the identity verification image acquisition means (13), the saved image acquisition means (15), the collation image acquisition means (16) and the collation image transmission. A service is executed between a service providing server (7) that performs processing by means (16) and is communicably connected to the mobile terminal and the identity verification server based on the collation result information and performs service processing. A personal identification system, comprising: a service start determination means (18) for determining whether or not to start processing.
A ninth aspect of the present invention provides a computer (1) possessed by the user, which has a photographing unit (23) and is used for confirming the identity of the user at the time of starting the service, in the identity verification document (3) in which a facial photograph is posted. A personal identification image that is an image, a personal identification image acquisition unit that acquires from the imaging unit, the personal identification document, and a saved image that is an image including both the face of the holder who has the personal identification document, A stored image acquisition unit that is acquired from the image capturing unit, a collation image acquisition unit that acquires a collation image that is an image including the face of the holder from the image capturing unit, and the personal identification that is acquired by the personal identification image acquisition unit. A program (21b) for causing the image and the collation image acquired by the collation image acquisition unit to function as a collation image transmission unit that transmits the image to the identity verification server (4) that performs identity verification using the image. ).

  According to the present invention, it is possible to provide a mobile terminal, an identification server, an identification system, and a program for performing identification using electronic data obtained by a method other than reading an IC card.

It is a figure showing the whole personal identification system composition concerning a 1st embodiment. It is a functional block diagram of the portable terminal which concerns on 1st Embodiment. It is a functional block diagram of the identity verification server according to the first embodiment. It is a flow chart which shows service start processing in a personal digital assistant concerning a 1st embodiment. It is a flow chart which shows personal identification processing in a personal digital assistant concerning a 1st embodiment. It is a flow chart which shows personal identification image processing in the personal identification system concerning a 1st embodiment. 6 is a flowchart showing saved image processing in the personal identification system according to the first embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on 1st Embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on 1st Embodiment. It is a flow chart which shows the collation image processing in the personal identification system concerning a 1st embodiment. It is the whole block diagram of the personal identification system concerning a 2nd embodiment, and the functional block diagram of a personal digital assistant and a personal identification server. It is a flow chart which shows personal identification image processing in the personal identification system concerning a 2nd embodiment. It is a functional block diagram of the personal digital assistant in the personal identification system concerning a 3rd embodiment. It is a flow chart which shows the personal identification image processing in the personal identification system concerning a 3rd embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on 3rd Embodiment. It is a figure which shows the example of the document position guide image displayed on the mobile terminal which concerns on 3rd Embodiment. It is a flow chart which shows saved image processing in the personal identification system concerning a 3rd embodiment. It is a flow chart which shows the collation image processing in the personal identification system concerning a 3rd embodiment. It is a flow chart which shows acquisition processing of a collation image in a personal digital assistant concerning a 3rd embodiment. It is a figure which shows the example of a display in the portable terminal which concerns on 3rd Embodiment.

  Hereinafter, modes for carrying out the present invention will be described with reference to the drawings. Note that this is merely an example, and the technical scope of the present invention is not limited to this.

(First embodiment)
FIG. 1 is a diagram showing an overall configuration of an identity verification system 100 according to the first embodiment.
FIG. 2 is a functional block diagram of the mobile terminal 1 according to the first embodiment.
FIG. 3 is a functional block diagram of the identity verification server 4 according to the first embodiment.
The personal identification system 100 shown in FIG. 1 is a system for performing personal identification required only when the first service of various services is started, using only the mobile terminal 1. The personal identification system 100 is a system that performs personal authentication required only when the service is started for the second time and thereafter only by the mobile terminal 1.

Here, various services are, for example, services related to opening an account in a bank, various banking services, car sharing services, renting real estate, and the like, and refer to various services that require identity verification. Currently, as a means of confirming the identity of a person who opens a bank account, etc., a mail that does not require transfer is sent to the home of the applicant who applied for the account opening. Recently, financial institutions are considering ways to complete online identity verification. The identity verification in the present invention refers to a verification process for enabling an online procedure using a mobile terminal or the like when starting a transaction with a financial institution such as opening an account. On the other hand, the personal authentication means a state in which the personal identification has already been confirmed, and for preventing impersonation etc. at the time of starting the service for the second time and thereafter.
In the following description, a case will be described as an example where the mobile terminal 1 is used to start a service that requires identification and authentication.

The identity verification system 100 includes a mobile terminal 1 and an identity verification server 4. The mobile terminal 1 can be connected to the communication network N via a base station R for wireless communication, for example. The identity verification server 4 is communicatively connected via the communication network N.
The personal identification system 100 is connected to the service providing server 7 via the communication network N. The service providing server 7 is a server for performing service processing with the mobile terminal 1.

The mobile terminal 1 is, for example, a terminal owned by a person who wants to receive the service provided by the service providing server 7. Hereinafter, a person who intends to receive the service is referred to as a user. The mobile terminal 1 is, for example, a mobile device having a computer function represented by a smartphone or a tablet.
As shown in FIG. 2, the mobile terminal 1 includes a control unit 10, a storage unit 20, a camera 23 (imaging unit), a touch panel display 25, and a communication interface unit 29.

The control unit 10 is a CPU (central processing unit) that controls the entire mobile terminal 1. The control unit 10 cooperates with the above-described hardware to execute various functions by appropriately reading and executing the operating system (OS) and various application programs stored in the storage unit 20.
The control unit 10 includes an application launching unit 11, an identity verification unit 12, an identity verification unit 17, and a service start determination unit 18 (service start determination means).

When the transaction application 21a is executed, the application activation unit 11 receives a request for identity authentication from the service providing server 7 that performs processing corresponding to the transaction application 21a, and activates the identity confirmation application 21b (program).
The identity verification unit 12 is a control unit that performs identity verification processing. The personal identification section 12 includes a personal identification image processing section 13 (personal identification image acquisition means, face photograph image extraction means), a guide image output section 14 (second guide image output means), and a saved image processing section 15 (save). The image acquisition unit, the second image confirmation unit) and the collation image processing unit 16 (collation image acquisition unit, collation image transmission unit).

The identity verification image processing unit 13 acquires the identity verification image that is the image of the identity verification document by the user operating the camera 23 to photograph his or her identity verification document. Here, the personal identification document is a publicly recognized identification card in which a facial photograph of the user is posted. A driver's license 3 is shown in FIG. 1 as an example of an identification document. In addition, the personal identification document may be a my number card or the like on which a user's face photo is posted.
Then, the personal identification image processing unit 13 extracts a facial photograph image from the acquired personal identification image. After that, the personal identification image processing unit 13 stores the extracted face photograph image in the storage unit 20.

  The guide image output unit 14 displays a guide image (second guide image) for acquiring an image including both the identity verification document and the face of the user (holder) who possesses the identity verification document on the touch panel display 25. Output to. The guide image is an auxiliary image showing the arrangement range, and serves as a guide for obtaining an image in which the identity verification document and the user's face are arranged at the positions as instructed.

The saved image processing unit 15 operates the camera 23 by the user to acquire a saved image that is an image including both the identity verification document and the face of the user who possesses the identity verification document. At that time, the user operates his / her own mobile terminal 1 so that both the personal identification document and the face of the user who possesses the personal identification document can fit within the guide position indicated by the guide image.
The saved image processing unit 15 confirms that the image of the personal identification document included in the acquired saved image and the face image of the user are within the range of the guide image. Then, the saved image processing unit 15 stores the saved image in the storage unit 20 when the saved image can be confirmed.

The collation image processing unit 16 acquires a collation image which is a user's face image and is used for collation by the user operating the camera 23 and photographing his / her face.
Then, the collation image processing unit 16 transmits the acquired collation image and the face photograph image stored in the storage unit 20 to the identity verification server 4.

The personal authentication unit 17 is a control unit that performs personal authentication. The user authentication unit 17 operates the camera 23 to capture a face image of the user when the user operates the camera 23 to capture a face image of the user, and obtains a match image used for matching, and stores the acquired match image in the storage unit 20. The stored face photograph image is transmitted to the identity verification server 4.
The service start determination unit 18 receives the matching result (matching result information) from the identity verification server 4 and determines whether the service can be started based on the received matching result.

The storage unit 20 is a storage area such as a semiconductor memory device for storing programs, data, and the like necessary for the control unit 10 to execute various processes.
The storage unit 20 includes a program storage unit 21. The program storage unit 21 is a storage area that stores various application programs (hereinafter, application programs are referred to as applications, applications, programs, or the like). The program storage unit 21 stores a transaction application 21a and an identity verification application 21b. In the present embodiment, the transaction application 21a and the identity verification application 21b will be described below, but an application in which the functions of the two applications are integrated may be used.

The transaction application 21a and the identity verification application 21b are installed in the mobile terminal 1 in advance, or downloaded to the mobile terminal 1 by communicating with an application distribution server (not shown) via the communication network N.
The transaction application 21a is a program for performing service processing with the service providing server 7.
The identity verification application 21b is a program for performing various functions of identity verification and identity authentication executed by the control unit 10 of the mobile terminal 1.

The camera 23 is a photographing device. The camera 23 has an in-camera 23a and an out-camera 23b. The in-camera 23a is a camera provided on the side of the touch panel display 25 of the mobile terminal 1. The out camera 23b is a camera provided on the back side of the mobile terminal 1.
The touch panel display 25 has a function as a display unit including a liquid crystal panel and the like, and a function as an input unit for inputting various operations by the user's finger.
The communication interface unit 29 is an interface for communicating with various servers via the communication network N, and serves as a transmission unit and a reception unit.

  Returning to FIG. 1, the driver's license 3 is in the shape of a card, and is a public identification card as described above. On the front surface of the driver's license 3, personal information of the user including the user's address, name, date of birth, and user's face photograph is described. The user of the mobile terminal 1 uses the mobile terminal 1 to photograph his / her driver's license 3 or the like in the personal identification system 100.

The identity verification server 4 is a server that performs identity verification with an online procedure with the mobile terminal 1 and performs identity verification by face authentication. The identity verification server 4 is owned by a company different from the company providing the service. The identity verification server 4 may be owned by a company that provides the service.
As shown in FIG. 3, the identity verification server 4 includes a control unit 40, a storage unit 50, and a communication interface unit 59.

The control unit 40 is a CPU that controls the entire identity verification server 4. The control unit 40 executes various functions in cooperation with the above-described hardware by appropriately reading and executing the OS and application programs stored in the storage unit 50.
The control unit 40 includes a face matching unit 43 (face matching unit) and a matching result transmitting unit 44 (result transmitting unit).

The face matching unit 43 receives the matching image transmitted by the mobile terminal 1 and the face photograph image. Then, the face collation unit 43 collates the received collation image with the face photograph image. Here, the face matching unit 43 may show the matching result between the matching image and the face photograph image by a score. For example, the face matching unit 43 performs image matching processing and calculates the degree of matching between the face photograph images as a score. Here, the score is represented by a numerical value from 0 to 100, for example, and the higher the degree of coincidence, the higher the numerical value.
The collation result transmitting unit 44 transmits the collation result to the mobile terminal 1.

The storage unit 50 is a storage area such as a hard disk or a semiconductor memory device for storing programs, data, and the like required by the control unit 40 to execute various processes.
The storage unit 50 includes a program storage unit 51. The program storage unit 51 is a storage area that stores various programs. The program storage unit 51 stores a personal identification program 51a.
The identity verification program 51a is a program for performing various functions executed by the control unit 40 of the identity verification server 4.
The communication interface unit 59 is an interface for performing communication with various servers such as the service providing server 7 and the mobile terminal 1 via the communication network N.

  Here, the computer refers to an information processing device including a control unit, a storage device, and the like, and the mobile terminal 1 and the identity verification server 4 are information processing devices including a control unit and a storage unit, respectively. Included in the concept.

The service providing server 7 shown in FIG. 1 is a server for performing service processing. Although only one service providing server 7 is shown in the personal identification system 100 in FIG. 1, a plurality of servers may be provided for each service.
Although not shown, the service providing server 7 includes a control unit, a storage unit, a communication interface unit, and the like.

The base station R shown in FIG. 1 is a base station for wireless communication, and relays for the mobile terminal 1 to communicate with various servers. The base station R is, for example, a base station of a wireless LAN (Local Area Network) or a base station for a mobile terminal communication network of a communication carrier.
The communication network N is a network between various servers or between various servers and the base station R, and is an internet line, a mobile terminal communication network, or the like.

Next, processing of the personal identification system 100 will be described.
FIG. 4 is a flowchart showing service start processing in the mobile terminal 1 according to the first embodiment.
First, the user who wants to start the service performs a touch operation such as a tap for selecting the icon (not shown) of the transaction application 21a displayed on the touch panel display 25 of the mobile terminal 1 of the user. By doing so, the control unit 10 of the mobile terminal 1 executes the transaction application 21a.

When the transaction application 21a is executed, the main screen (not shown) of the transaction application 21a is displayed on the touch panel display 25 of the mobile terminal 1. Therefore, when the user performs an operation for starting a service (for example, an operation for selecting a menu) from the main screen, the control unit 10 of the mobile terminal 1 is performed in step S10 (hereinafter, simply referred to as “S”) in FIG. Accepts a service start operation.
Next, in S11, the control unit 10 (application activation unit 11) activates the personal identification application 21b.

In S12, the control unit 10 determines whether the identity has been confirmed. The case where the identity verification has been completed means that the service using the transaction application 21a has already been performed, and therefore the identity verification processing has already been performed. Therefore, the case where the identity has been confirmed corresponds to the case where the second and subsequent service starts are accepted. Further, the case where the identity has not been confirmed corresponds to the case where the service start using the transaction application 21a is accepted for the first time.
It should be noted that the control unit 10 may determine whether or not the identity has been confirmed, for example, by storing the identity confirmation application 21b in association with the identity-confirmed transaction application 21a. Good. Further, when the person has been confirmed, since the facial photograph image is stored in the storage unit 20, the control unit 10 may determine whether or not the facial photograph image is stored in the storage unit 20. ..
When the identity has been confirmed (S12: YES), the control unit 10 shifts the processing to S14. On the other hand, when the identity has not been confirmed (S12: NO), the control unit 10 shifts the processing to S13.
In S13, the control unit 10 (identity confirmation unit 12) performs an identification process. The identification process will be described later. After that, the control unit 10 shifts the processing to S15.

  On the other hand, in S14, the control unit 10 (personal authentication unit 17) performs the person authentication process. The control unit 10 first activates the in-camera 23a and causes the user to take a picture of his / her own face, thereby acquiring a collation image that is a face image of the user. Then, the control unit 10 transmits the acquired collation image and the face photograph image obtained from the personal identification document stored in the storage unit 20 to the personal identification server 4. In the personal identification server 4, the control unit 40 (face matching unit 43, matching result transmitting unit 44) matches the matching image with the face photograph image, and transmits the matching result to the mobile terminal 1.

In S15, the control unit 10 (service start determination unit 18) determines whether or not the authentication is successful based on the collation result. For example, when the score obtained as the matching result is equal to or more than the threshold value, the control unit 10 determines that the authentication is successful. When it is determined that the authentication is successful (S15: YES), the control unit 10 shifts the processing to S16. On the other hand, when it is not determined that the authentication is successful (S15: NO), the control unit 10 moves the process to S18.
In S16, the control unit 10 ends the execution of the personal identification application 21b.
In S17, the control unit 10 performs service processing with the service providing server 7 corresponding to the transaction application 21a. After that, the control unit 10 ends this processing.

On the other hand, in S18, the control unit 10 outputs a message of the authentication error to the touch panel display 25.
In S19, the control unit 10 ends the execution of the identity verification application 21b. After that, the control unit 10 ends this processing.

Next, an identification process performed by the mobile terminal 1 will be described.
FIG. 5 is a flowchart showing the identity verification processing in the mobile terminal 1 according to the first embodiment.
FIG. 6 is a flowchart showing the personal identification image processing in the personal identification system 100 according to the first embodiment.
FIG. 7 is a flowchart showing stored image processing in the personal identification system 100 according to the first embodiment.
8 and 9 are diagrams showing display examples on the mobile terminal 1 according to the first embodiment.
FIG. 10 is a flowchart showing the collation image processing in the personal identification system 100 according to the first embodiment.

In S30 of FIG. 5, the control unit 10 (identity confirmation image processing unit 13) of the mobile terminal 1 performs identity confirmation image processing.
Here, the personal identification image processing will be described with reference to FIG.
In S40 of FIG. 6, the control unit 10 of the mobile terminal 1 activates the out-camera 23b and causes the touch panel display 25 to display a through image (image of the shooting range) of the out-camera 23b so that the identity verification document can be captured. To do. Then, the user taps a photographing button (not shown) so that the identity verification document is reflected in the through image, and the control unit 10 acquires the identity verification image.

At that time, the control unit 10 first displays an identification card selection screen (not shown) for selecting what is to be photographed as the identity verification document (driver's license 3 or my number card), and the user is identified. You may select the type of confirmation document. Then, the control unit 10 may change the guidance for shooting according to the type of the personal identification document. For example, in the case of a driver's license 3, it is necessary to photograph the front and back sides. On the other hand, with my number card, it is sufficient to shoot only the front side.
Further, the control unit 10 confirms that the personal identification document is authentic, so that an image from a direction where the thickness of the card, which is the side of the personal identification document, can be seen is used as a part of the personal identification image. You may get it.
Furthermore, the control unit 10 may acquire a moving image in which a series of operations is recorded, instead of a plurality of still images.

In S41, the control unit 10 extracts a facial photograph image from the personal identification image. The control unit 10 analyzes the personal identification image and identifies the personal identification document, for example. Then, the control unit 10 specifies the image position of the face photograph from the personal identification document and extracts the image at the specified image position.
In S42, the control unit 10 stores the extracted face photograph image in the storage unit 20. After that, the control unit 10 shifts the processing to S31 of FIG.

In S31 of FIG. 5, the control unit 10 performs saved image processing.
Here, the saved image processing will be described based on FIG. 7.
In S50 of FIG. 7, the control unit 10 causes the touch panel display 25 to display an explanation screen for shooting a stored image.
FIG. 8A is an example of the explanation screen 80. The explanation screen 80 is a screen for explaining that the face of the user and the personal identification document are to be photographed together.

In S51 of FIG. 7, the control unit 10 (guide image output unit 14) identifies the guide image and causes the touch panel display 25 to display the identified guide image.
FIG. 8B is an example of the guide image 81. The guide image 81 is an image including a face position area 81a in which the user's face is placed and a document position area 81b in which the identity verification document is placed.

Here, the control unit 10 may randomly specify one guide image from the guide images prepared in advance and have different document position areas, and display the guide image on the touch panel display 25. Here, the document position area is provided at a position that does not overlap the face position area. Further, the document position area is an area of a predetermined size or more such that the personal identification document can be visually recognized.
The guide image is not limited to a still image and may be a moving image. Further, the guide image in which the position of the document position area is different includes that in which the position of the document position area is different in the vertical and horizontal directions with respect to the photographing position, and that in which the depth direction is different from the photographing position.

In S52 of FIG. 7, the control unit 10 (saved image processing unit 15) activates the in-camera 23a and causes the touch panel display 25 to display the through image of the in-camera 23a together with the guide image.
As shown in FIG. 9, the user positions his / her face so that his / her own face is reflected in the face position area 81 a of the guide image 81, and the identity verification document is reflected in the document position area 81 b of the guide image 81. Adjust the position of the identification document. Then, when the user taps the shooting button 81c, the control unit 10 acquires a saved image including the face of the user captured by the in-camera 23a and the identity verification document.

  Here, the control unit 10 may sequentially display a plurality of guide images having different positions of the document position area so that the user can obtain a plurality of stored images. In addition, the control unit 10 may acquire the stored image of the moving image for the user according to the guide image of the moving image. That is, the control unit 10 outputs a different guide image each time the person is confirmed. Therefore, the user associates the position of the face and the position of the personal identification document with the guide image in order to obtain the saved image corresponding to the output guide image.

In S53 of FIG. 7, the control unit 10 (saved image processing unit 15) confirms the acquired saved image. Specifically, the control unit 10 confirms that the image of the personal identification document included in the acquired saved image and the face image of the user roughly match the range of the guide image.
In S54, the control unit 10 (saved image processing unit 15) stores the saved image in the storage unit 20 on condition that the saved image can be confirmed. If the saved image cannot be confirmed, that is, if the image of the personal identification document included in the acquired saved image and the face image of the user are not within the range of the guide image, an error will occur. The subsequent processing may not be performed, or the processing from S51 may be performed again. After that, the control unit 10 shifts the processing to S32 in FIG.

In S32 of FIG. 5, the control unit 10 (collation image processing unit 16) performs collation image processing.
Here, the collation image processing will be described with reference to FIG.
In S60 of FIG. 10, the control unit 10 of the mobile terminal 1 activates the in-camera 23a and causes the touch panel display 25 to display a through image of the in-camera 23a. Then, the user taps a shooting button (not shown) so that the user's face is reflected in the through image, and the control unit 10 acquires the collation image.

In S61, the control unit 10 transmits the acquired collation image together with the facial photograph image stored in the storage unit 20 to the identity verification server 4. Here, the face photograph image stored in the storage unit 20 is the one captured and stored by the mobile terminal 1 in the personal identification image processing.
In S62, the control unit 40 (face matching unit 43) of the identity verification server 4 receives the matching image and the face photograph image from the mobile terminal 1.

In S63, the control unit 40 (face matching unit 43) performs face matching processing for matching the matching image and the face photograph image. The control unit 40 performs the image matching process, and calculates, for example, the degree of matching between the matching image and the face photograph image as a score.
In S64, the control unit 40 (matching result transmitting unit 44) transmits the matching result to the mobile terminal 1. After that, the control unit 40 ends this processing.
In S65, the control unit 10 (the matching image processing unit 16) of the mobile terminal 1 receives the matching result. After that, the control unit 10 shifts the processing to S15 of FIG. 4, and performs the determination processing of determining whether or not the verification is successful, as described above.
When it is determined by the determination process that the authentication cannot be performed (NO in S15 of FIG. 4), it is desirable that the control unit 10 delete the facial photograph image and the saved image stored in the storage unit 20.

Thus, according to the first embodiment, the personal identification system 100 has the following effects.
(1) When performing a service using the mobile terminal 1, if the mobile terminal 1 has the camera 23, it is possible to perform identity verification using only the mobile terminal 1. Therefore, even if the mobile terminal 1 does not have a reading function such as a reader, it is convenient because the camera 23 that the general mobile terminal 1 may have can be used to confirm the identity. .. Then, the user can provide the user with an environment for verifying the identity anytime, anywhere without the hassle of going to a window or the like. In addition, security is high because the identification is performed using biometric information. Further, it is not necessary to register biometric information in advance such as a fingerprint, which is highly convenient.

(2) Identity verification by collating the face photograph image acquired from the identity verification document with the face image of the user taken by the user, and identity verification that acquires a saved image including both the identity verification document and the user's face. Is convenient because it can be performed in one flow.
(3) Only saved images that follow the instructions of the guide image are allowed. Then, the guide image shows the arrangement range of the personal identification document and the arrangement range of the face of the user, so that the guide image can be shown to the user in an easy-to-understand manner. Further, as the guide images, those having different arrangement positions of the personal identification images are randomly output. Therefore, it becomes a mechanism that cannot use images taken in advance, and it is possible to make it difficult to act fraudulently.

(Second embodiment)
In the second embodiment, a part of the process based on the personal identification document is performed by the personal identification server. In the following description, parts having the same functions as those of the first embodiment described above will be denoted by the same reference numerals or the same reference numerals at the end, and redundant description will be appropriately omitted.

FIG. 11 is an overall configuration diagram of the personal identification system 200 according to the second embodiment and functional block diagrams of the mobile terminal 201 and the personal identification server 204.
The identity verification system 200 includes a mobile terminal 201 and an identity verification server 204, and is communicably connected via a communication network N and a base station R (not shown). Although not shown, the personal identification system 200 is connected to the service providing server 7 via the communication network N.
The mobile terminal 201 includes a control unit 210, a storage unit 220, a camera 23, a touch panel display 25, and a communication interface unit 29.

The control unit 210 includes an application activation unit 11, an identity verification unit 212, an identity authentication unit 17, and a service start determination unit 18.
The identity verification unit 212 includes an identity verification image processing unit 213, a guide image output unit 14, a saved image processing unit 15, and a collation image processing unit 16.
The personal identification image processing unit 213 operates the camera 23 by the user to acquire a personal identification image. Then, the personal identification image processing unit 213 extracts a facial photograph image from the acquired personal identification image, and stores the facial photograph image in the storage unit 220. Further, the personal identification image processing unit 213 transmits the acquired personal identification image to the personal identification server 204.

  The storage unit 220 includes a program storage unit 221. The program storage unit 221 stores a transaction application 21a and an identity verification application 221b. The identity verification application 221b is a program for performing various functions of identity verification and identity authentication executed by the control unit 210 of the mobile terminal 201.

The identity verification server 204 includes a control unit 240, a storage unit 250, and a communication interface unit 59.
The control unit 240 includes an identity verification image analysis unit 241, a face verification unit 43, and a verification result transmission unit 44.
The personal identification image analysis unit 241 receives, from the mobile terminal 201, the personal identification image obtained by photographing the driver's license 3. Then, the personal identification image analysis unit 241 converts the characters included in the personal identification document into text by OCR (optical character recognition). After that, the personal identification image analysis unit 241 transmits the text-formed character string to the mobile terminal 201.
The storage unit 250 includes a program storage unit 251. The program storage unit 251 stores a personal identification program 251a, which is a program for performing various functions executed by the control unit 240 of the personal identification server 204.

Next, processing of the personal identification system 200 will be described.
The service start process and the personal identification process are the same as those in the first embodiment (FIGS. 4 and 5).
The personal identification image processing performed in the personal identification processing will be described with reference to FIG.
FIG. 12 is a flowchart showing the personal identification image processing in the personal identification system 200 according to the second embodiment.
The processing from S240 to S242 is the same as the processing from S40 to S42 of the first embodiment (FIG. 6).
In step S243, the control unit 210 of the mobile terminal 201 transmits the acquired personal identification image to the personal identification server 204.
The processes of S241 and S242 and the process of S243 do not have to be in order, and the processes of S241 and S243 may be performed in parallel, or the process of S243 may be performed before the process of S241. Good.

In S244, the control unit 240 of the identity verification server 204 receives the identity verification image.
In S245, the control unit 240 (personal identification image analysis unit 241) acquires character information in which the personal identification document included in the received personal identification image is converted into text.
In step S246, the control unit 240 transmits the acquired character information to the mobile terminal 201. After that, the control unit 240 ends this processing.

In step S247, the control unit 210 (personal identification unit 212) of the mobile terminal 201 receives the character information and causes the touch panel display 25 to display the received character information. The control unit 210 of the mobile terminal 201 may have a function of correcting the displayed character information. Then, the acquired character information can be transmitted to the service providing server 7, for example, when the identity can be confirmed. After that, the control unit 210 shifts the processing to the saved image processing (processing corresponding to S31 of FIG. 5) of the identity confirmation processing.
The stored image processing and the collation image processing are the same as those in the first embodiment (FIGS. 7 and 10).

As described above, according to the second embodiment, the personal identification system 200 has the following effects.
(1) The mobile terminal 201 transmits the acquired personal identification image to the personal identification server 204, so that the personal identification server 204 converts the character information included in the personal identification image into text and transmits it to the mobile terminal 201. Therefore, it is sufficient to use the textual character information for the name and the like required when the user uses the service, and the user can save the trouble of inputting.
(2) By making it possible to correct the character information received by the mobile terminal 201, even if there is an erroneous conversion or the name or address described in the identity verification document is changed, only the changed part It can be changed and can be made more convenient.

(Third Embodiment)
In the third embodiment, various photographing modes performed by a user using a mobile terminal, which are devised to improve security, will be described.

FIG. 13 is a functional block diagram of the mobile terminal 301 in the personal identification system 300 according to the third embodiment.
The personal identification system 300 includes a mobile terminal 301. Although not shown, in the personal identification system 300, the mobile terminal 301 is connected to the personal identification server 4 and the service providing server 7 via the communication network N and the base station R (not shown) (see FIG. 1). ..
The mobile terminal 301 includes a control unit 310, a storage unit 320, a camera 23, a touch panel display 25, and a communication interface unit 29.
The control unit 310 includes an application activation unit 11, a personal identification unit 312, a personal authentication unit 17, and a service start determination unit 18.

  The personal identification unit 312 includes a personal identification image processing unit 313 (first guide image output unit, personal identification image acquisition unit, first repeating unit, first image confirmation unit, face photograph image extraction unit), and a saved image. Processing unit 315 (second guide image output unit, saved image acquisition unit, second repeating unit, second image confirmation unit, saved image transmission unit), and collation image processing unit 316 (instruction output unit, collation image acquisition) Means, collation image transmitting means, third repeating means, third image confirming means).

  The personal identification image processing unit 313 causes the touch panel display 25 to output the document position guide image (first guide image). The document position guide image is an image that designates a shooting direction in which a face photograph can be confirmed and an arrangement position of the personal identification document including the size of the personal identification image. The user operates his / her own mobile terminal 301 and imprints his / her own identification document according to the guide position indicated by the document position guide image. Then, when the user operates the camera 23 to take a picture, the personal identification image processing unit 313 acquires a personal identification image which is an image of the personal identification document.

  The personal identification image processing unit 313 next outputs a document position guide image having a different shooting direction and arrangement position from the previous one, and performs processing for causing the user to acquire the personal identification image according to the document position guide image. The personal identification image processing unit 313 outputs the document position guide image and repeats a process of causing the user to photograph the personal identification document a plurality of times. Then, the personal identification image processing unit 313 stores the acquired personal identification image in the storage unit 320. Here, one of the document position guide images output by the personal identification image processing unit 313 is the shooting direction perpendicular to the surface of the personal identification document that includes the face photograph. It is for shooting documents.

  The personal identification image processing unit 313 extracts a facial photograph image from one of the stored personal identification images. At this time, it is preferable that the personal identification image processing unit 313 extracts the facial photograph image using the personal identification image obtained by photographing the personal identification document from the front of the personal identification document. By doing so, the control unit 310 does not need to perform additional processing such as correcting the distortion of the acquired face photograph image, for example. After that, the personal identification image processing unit 313 stores the extracted face photograph image in the storage unit 320.

  Further, the personal identification image processing unit 313 confirms the acquired personal identification image based on the document position guide image used at the time of acquisition. The personal identification image processing unit 313 analyzes the personal identification image and confirms that, for example, the personal identification document in which the facial photograph is posted is photographed from the designated photographing direction within the frame indicated by the document position guide image. To do.

  The saved image processing unit 315 operates the camera 23 by the user to acquire a saved image that is an image including both the identity verification document and the face of the user who possesses the identity verification document. At that time, the saved image processing unit 315 outputs the multi-position guide image (second guide image) to the touch panel display 25. The user operates his / her own mobile terminal 301 so that both the personal identification document and the face of the user who possesses the personal identification document fit within the guide positions indicated by the multi-position guide image. The saved image processing unit 315 may randomly acquire and output a plurality of position guide images from the storage unit 320 that stores a plurality of position guide images having different guide positions (arrangement positions) of the personal identification document.

The saved image processing unit 315 outputs a plurality of position guide images with different guide positions of the personal identification document, and repeats a process of acquiring the saved image according to the plurality of position guide images a plurality of times. Then, the saved image processing unit 315 stores the acquired saved image in the storage unit 320.
The saved image processing unit 315 confirms that the image of the identity verification document and the user's face image included in the acquired saved image are within the guide range of the multi-position guide image. Then, the saved image processing unit 315 stores the saved image in the storage unit 320 when the saved image can be confirmed.
Further, the saved image processing unit 315 may send the acquired saved image to the identity verification server 4.

The matching image processing unit 316 acquires a matching image used for matching, which is a face image of the user, by photographing its own face.
The matching image processing unit 316 first causes the touch panel display 25 to output a face shooting guide image (instruction information) for shooting the face in front of itself. The user operates his / her own portable terminal 301 to activate the in-camera 23a so that his / her face is oriented in the front direction, so that the user can fit within the guide position indicated by the face photographing guide image. Then, the collation image processing unit 316 acquires the collation image when the user operates the camera 23 to take a photograph.

The collation image processing unit 316 then causes the touch panel display 25 to output an instruction to change the orientation of the user's face, and outputs the instruction from, for example, a speaker (not shown) of the mobile terminal 301. When the direction of the face is changed, the user cannot visually recognize the touch panel display 25, and thus the voice instruction is effective. Then, for example, the collation image processing unit 316 outputs the countdown as a voice, and then controls the camera 23 at the shooting timing to acquire the collation image.
Note that the matching image processing unit 316 may perform the process of changing the orientation of the user's face in, for example, the up, down, left, and right directions and shooting the matching image a plurality of times.
Then, the matching image processing unit 316 stores the acquired matching image in the storage unit 320.

Further, the collation image processing unit 316 confirms whether or not the obtained collation image satisfies the photographing condition, that is, whether or not the collation image is in accordance with the instruction content output before that.
Further, the collation image processing unit 316 transmits the acquired plurality of collation images and the face photograph image stored in the storage unit 320 to the identity verification server 4.
It should be noted that the personal identification server 4 collates, for example, a collation image obtained by photographing the front face of the user with a facial photograph image, out of the plurality of collation images received from the mobile terminal 301.

  The storage unit 320 includes a program storage unit 321. The program storage unit 321 stores a transaction application 21a and an identity verification application 321b. The identity verification application 321b is a program for performing various functions of identity verification and identity authentication executed by the control unit 310 of the mobile terminal 301.

Next, the processing of the personal identification system 300 will be described.
The service start process and the personal identification process are the same as those in the first embodiment (FIGS. 4 and 5).
The personal identification image processing performed in the personal identification processing will be described with reference to FIG.
FIG. 14 is a flowchart showing the personal identification image processing in the personal identification system 300 according to the third embodiment.
FIG. 15 is a diagram showing a display example on the mobile terminal 301 according to the third embodiment.
FIG. 16 is a diagram showing an example of the document position guide image displayed on the mobile terminal 301 according to the third embodiment.

In S340 of FIG. 14, the control unit 310 (identity confirmation image processing unit 313) of the mobile terminal 301 activates the out-camera 23b and causes the touch panel display 25 to display a through image of the out-camera 23b. Further, the control unit 310 outputs a document position guide image 350 in which the front side of the identity verification document is in the shooting direction, as shown in FIG. 15A, to the touch panel display 25.
The document position guide image 350 shown in FIG. 15 (A) is an image including a document position area 350a for arranging an identity verification document and a shooting button 350b. The document position guide image 350 is a transparent image. This is for outputting together with the through image of the out camera 23b, and at least the document position area 350a needs to be transparent.

When the user arranges the identity verification document so as to be reflected in the document position area 350a and taps the photographing button 350b, in S341 of FIG. 14, the control unit 310 (identity verification image processing unit 313) causes the identity verification document to be verified. Get an image. Then, control unit 310 causes storage unit 320 to store the acquired identity verification image.
In S342, control unit 310 (identity confirmation image processing unit 313) restarts out-camera 23b and causes touch-panel display 25 to display the through image of out-camera 23b again. Then, the control unit 310 outputs, to the touch panel display 25, a document position guide image 351 including a side face of the personal identification document, for example, as shown in FIG.

The document position guide image 351 shown in FIG. 15 (B) is an image including a document position area 351a for arranging an identity verification document and a shooting button 351b. Further, like the document position guide image 350 described above, the document position guide image 351 is an image having transparency. The document position area 351a indicates the photographing direction from an oblique direction so that the face photograph of the personal identification document can be confirmed and the thickness of the long side of the personal identification document can be confirmed. Therefore, the document position area 351a is shown as a trapezoid with a long base.
Note that this is an example, and for example, the document position area 351a may not be indicated by a frame line, but may be indicated by lines indicating the positions of the two long sides of the personal identification document. The document position area 351a may be used to confirm the thickness of the short side of the personal identification document.

  When the user arranges the identity verification document so as to be reflected in the document position area 351a and taps the photographing button 351b, in S343 of FIG. 14, the control unit 310 (identity verification image processing unit 313) causes the identity verification Get an image. Then, control unit 310 causes storage unit 320 to store the acquired identity verification image.

In step S344, the control unit 310 (personal identification image processing unit 313) confirms the acquired personal identification image based on the document position guide image.
For example, the control unit 310 confirms that the identity verification document is within the document position area 350a and that the identity verification image can be acquired in a predetermined size, and that the acquired identity verification image is the identity verification document. Confirm that there is (for example, the front of the driver's license 3). The control unit 310 performs this confirmation on all the acquired personal identification images. Further, the control unit 310 confirms whether or not the resolution of the acquired personal identification image satisfies a predetermined condition. For example, if the amount of light is insufficient due to camera shake or the like, the resolution of the personal identification image does not satisfy the predetermined condition.

  It should be noted that the control unit 310 performs the processing from S340 again up to a predetermined number of retries when the confirmation cannot be confirmed by the confirmation of the personal identification image, that is, when the condition as the personal identification image is not satisfied. Good. In addition, for example, when the number of retries has exceeded the predetermined number, the control unit 310 may not perform the subsequent processing as an error.

In S345, the control unit 310 (identity confirmation image processing unit 313) extracts a facial photograph image from the identity confirmation image. The control unit 310 extracts, for example, a facial photograph image from the personal identification image obtained by photographing the personal identification document acquired in S341 from the front.
In step S <b> 346, the control unit 310 (personal identification image processing unit 313) stores the extracted face photograph image in the storage unit 320. After that, the control unit 310 performs the stored image processing (see S31 of FIG. 5) described below.

  In this example, first, the document position guide image 350 in which the front side of the personal identification document is in the photographing direction is output and photographed, and then the document position guide image 351 in different photographing directions and photographing positions is outputted and photographed. However, the present invention is not limited to this. The order is not limited as long as one of a plurality of times of photographing the personal identification document is the front of the personal identification document in the photographing direction. Further, the number of times of photographing is not limited to two, and may be three or more.

  Further, in the personal identification system 300, the document position guide images of various shooting directions and shooting positions are stored in the storage unit 320, and the control unit 310 randomly selects the document position guide images from the stored image. , May be output. The document position guide images at various shooting directions and shooting positions are, for example, a document position guide image 361 shown in FIG. 16 (A) to a document position guide image 364 shown in FIG. 16 (D).

The document position guide image 361 shown in FIG. 16 (A) is a shooting direction in which the thickness of the personal identification document can be photographed, and a document position area 361a for placing the personal identification document on the upper part of the screen at a short distance and photographing the same. Including.
The document position guide image 362 shown in FIG. 16B is a shooting direction in which the thickness of the personal identification document can be photographed, and a document position area 362a for placing the personal identification document at the bottom of the screen at a short distance and capturing the image. Including.
On the other hand, the document position guide image 363 shown in FIG. 16 (C) and the document position guide image 364 shown in FIG. 16 (D) are shooting directions in which the thickness of the personal identification document can be shot, and the upper or lower part of the screen at a long distance. Document position areas 363a and 364a for arranging an identity verification document and photographing it.
Note that the storage unit 320 may also store a document position guide image that is captured by arranging the document position area in the center of the screen.

  In addition, in this example, the person confirmation image is confirmed after the person confirmation document is photographed a plurality of times from different photographing directions, but the invention is not limited to this. The control unit 310 may check the personal identification image each time the personal identification image is acquired. In addition, the control unit 310 causes the user to confirm the personal identification image after acquiring the personal identification image, and stores the acquired personal identification image when the user inputs an instruction to save the confirmation image. May be re-enabled by.

Next, the stored image processing performed in the personal identification processing will be described with reference to FIG.
FIG. 17 is a flowchart showing stored image processing in the personal identification system 300 according to the third embodiment.
S350 is the same as the process of S50 of the first embodiment (FIG. 7).
In step S351, the control unit 310 (saved image processing unit 315) specifies one document position guide image from the plurality of document position guide images stored in the storage unit 320, and specifies the specified document position guide image, for example. , On the touch panel display 25. Here, the plurality of document position guide images stored in the storage unit 320 are different in the guide position of the personal identification document, and the guide position of the personal identification document does not overlap the guide position of the face image. There is something.
S352 is similar to the process of S50 of the first embodiment (FIG. 7).

In step S353, the control unit 310 (saved image processing unit 315) extracts the document position guide image from the plurality of document position guide images stored in the storage unit 320, except for the one document position guide image specified earlier. One is specified and the specified document position guide image is displayed on the touch panel display 25.
S354 is similar to the process of S352.
In S355, the control unit 310 (saved image processing unit 315) confirms the plurality of acquired saved images. Specifically, the control unit 310 confirms that the image of the identity verification document included in the acquired saved image and the face image of the user approximately match the guide range of each document position guide image.

  In step S356, the control unit 310 (saved image processing unit 315) stores the saved image in the storage unit 320 on condition that the saved image can be confirmed. Note that if the saved image cannot be confirmed, that is, if the image of the identity verification document included in the acquired saved image and the user's face image are not within the guide range of the guide image, an error occurs. The subsequent processing may not be performed, or the processing from S351 and S353 may be performed again. At that time, the control unit 310 may output a document position guide image different from the document position guide image output so far, or may output the document position guide image in error again.

  In step S <b> 357, the control unit 310 (saved image processing unit 315) transmits the saved image stored in the storage unit 320 to the identity verification server 4. By doing so, the identification server 4 may also perform confirmation processing on the saved image. After that, the control unit 310 performs the collation image process (see S32 of FIG. 5) described below.

Next, collation image processing performed in the personal identification processing will be described with reference to FIG.
FIG. 18 is a flowchart showing matching image processing in the personal identification system 300 according to the third embodiment.
FIG. 19 is a flowchart showing a collation image acquisition process in the mobile terminal 301 according to the third embodiment.
FIG. 20 is a diagram showing a display example on the mobile terminal 301 according to the third embodiment.
In S350 of FIG. 18, the control unit 310 (matching image processing unit 316) of the mobile terminal 301 performs a matching image acquisition process.

Here, the acquisition process of the matching image will be described with reference to FIG.
In S370 of FIG. 19, the control unit 310 (collation image processing unit 316) activates the in-camera 23a and causes the touch panel display 25 to display a through image of the in-camera 23a. Then, the control unit 310 outputs a face shooting guide image 370 as shown in FIG. 20A to the touch panel display 25, for example.
The face shooting guide image 370 shown in FIG. 20A is for acquiring a face image from the front of the user. The face shooting guide image 370 is an image including a face position area 370a that specifies the position of the user's own face.

  In S371 of FIG. 19, the control unit 310 (the matching image processing unit 316) acquires the matching image and stores it in the storage unit 320. To obtain the collation image, the user moves the position of the mobile terminal 301 so that his / her face is reflected in the face position area 370a, and taps a photographing button (not shown) so that the control unit 310 photographs. You may make it control and acquire a user's face image. In addition, the control unit 310 may control to automatically shoot after a predetermined time (for example, 5 seconds) has elapsed after outputting the face shooting guide image 370 to acquire the face image of the user. .. Further, the control unit 310 may be provided with a function of detecting blinking of the face and eyes, and the control unit 310 may be controlled to photograph after detecting the blinking to acquire the face image of the user.

  In step S372, the control unit 310 (matching image processing unit 316) outputs an instruction to change the face direction to the user. Here, the instruction from the control unit 310 may be displayed, for example, like the face shooting guide image 371 shown in FIG. The face shooting guide image 371 shown in FIG. 20B is directed to the right of the user, but the user's face may be directed to any of up, down, left, and right.

  In step S373, the control unit 310 (matching image processing unit 316) acquires the matching image and stores it in the storage unit 320. In this shooting, since the face of the user is not facing the touch panel display 25 of the mobile terminal 301, it is difficult for the user to tap a shooting button (not shown). Therefore, the control unit 310 controls to automatically shoot after counting a predetermined time (for example, 5 seconds) by voice after outputting the face shooting guide image 371, and acquires the face image of the user. You may. In addition, the control unit 310 automatically detects that the face is included in the face position area 371a within a predetermined time (for example, 5 seconds) after outputting the face shooting guide image 371. The face image of the user may be acquired by controlling the image capturing to be performed.

In S374, control unit 310 (matching image processing unit 316) confirms whether or not the acquired match image is in accordance with the corresponding instruction content. Then, if the confirmation is successful, control unit 310 moves the process to S361 in FIG. If the confirmation cannot be made, the control unit 310 may not perform the subsequent processing.
The processing from S361 to S365 in FIG. 18 is the same as the processing from S61 to S65 in the first embodiment (FIG. 10).

As described above, according to the third embodiment, the personal identification system 300 has the following effects.
(1) The mobile terminal 301 captures a personal identification document from a plurality of different photographing directions in which a facial photograph can be confirmed, and obtains a plurality of personal identification images. Then, at least one of the plurality of identity verification images is configured to be capable of confirming the thickness of the identity verification document. Therefore, it is possible to determine that the document is a true identity verification document by the identity verification image, and the security can be improved.
(2) The mobile terminal 301 is configured to output a document position guide image for acquiring a personal identification image. Then, the document position guide image is an image that specifies the shooting direction in which the face photograph can be confirmed and the arrangement position of the personal identification document including the size of the personal identification image. You can shoot a document. In addition, since the shooting direction and the arrangement position are specified, it becomes necessary to prepare an identity verification document on the spot to shoot, and it is possible to prevent an illegal act such as using an image prepared in advance.
(3) The mobile terminal 301 includes a device that changes the direction of the user's face based on the instruction information and captures a face image of the user. Therefore, it is necessary to take a picture according to the instruction information, and it is possible to prevent an illegal act such as using an image prepared in advance.
Also, when the orientation of the user's face is changed, voice instructions or automatic shooting are performed. Therefore, even when it is difficult for the user to perform an operation for taking a picture, it is possible to deal with the situation, and it is possible to provide the convenience.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. In addition, the effects described in the embodiments are merely enumeration of the most suitable effects generated by the present invention, and the effects according to the present invention are not limited to those described in the embodiments. It should be noted that the above-described embodiment and the modification described later can be appropriately combined and used, but detailed description thereof will be omitted.

(Variation)
(1) In each of the embodiments, the personal identification server performs image matching processing, and in the second embodiment, the personal identification server performs character recognition processing. However, the present invention is not limited to this. For example, an image matching server communicatively connected to a personal identification server and an OCR server are provided, face images are matched using the image matching server, and character recognition processing is performed using the OCR server. It may be.
(2) In each embodiment, the process using the identity verification server and the mobile terminal has been described, but the content of the process performed by the identity verification server and the mobile terminal is not limited to this. The process other than the image acquisition performed by the mobile terminal may be performed by the identity verification server, or the process performed by the identity verification server may be performed by the mobile terminal. For example, the confirmation process of each acquired image may be performed by the personal identification server. Further, the confirmation processing of each image may be performed by both the personal identification server and the mobile terminal.

(3) In each of the embodiments, as the personal identification processing, the processing of the personal identification image and then the processing of the saved image has been described as an example, but the order of the processing may be reversed.
(4) In each of the embodiments, the case where the transaction confirmation application is executed and then the personal identification application is activated has been described as an example, but the present invention is not limited to this. The identity verification application may be executed independently.
(5) In each of the embodiments, no particular processing is performed on the personal identification document included in the personal identification image and the personal identification document included in the saved image, but it may be verified whether they are the same. .. In addition, when a plurality of images are acquired in each process, it may be confirmed that the subjects are the same.

  (6) In the third embodiment, an example has been described in which a plurality of person confirmation images are acquired and a facial photograph image is extracted from the person confirmation images photographed from the front, but the present invention is not limited to this. A face photograph image may be extracted from the personal identification image acquired by photographing from the photographing direction in which the thickness of the personal identification document can be confirmed. In that case, for example, a known image correction technique may be used to correct distortion in an image captured from the front, and the image may be used for face matching processing. Further, using this correction technique, it may be confirmed that a plurality of personal identification images are images obtained by photographing the same personal identification document.

  (7) In the third embodiment, the guide image is output at the time of shooting. At that time, although the time from the output of the guide image to the shooting is not specifically described, the control unit does not perform the subsequent processing as an error if the shooting is not performed within the predetermined time. You can

  (8) Various services are applicable as services for performing identity verification and identity authentication in each embodiment. For example, in the case of banking business, it can be used for account opening and account linking processing, cash withdrawal processing using ATM, and the like. To use an ATM, the mobile terminal can be used as a cash card instead of the cash card by performing personal authentication on the mobile terminal, displaying a QR code (registered trademark) when the mobile terminal is authenticated, and causing the ATM to read it. You can In addition, in the case of an agent acting on behalf of the person at the counter, the agent receives information proving that the person has been authenticated, without having to prepare a power of attorney in advance. I'm done.

1,201,301 Mobile terminal 3 Driver's license 4,204 Identity verification server 7 Service providing server 10, 40, 210, 240, 310 Control section 12, 212, 312 Identity verification section 13, 213, 313 Identity verification image processing section 14 Guide image output unit 15,315 Saved image processing unit 16,316 Matching image processing unit 17 Personal authentication unit 18 Service start determination unit 20, 50, 220, 250, 320 Storage unit 21a Transaction application 21b, 221b, 321b Identity confirmation application 23 camera 25 touch panel display 43 face collation unit 44 collation result transmission unit 100, 200, 300 personal identification system 241 personal identification image analysis unit

Claims (17)

A mobile terminal having a photographing unit,
The data necessary to confirm the identity of the user at the start of the service is acquired using only the mobile terminal owned by the user,
An identity verification image acquisition means for acquiring an identity verification image, which is an image of an identity verification document on which a face photo is posted, from the photographing unit,
A saved image acquisition unit that acquires a saved image that is an image including both the identity verification document and the face of the holder who possesses the identity verification document from the imaging unit,
A collation image acquisition unit that acquires a collation image that is an image including the face of the holder from the photographing unit,
The personal identification image acquired by the personal identification image acquisition unit and the collation image acquired by the collation image acquisition unit, a collation image transmission unit that transmits to the personal identification server that performs personal identification using an image,
Mobile terminal equipped with. The mobile terminal according to claim 1,
The personal identification image acquisition unit is configured to capture a plurality of the personal identification documents from a plurality of different image capturing directions in which the facial photograph can be confirmed, including a plurality of capturing directions perpendicular to a face of the personal identification document including the facial photograph. The personal identification image of is acquired from the photographing unit,
The collation image transmitting unit transmits at least one or more of the personal identification images among the plurality of personal identification images acquired by the personal identification image acquisition unit, and the collation image to the personal identification server,
A mobile terminal characterized by. The mobile terminal according to claim 2,
At least one of the personal identification images among the plurality of personal identification images acquired by the personal identification image acquisition means is capable of confirming the thickness of the personal identification document together with the facial photograph,
A mobile terminal characterized by. In the mobile terminal according to claim 2 or 3,
A display unit for displaying an image of a shooting range by the shooting unit,
A first guide image output unit for outputting to the display unit a first guide image that specifies the shooting direction and that specifies the arrangement position of the identity verification document that includes the size of the identity verification image;
First repeating means for repeating the processing by the first guide image output means and the personal identification image acquisition means a predetermined number of times;
Equipped with
The first guide image output by the first guide image output means has different designation of the photographing direction and designation of the arrangement position each time it is output,
A mobile terminal characterized by. The mobile terminal according to claim 4,
A first image confirmation unit for confirming the personal identification image acquired by the personal identification image acquisition unit based on the first guide image output by the first guide image output unit;
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 5,
A display unit for displaying an image of a shooting range by the shooting unit,
Second guide image output means for outputting to the display section a second guide image showing the personal identification document and the arrangement range of the face of the holder.
Be equipped with,
A mobile terminal characterized by. The mobile terminal according to claim 6,
The second guide image output means shows the arrangement range of the personal identification document at a random position not overlapping the position of the face and in a range of a predetermined size or more. Is output to the display unit,
A mobile terminal characterized by. The mobile terminal according to claim 7,
A second repeating unit that repeats the processing by the second guide image output unit and the saved image obtaining unit a predetermined number of times,
The second guide image output by the second guide image output means has a different arrangement position indicating the arrangement range of the personal identification document every time the second guide image is output.
A mobile terminal characterized by. The mobile terminal according to claim 8,
A plurality of saved images acquired by the saved image acquisition means, and a saved image transmission means for sending to the identity verification server,
A mobile terminal characterized by. The mobile terminal according to any one of claims 6 to 9,
The second guide image output by the second guide image output means regarding the positions of the image of the personal identification document included in the saved image acquired by the saved image acquisition means and the face image of the holder. A second image checking means for checking based on
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 10,
An instruction output unit for outputting instruction information that is an instruction regarding the orientation of the face of the holder,
The collation image acquisition means, after outputting the instruction information by the instruction output means, acquires the collation image in response to satisfaction of a photographing condition,
A mobile terminal characterized by. The mobile terminal according to claim 11,
A third repeating means for repeating the processing by the instruction output means and the collation image acquisition means a plurality of times,
One of the instruction information output by the instruction output unit is related to an instruction to make the face of the holder face forward.
A mobile terminal characterized by. The mobile terminal according to claim 11 or 12,
A third image confirmation unit for confirming the collation image acquired by the collation image acquisition unit based on the instruction information output by the instruction output unit;
A mobile terminal characterized by. The mobile terminal according to any one of claims 1 to 13,
A face photograph image extracting means for extracting a face photograph image from the personal identification image acquired by the personal identification image acquiring means,
The collation image transmitting means transmits, to the personal identification server, the face photograph image extracted by the face photograph image extracting means and the collation image,
A mobile terminal characterized by. A personal identification server that is communicably connected to the mobile terminal according to any one of claims 1 to 14 and performs personal identification processing,
Face collating means for collating the collation image received from the mobile terminal with the face photograph image included in the personal identification image,
Collating result information by the face collating means, result transmitting means for transmitting to the mobile terminal,
An identity verification server equipped with. The mobile terminal according to any one of claims 1 to 14,
An identity verification server according to claim 15;
An identity verification system including
The mobile terminal is
When the identity verification is not performed, the processing by the identity verification image acquisition means, the saved image acquisition means, the verification image acquisition means and the verification image transmission means is executed,
Based on the collation result information, a service start determination that determines whether to start service processing with a service providing server that is communicably connected to the mobile terminal and the identity verification server and performs service processing Providing means,
A personal identification system characterized by. A computer possessed by the user, which has an image capturing unit and is used to confirm the identity of the user when the service starts,
An identity verification image acquisition means for acquiring an identity verification image, which is an image of an identity verification document on which a face photo is posted, from the photographing unit,
A saved image acquisition unit that acquires a saved image that is an image including both the identity verification document and the face of the holder who possesses the identity verification document from the imaging unit,
A collation image acquisition unit that acquires a collation image that is an image including the face of the holder from the photographing unit,
A matching image transmitting unit that transmits the personal identification image acquired by the personal identification image acquisition unit and the collation image acquired by the collation image acquisition unit to a personal identification server that performs personal identification using an image,
And a program to make it work.

Images