JP7024904B2 - Terminal devices, methods for controlling terminal devices, and programs - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act September 3, 2019 Publication on the applicant's official website "https://www.dnp.co.jp/biz/solution/products/dateil/1192718_1567.html" [Publications, etc. ] October 24, 2019 Published by exhibition at the International Financial Information Technology Exhibition "FIT2019" [Publications, etc.] October 25, 2019 Applicant's official website "https://www.dnp.co.jp/news/dateil" Publication on "/1193094_1587.html" [Publications, etc.] January 20, 2020 Publication on the applicant's official website "https://www.dnp.co.jp/news/dateil/10157643_158.html" [Publications, etc.] January 24, 2020 release on the payment navi card information portal site "https://paymentnavi.com/cardnavi/90849.html" [Publications, etc.] January 18, 2021 Mizuho Bank, Ltd. eKYC News Release "https: //Www.mizuhobank.co.jp/release/pdf/20210118release_jp.pdf "

The present disclosure relates to a terminal device that performs identity verification by an identification card online, a method for controlling the terminal device, and a program.

Identity verification is required for contracts for services that exchange funds, such as opening a bank account. This identity verification generally authenticates the identity by mailing a copy of the identification card and the service provider confirming the copy.

In recent years, when performing this identity verification, a method has been known in which a user takes a picture of his / her identification card with a terminal device such as a smartphone, sends the image to a service provider, and completes the identity verification online. (For example, Patent Document 1). By completing the identity verification online in this way, the convenience of the user can be improved.

Japanese Unexamined Patent Publication No. 2015-45910

However, with the method of completing identity verification online by sending an image of an ID card as described above, an image of an illegally obtained ID card or an image of a reused ID card may be sent. Yes, fraud cannot be prevented.

Therefore, when taking an identification card, a method is conceivable in which a guide frame is randomly displayed on the screen of the terminal device, the identification card is taken along the guide frame, and the image is transmitted. According to this method, it can be presumed that the user's identification card is taken on the spot instead of the fraudulent image prepared in advance, thereby ensuring the liveness and preventing fraud. Can be done.

In addition, an illegal image of an ID card is often an image of only one side of the ID card, such as the surface of the ID card on which a face photograph is posted. Therefore, fraud can be prevented by taking a picture including not only the face photo of the ID card but also the thickness (side surface) of the ID card.

However, if you want to take a picture including not only one side of the ID card on which the face photo is posted but also the thickness of the ID card, it is necessary to take a picture at an angle to the ID card. In addition, since the guide frame is displayed randomly, it may be difficult to take a picture of the ID card along the guide frame.

The present disclosure has been made to solve the above-mentioned problems, and an object thereof is a terminal device capable of facilitating identity verification while ensuring liveness, a method for controlling the terminal device, and a method for controlling the terminal device. To provide a program.

The terminal device of the present disclosure includes an imaging unit that captures an image including an identification card, a display unit that displays an image in an imaging region of the imaging means on a screen, and a processing unit that controls the imaging of the image. The display unit displays a trapezoidal guide frame in the screen in a random display mode, and the processing unit takes an image that the contour including the thickness of the identification card is along the guide frame. It is characterized by having a determination unit for determining whether or not the conditions are satisfied, and a imaging control unit for causing the imaging unit to capture the image when the determination unit determines that the imaging conditions are satisfied. do.

The method of controlling the terminal device of the present disclosure controls a terminal device including an image pickup means for capturing an image including an identification card and a display unit for displaying an image of an image capture area of the image pickup means on a screen. It is said that the guide frame display step of displaying a trapezoidal guide frame in a random display mode on the screen of the display unit and the outline including the thickness of the identification card are along the guide frame. It is characterized by including a determination step for determining whether or not the imaging conditions are satisfied, and a imaging control step for causing the imaging means to capture the image when the determination step determines that the imaging conditions are satisfied. And.

The program of the present disclosure is a program executed on a terminal device including an imaging means for capturing an image including an identification card, a display unit for displaying an image of an imaging area of the imaging means on a screen, and a computer. A guide frame display step for causing the computer to display a trapezoidal guide frame in a random display mode on the screen of the display unit, and a contour including the thickness of the identification card are along the guide frame. A determination step for determining whether or not the imaging condition is satisfied, and a imaging control step for causing the imaging means to capture the image when the determination step determines that the imaging condition is satisfied are executed. It is characterized by that.

The terminal device of the present disclosure includes an image pickup unit that captures an image including an identification card, a display unit that displays an image in a capture area of the image pickup means on a screen, and a processing unit that controls the capture of the image. The display unit displays a guide frame having the shape of the ID card viewed from a direction other than the direction perpendicular to the ID card in the screen in one of a plurality of display modes. The processing unit determines whether or not the shooting condition that the contour including the thickness of the ID card satisfies the shooting condition along the guide frame, and the judgment unit satisfies the shooting condition. It is characterized by having a photographing control unit which causes the imaging means to capture the image when it is determined.

The method of controlling the terminal device of the present disclosure controls a terminal device including an image pickup means for capturing an image including an identification card and a display unit for displaying an image in a capture area of the image pickup means on a screen. It is a method, and a guide frame having a shape seen from a direction other than the direction perpendicular to the identification card is displayed on the screen of the display unit in one of a plurality of display modes. The shooting condition is satisfied by the guide frame display step, the determination step of determining whether or not the contour including the thickness of the identification card satisfies the shooting condition of being along the guide frame, and the determination step. It is characterized by comprising a photographing control step for causing the imaging means to capture the image when it is determined that the image has been taken.

The program of the present disclosure is a program executed on a terminal device including an imaging means for capturing an image including an identification card, a display unit for displaying an image of an imaging area of the imaging means on a screen, and a computer. Therefore, on the computer, a guide frame having a shape obtained by viewing the identification card from a direction other than the direction perpendicular to the identification card is displayed on the screen of the display unit in one of a plurality of display modes. The guide frame display step to be displayed in the embodiment, the determination step of determining whether or not the contour including the thickness of the identification card satisfies the imaging condition of being along the guide frame, and the imaging condition by the determination step. It is characterized in that the imaging control step of causing the imaging means to capture the image when it is determined that the above conditions are satisfied.

According to the present disclosure, it is possible to obtain a terminal device, a method for controlling the terminal device, and a program capable of facilitating identity verification while ensuring liveness.

It is a figure which shows the structure of the system to which the terminal apparatus which concerns on embodiment of this invention is applied. It is a functional block diagram of the terminal apparatus which concerns on embodiment. The guide frame when (size, position, type) = (large, center, a) of the guide frame is shown. The guide frame when (size, position, type) = (small, top, b) of the guide frame is shown. The guide frame when (size, position, type) = (large, bottom, a) of the guide frame is shown. It is a figure for demonstrating the determination process by a determination unit. It is a figure which shows the screen when the identification card does not follow the guide frame. It is a figure which shows the screen when the identification card is along the guide frame. It is a figure which shows the extracted image. It is a figure which shows the extracted image which applied the masking. It is a figure which shows the table which the size, position, and the type of a guide frame are associated. It is a flowchart which shows the operation of the terminal apparatus which concerns on embodiment. (A) It is a figure which shows the terminal apparatus in the state which the top screen is displayed. (B) Confirmation item It is a figure which shows the terminal device in the state which the consent screen is displayed. (A) It is a figure which shows the terminal device in the state which the caution screen at the time of shooting is displayed. (B) It is a figure which shows the terminal device in the state which the identification card surface photography screen is displayed. It is a figure which shows the terminal device in the state which the shooting result confirmation screen is displayed. It is a flowchart which shows the operation of a server. It is a flowchart which shows the operation which the terminal apparatus which concerns on embodiment takes a picture of an identification card from an angle.

[overall structure]
FIG. 1 is a diagram showing a configuration of a system to which the terminal device according to the embodiment of the present invention is applied. As shown in FIG. 1, the system 1 includes a server 2, a network 3, and a terminal device 4, and the server 2 and the terminal device 4 are connected via the network 3. As shown in FIG. 1, a plurality of terminal devices 4 may be connected to the server 2.

The server 2 is used for identity verification before providing a predetermined service to a user. The prescribed service is a service for exchanging funds, and requires identification by identification card before receiving the service. In the present embodiment, the predetermined service is a bank account opening and fund deposit / withdrawal service using a bank account, and the server 2 is used for identity verification. As will be described later, the server 2 generates a random value and transmits the random value to the terminal device 4 via the network 3.

The network 3 is a communication network that provides a connection path that enables data transmission / reception between the server 2 and the terminal device 4. The network 3 is a wired or wireless network and is composed of one or a plurality of networks. Examples of the network include, but are not limited to, a local area network (LAN), a wide area network (WAN), the Internet, a mobile communication network, and the like, and a known network can be used.

The terminal device 4 is an information processing device used by a user, and includes a CPU, a memory (RAM), and a storage device (flash ROM, etc.) connected to each other by a bus (none of them are shown). .. Typically, an OS and an application program are stored in a storage device, and when the terminal device 4 according to the embodiment is operated, they are appropriately read into a memory and executed by the CPU, whereby the embodiment Various functions including the function of the terminal device 4 according to the above are executed. Examples of the terminal device 4 include, but are not limited to, smartphones, mobile phones, computers equipped with displays and cameras, wearable terminals, and the like.

In the present embodiment, the identity verification necessary for opening a bank account is performed online using the server 2 and the terminal device 4. Specifically, an identification card including its thickness is photographed using the terminal device 4, the photographed image is transmitted to the server 2 via the network 3, and the service provider of the server 2 uses the image. Confirm your identity.

[Detailed configuration]
FIG. 2 is a functional block diagram of the terminal device 4 according to the embodiment. As shown in FIG. 2, the terminal device 4 includes an image pickup means 41, a display unit 42, a processing unit 43, a storage unit 44, and a communication unit 45.

The image pickup means 41 is a camera that captures an image including an image pickup element such as a CMOS image sensor. The image pickup means 41 captures an image including an identification card. The ID card is, for example, a driver's license, a passport, a Basic Resident Register card, an Individual Number Card, or other identification documents with a photo of the person's face. The ID card may be a health insurance card, residence card, special permanent resident certificate, pension certificate, disability certificate, notification card, copy of resident's card, seal registration certificate, or a photo of your face. It may be an identity verification document that has not been submitted. Moreover, since the identification card is a thing, it has a thickness. That is, even if the identification card is thin like a piece of paper, it has a thickness if it is the actual product. The thickness is formed by the sides of the ID card (especially the side without the description of proof items such as letters or facial photographs), and if the ID card is as thin as a piece of paper, for example, the edge of the ID card. Is formed by. The identification card has a rectangular shape (rectangular shape) in the present embodiment, but may have a square shape such as a square or a rhombus, or may have a shape other than a square shape such as a round shape. Is also good.

The display unit 42 has a screen such as an organic EL or a liquid crystal display, and displays an image of the image pickup region of the image pickup means 41 on the screen. The display unit 42 displays the trapezoidal guide frame F in a random display mode on the screen. The guide frame F is a frame for accommodating the image of the identification card when the image of the identification card necessary for identity verification is taken. The guide frame F may have a shape as long as the identification card is viewed from a direction other than the vertical direction with respect to the identification card. In other words, the guide frame F has a shape in which the identification card is viewed from an oblique direction with respect to one side of the identification card. The shape of the ID card viewed from a direction other than the direction perpendicular to the ID card includes the shape of the ID card viewed from the direction perpendicular to the ID card, that is, the shape of the ID card itself. do not have. This is to take a picture of the ID card including the thickness of the ID card. The guide frame F can be, for example, a quadrangular shape, a trapezoidal shape, a round shape, or an elliptical shape, and in the present embodiment, the guide frame F has a trapezoidal shape. Further, when the rectangular identification card is stored along the trapezoidal guide frame F and photographed, the obtained image includes the thickness of the identification card (see FIG. 8). That is, the user uses the image pickup means 41 of the terminal device 4 to take an image from an oblique angle with respect to the identification card. The random display mode of the guide frame F is a mode in which at least one of the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F is randomly determined. For example, if the size of the guide frame F is two types, large and small, the position is three types of the upper side, the center, and the lower side in the screen, and the trapezoidal shape type is two types a and b, the guide frame F of 12 patterns One guide frame F is randomly displayed from the inside. The type of trapezoidal shape of the guide frame F is the type of the identification card projected on the screen of the display unit 42 when the surface is photographed at a predetermined shooting angle θ with respect to the perpendicular line perpendicular to the surface (for example, the surface) of the identification card. Shows a trapezoidal shape. Type a is a trapezoid with an internal angle of 60 ° projected when the shooting angle θ = α, and type b is a trapezoid with an internal angle of 45 ° projected when the shooting angle θ = β. That is, since the shooting angle β is larger than the shooting angle α (α <β), the height of the guide frame F of the type b is lower than that of the type a if the length of the lower base is the same. FIG. 3 shows the guide frame F when (size, position, type) = (large, center, a) of the guide frame F, and FIG. 4 shows the guide frame F (size, position, type) = (size, position, type). The guide frame F in the case of (small, top, b) is shown, and FIG. 5 shows the guide frame F in the case of (size, position, type) = (large, bottom, a) of the guide frame F.

In the present embodiment, the display unit 42 displays the guide frame F on the screen in a random display mode, but the display unit 42 displays the guide frame F in one of a plurality of display modes. It may be displayed. The display of the guide frame F in one of the plurality of display modes includes the display of the guide frame F in the random display mode of the guide frame F as described above. The selection of one of the plurality of display modes may be irregular (random) or may have regularity. The plurality of display modes may be predetermined by a person other than the user of the terminal device 4, such as the administrator of the terminal device 4.

The display unit 42 may display the guidance G that guides the user so that the contour including the thickness of the identification card can be matched with the guide frame F. Guidance G includes letters, figures, symbols, or a combination of two or more of these. For example, as shown in FIG. 7, when the image of the identification card C is shifted to the left with respect to the guide frame F, a message prompting the terminal device 4 (imaging means 41) to be shifted to the left is displayed. If the image of the ID card is too small, a message prompting the terminal device 4 (imaging means 41) to come closer to the ID card is displayed. Examples of the figures and symbols include arrows indicating the direction in which the terminal device 4 (imaging means 41) is moved. Guidance G may display a message so that the background of the ID card is not the same color as or similar to the color of the ID card. As a result, the boundary between the background and the identification card can be easily detected by the processing unit 43 (for example, the determination unit 431 and the image extraction unit 433 described later). Guidance G may display a message to rotate the orientation of the ID card. In one example, a message may be displayed prompting the ID card to be rotated to fit the guide frame F, for example, left or right. In another example, a photo of the ID card or a message prompting you to turn the ID card by rotating it so that the characters on the ID card are turned upside down is displayed. Is also good. As a result, it becomes necessary for the user of the terminal device 4 to respond in real time, so that live performance can be ensured and fraudulent identity verification can be prevented.

The processing unit 43 is configured by, for example, executing an application program by a CPU, and includes a determination unit 431, a shooting control unit 432, an image extraction unit 433, and a display control unit 434. The determination unit 431 determines whether or not the shooting conditions are satisfied. This shooting condition is a condition that the contour E including the thickness of the ID card is along the guide frame F (see, for example, FIG. 8), for example, the contour E of the ID card C is on the guide frame F. The case where the contour E of the identification card C is parallel to the guide frame F and fits within the guide frame F is included, but the present invention is not limited to these. For example, the contour E of the identification card C does not necessarily have to be parallel to the guide frame F, and as shown in FIG. 6, if it fits in the inner and outer margin areas including the guide frame F, the contour of the identification card C does not necessarily have to be parallel to the guide frame F. May be determined to be along the guide frame F. The margin area is surrounded by a guide frame F that is one size larger than the guide frame F and a first frame F1 that is similar to the guide frame F, and a guide frame F that is one size smaller than the guide frame F and a second frame F2 that is similar to the guide frame F. This is an area including the border of F. In the present embodiment, the determination unit 431 has an identification card having a trapezoidal shape on the screen of the display unit 42 taken from an angle, and the area of the identification card is within a predetermined area, or the identification unit has the trapezoidal shape. Whether or not the above shooting conditions are satisfied is determined based on whether or not the internal angle of the certificate is within a predetermined angle range. This area and internal angle can be obtained, for example, by extracting the outline of the ID card from the image of the ID card.

The shooting control unit 432 includes, for example, a CPU and controls the image pickup means 41. Specifically, as shown in FIG. 8, the photographing control unit 432 causes the imaging means 41 to acquire an image including an identification card when the determination unit 431 determines that the above imaging conditions are satisfied. That is, the imaging means 41 automatically captures the identification card on condition that the above imaging conditions are satisfied.

The image extraction unit 433 is configured to include, for example, a CPU, and extracts the inside of the contour E including the thickness of the identification card C from the image (see FIG. 8) taken by the image pickup control unit 432 and the image pickup means 41. Generate an extracted image (see FIG. 9). The extracted image is an image including the contour E including the thickness of the identification card C and the area inside the contour E. The extracted image may be generated by deleting the upper and lower backgrounds in FIG. The extracted image is, for example, an image including the front surface S1 and the side surface S2 having a face photograph of the identification card C (see FIG. 9), or an image including the back surface and the side surface of the identification card.

As shown in FIG. 10, the image extraction unit 433 masks a predetermined region R in the extracted image. Masking is a process of filling a predetermined area R, for example, blackening the predetermined area R. The predetermined area R is an area in which information that is not necessarily necessary for identity verification is described, and is, for example, an area in which sensitive information is described. If the ID card is a driver's license, it is the area between the place where the expiration date of the driver's license is written and the place where the driver's license number is written.

The display control unit 434 includes, for example, a CPU, and controls the display unit 42. The display control unit 434 determines the guide frame F to be displayed on the display unit 42, and causes the display unit 42 to display the guide frame F. In the present embodiment, the terminal device 4 receives a random value from the server 2 which is an external device, and the display control unit 434 determines the guide frame F to be displayed on the display unit 42 based on the random value.

Specifically, the storage unit 44 stores a table in which the size, position, and type of trapezoidal shape of the guide frame F in the screen are linked. For example, assuming that the size of the guide frame F is two types, large and small, the position is three types of the upper side, the center, and the lower side in the screen, and the trapezoidal shape type is two types a and b, as shown in FIG. The table has three degrees of freedom and has a total of 12 patterns of information (size, position, type) of the guide frame F. The display control unit 434 determines the guide frame F to be displayed on the display unit 42 from the 12 patterns of the guide frame F.

For example, when a predetermined numerical range that does not overlap with each other is associated with the table, the display control unit 434 determines which pattern the random value from the obtained server 2 belongs to. , The information of the guide frame F of the corresponding pattern is read from the storage unit 44, and the corresponding guide frame F is displayed on the display unit 42. Alternatively, when a predetermined numerical range is associated with each type of the guide frame F, the size, the position, and the trapezoidal shape, the size of the guide frame F obtained from the server 2 is used on the table. By determining the numerical range to which each random value for the position and the type of trapezoid belongs, the information of the guide frame F of the corresponding pattern is read from the storage unit 44, and one corresponding guide frame F is displayed on the display unit 42. ..

When the determination unit 431 determines that the shooting conditions are not satisfied, the display control unit 434 sets the user so that the outline of the identification card can be matched with the guide frame F as shown in FIG. 7. The guidance G to be guided is displayed on the display unit 42.

The display control unit 434 extracts the outline of the identification card from the image of the identification card obtained by the image pickup means 41, compares the outline on the screen with the guide frame F, and thereby, the guide frame in the screen. The action for eliminating the deviation of the outline of the identification card with respect to F is sought.

For example, the display control unit 434 calculates the deviation between the outline of the identification card and the center position of the guide frame F. When the center position of the outline of the identification card is shifted to the left with respect to the center position of the guide frame F, the guidance G for shifting the terminal device 4 (imaging means 41) to the left is displayed. Further, the display control unit 434 compares the area in the contour of the identification card with the area in the guide frame F, and if the area in the contour of the identification card is larger than the area in the guide frame F, the terminal device. 4 (imaging means 41) determines that the ID card is too close to the ID card, displays guidance G prompting the user to move away from the ID card, and conversely, the area within the contour of the ID card is within the guide frame F. If it is smaller than the area, it is determined that the terminal device 4 (imaging means 41) is too far from the identification card, and guidance G prompting the user to approach the identification card is displayed. Further, the display control unit 434 has an internal angle of the trapezoidal contour of the identification card (hereinafter, also referred to as an internal angle of the contour) and an internal angle of the corresponding trapezoidal guide frame F (hereinafter, also referred to as an internal angle of the guide frame F). When the internal angle of the contour is larger than the internal angle of the guide frame F, it is determined that the identification card and the terminal device 4 are closer to parallel to each other, and the identification card is not checked. Guidance G for tilting the terminal device 4 so as to be tilted is displayed. On the contrary, when the internal angle of the contour is smaller than the internal angle of the guide frame F, the terminal device 4 is tilted too much with respect to the identification card. Is determined, and the guidance G for tilting the terminal device 4 so as to be more parallel to the identification card is displayed.

The storage unit 44 is a storage device such as an HDD, SSD, and flash memory, and information necessary for causing the CPU to execute a function as the processing unit 43, such as a program and the table, is stored in advance. The program is a program that causes the terminal device 4 to execute the function as the processing unit 43.

The communication unit 45 is an interface for transmitting and receiving various data via the network 3. The communication unit 45 transmits / receives data to / from the server 2 which is an external device. For example, the communication unit 45 receives a random value from the server 2. When the display control unit 434 determines the size, position, and trapezoidal shape of the guide frame F, random values for the size, position, and trapezoidal shape of the guide frame F, that is, three random values. Receive. Further, the communication unit 45 transmits the extracted image to the server 2.

[motion]
The operation of the terminal device 4 and the server 2 will be described with reference to FIGS. 12 to 17. FIG. 12 is a flowchart showing the operation of the terminal device.

Here, as an example of identity verification, a case of applying for opening a bank account using a terminal device 4 which is a smartphone will be described. An application program for account application is incorporated in the terminal device 4 of the applicant. In order to incorporate this application program into the terminal device 4, for example, a predetermined server is accessed via a network, and a dedicated application program is downloaded as a so-called application for applying for an account. This server may be a server different from the server 2, or the server 2 itself may provide the application program to the terminal device 4.

As shown in FIG. 12, when the application program for account application is started by the applicant (step S01), the CPU of the terminal device 4 causes the display unit 42 to display the top screen (step S02). FIG. 13A is a diagram showing a terminal device 4 in a state where the top screen is displayed. As shown in FIG. 13 (a), two buttons "open an account" and "restart application" are displayed on the top screen. The applicant can tap these two buttons to give instructions. The tap means an operation of touching the screen by tapping the touch panel once. By tapping a button displayed on the screen, you can give an instruction corresponding to that button. Since the terminal device 4 is a smartphone here, the touch panel serves as an input means for accepting taps.

The "Resume application" button is used to resume the application process when it is temporarily suspended. When the "Resume application" button is tapped by the applicant, the information stored in the storage unit 44 as part or all of the information necessary for opening an account is read out, the corresponding screen is displayed, and the application is made. Resume processing.

The "Open Account" button is used to start the process of opening an account. When the "Open an account" button is tapped by the applicant, the CPU of the terminal device 4 causes the display unit 42 to display the confirmation item consent screen (step S03). As shown in FIG. 13B, on the confirmation item consent screen, an explanation of the confirmation item is displayed at the upper part of the screen. Further, at the top of the screen, a "back" button for returning to the top screen shown in FIG. 13A is displayed. At the bottom, there are links about the privacy policy, application handling rules, and statements / promises of non-antisocial forces, and you can check the linked contents by clicking the icon on the right side of each. ing. At the top, there is a square checkbox if you agree. Although not shown, the "Apply" button is displayed by scrolling the screen. Check by tapping the displayed check box. When the "Apply" button is tapped by the applicant with the check box checked, the CPU of the terminal device 4 causes the display unit 42 to display a caution screen at the time of shooting (step S04). The display of the caution screen at the time of shooting will be done in exchange for consent of various confirmation items.

FIG. 14A is a diagram showing a terminal device 4 in a state where a caution screen at the time of shooting is displayed. As shown in FIG. 14A, on the shooting caution screen, an explanation of precautions during shooting is displayed at the upper part of the screen. Further, at the top of the screen, a "back" button for returning to the confirmation item consent screen shown in FIG. 13 (b) is displayed. A "Next" button is displayed at the bottom of the screen. When the "Next" button is tapped by the applicant, the CPU of the terminal device 4 activates the image pickup means 41. As a result, the screen of the display unit 42 is switched to the shooting screen through the lens of the imaging means 41, and a frame line for positioning the identification card is displayed on the shooting screen.

FIG. 14B is a diagram showing a terminal device in a state where the identification card surface photographing screen is displayed. As shown in FIG. 14B, when switching to the shooting screen, the CPU of the terminal device 4 switches to a horizontally long screen format for display. This is because when taking a driver's license as an identification card, it is more suitable to be horizontally long. As shown in FIG. 14B, a "back" button for returning to the shooting caution screen is displayed at the top of the identification card surface shooting screen. Then, a "shooting" button is displayed on the right side of the screen. The frame line for positioning the driver's license may be in any form, but in the example of FIG. 14B, it is formed by four broken lines constituting the rectangle. In this state, when the applicant aligns the driver's license with the position of the frame line and taps the "shooting" button, the imaging means 41 takes a picture and acquires the shot image (step S05). Specifically, the license surface image is temporarily stored in the memory of the terminal device 4.

Then, the CPU of the terminal device 4 causes the display unit 42 to display the shooting result confirmation screen on the surface of the driver's license using the acquired license surface image (step S06). FIG. 15 is a diagram showing a terminal device in a state where a shooting result confirmation screen is displayed. As shown in FIG. 15, the driver's license surface image is displayed in the center of the shooting result confirmation screen. Also, on the left side of the top of the screen, a "back" button for returning to the driver's license surface shooting screen is displayed. In addition, a "Next" button is displayed on the right side of the top of the screen.

Further, the CPU of the terminal device 4 extracts the face image included in the driver's license surface image which is the face image of the identification card (step S07), and stores the face image extracted in the terminal device 4 in a memory or the like (step). S08).

When the applicant confirms the license surface image which is the shooting result on the shooting result confirmation screen and the "Next" button on the shooting result confirmation screen is tapped by the applicant, the CPU of the terminal device 4 is used as an imaging means. Start 41. Then, the screen of the display unit 42 is switched to the shooting screen through the lens of the imaging means 41, and a frame line for positioning the back surface of the driver's license is displayed on the shooting screen. That is, when the image pickup means 41 is activated, the screen of the display unit 42 is switched to the identification card back side shooting screen.

The back side shooting screen of the ID card is not shown, but it is a horizontally long screen similar to the front side shooting screen of the ID card shown in FIG. 14 (b), and the "Back" button and the "Shooting" button are displayed. To. In addition, a frame line for positioning the driver's license is displayed in the central portion. In this state, when the applicant aligns the back surface of the driver's license with the position of the frame line and taps the "shooting" button, the imaging means 41 takes a picture and acquires the shot image (step S09). The CPU of the terminal device 4 extracts the portion corresponding to the frame line from the obtained image and acquires it as the back side image of the driver's license. Specifically, the image on the back side of the driver's license is temporarily stored in the memory of the terminal device 4.

The CPU of the terminal device 4 causes the display unit 42 to display a shooting result confirmation screen on the back surface (step S10). Although not shown, the back side shooting result confirmation screen is a horizontally long screen similar to the front side shooting result confirmation screen shown in FIG. 15, and the left side at the top of the screen is for returning to the back side shooting screen. A "Back" button is displayed, and a "Next" button is displayed on the upper right side of the screen. When the "Next" button is tapped by the applicant, the CPU of the terminal device 4 activates the image pickup means 41 and photographs the identification card from an angle (step S11). The details of this oblique shooting will be described later.

After photographing the identification card from an angle, the CPU of the terminal device 4 activates the in-camera possessed by the terminal device 4 (step S12), and photographs the face of the applicant by this in-camera (step S13). Then, the terminal device 4 performs face recognition processing of the captured image taken by the in-camera by, for example, the processing unit 43 (step S14), and detects the matching face image (step S15). The face recognition process of a photographed image is a process of extracting a person's face included in the photographed image from the photographed image. The collation image is an image including the face image of the applicant taken by the in-camera. The CPU of the terminal device 4 may store the obtained collation face image in the memory of the terminal device 4. In addition, when taking a picture of the applicant's face with the in-camera, in order to ensure liveliness, the applicant himself gives instructions to blink his eyes and turn his face up, down, left and right, and the applicant himself performs the action. It may be equipped with a function to confirm whether or not it is implemented.

The terminal device 4 transmits the face image acquired from the identification card and the collation face image acquired from the in-camera to the server 2 by the communication unit 45 (step S16). Then, the terminal device 4 receives the collation result information of the face image acquired from the identification card in the server 2 and the collation face image acquired from the in-camera by the communication unit 45 (step S17). The CPU of the terminal device 4 determines whether or not the collation result information indicates a match of the collation results, and if it indicates that they match (YES in step S18), ends the identity verification process and indicates a mismatch. If (NO in step S18), the process returns to step S13 or S14. If it is determined in step S18 that the mismatch occurs a predetermined number of times, the identity verification process may be terminated, and the face image obtained from the identification card may be difficult to recognize due to light reflection. You may go back to the shooting of the surface of the ID card of S05 and try again.

FIG. 16 is a flowchart showing the operation of the server. As shown in FIG. 16, when the server 2 receives the face image of the identification card and the matching face image acquired by the in-camera from the terminal device 4 (step S21), the face matching process of matching the faces by both images is performed. (Step S22). In addition to the face matching process, the server 2 receives from the terminal device 4 an image including the surface and sides of the identification card obtained by photographing the identification card from an angle, so that the identification card is the applicant himself / herself. It may be determined that it is. The server 2 transmits the collation result of the face collation process to the terminal device 4 as the collation result information (step S23), and ends.

(Processing to take an ID card from an angle)
As shown in FIG. 17, first, the terminal device 4 acquires a random value from the server 2 by the communication unit 45 (step S31). The random value can be acquired, for example, by requesting the server 2 from the terminal device 4, and the server 2 generating a random value as a trigger of this request and transmitting the random value to the terminal device 4. Here, as an example, it is assumed that random values are generated for the size, position, and trapezoidal shape of the guide frame F, and each random value is received by the terminal device 4.

Next, the display control unit 434 determines the guide frame F to be displayed on the screen of the display unit 42 (step S32). For example, the display control unit 434 determines the size of the guide frame F from the table stored in the storage unit 44 and the random value with respect to the size of the guide frame F, and the table and the guide stored in the storage unit 44. The position of the guide frame F is determined from the random value with respect to the position of the frame F, and the type of the trapezoidal shape of the guide frame F is determined from the random value with respect to the type of the trapezoidal shape of the table and the guide frame F stored in the storage unit 44. By doing so, one guide frame F is determined.

When the guide frame F is determined, the display control unit 434 causes the display unit 42 to display the determined guide frame F (step S33).

Next, the determination unit 431 determines whether the contour including the thickness of the identification card is along the guide frame F (step S34). When the determination unit 431 determines that the guide frame F is not in line (NO in step S34), the determination result is output to the display control unit 434, and the display control unit 434 outputs the guidance G to the display unit 42. Display (step S35). As a result, the user is guided so that the identification card follows the guide frame F. It should be noted that the determination by the determination unit 431 whether or not it is along the guide frame F may include some errors for the sake of convenience.

On the other hand, when the determination unit 431 determines that the image is along the guide frame F (YES in step S34), the determination result is output to the imaging control unit 432, and the imaging control unit 432 controls the imaging means 41. The ID card is automatically photographed (step S36). After the automatic shooting, the shot image may be displayed on the display unit 42 to prompt the confirmation of the user (applicant).

An image extraction unit 433 generates an extracted image from the image obtained by automatic shooting (step S37). The extracted image is, for example, an image in which the backgrounds above and below the guide frame F of the image obtained by automatic shooting are deleted.

Further, after generating the extracted image, the predetermined area R in the guide frame F is masked (step S38). After that, the masked extracted image is transmitted to the server 2 by the communication unit 45 (step S39), and the process ends. Instead of the transmission in step S39, in step S16, the face image acquired from the identification card and the collation image acquired from the in-camera may be transmitted to the server 2.

[Action / Effect]
(1) The terminal device 4 of the present embodiment controls an image pickup means 41 for capturing an image including an identification card, a display unit 42 for displaying an image in a capture area of the image pickup means 41 on a screen, and image capture. The processing unit 43 includes a processing unit 43, and the display unit 42 displays a trapezoidal guide frame F in a random display mode on the screen, and the processing unit 43 has a contour including the thickness of the identification card F. A determination unit 431 that determines whether or not the shooting condition of being in line with the above is satisfied, and a shooting control unit 432 that causes the image pickup means 41 to shoot an image when the determination unit 431 determines that the shooting condition is satisfied. I tried to have it.

This makes it possible to facilitate identity verification while ensuring live performance. That is, since the guide frame F for accommodating the identification card is randomly displayed, the live performance can be ensured by taking a picture of the identification card so as to fit the guide frame F. On the other hand, since the guide frame F has a trapezoidal shape, the user needs to take a picture of the identification card from an angle, which increases the difficulty of taking a picture. Therefore, the determination unit 431 and the shooting control unit 432 automatically shoot the identification card on condition that the identification card is along the guide frame F so that the thickness can be known. It is possible to improve the convenience of the user in confirming. In addition, since it is determined that the image is along the guide frame F and the identification card is automatically photographed including its thickness, the presence of the obtained image presumes the guarantee of live performance. It will be. Therefore, even if the service provider visually confirms the obtained image, it is only a simple confirmation, and the time and effort of the identity verification process can be reduced.

(2) A communication unit 45 that communicates with the server 2 that is an external device via the network 3 is provided, and the processing unit 43 obtains the thickness of the identification card from the image taken by the image pickup control unit 432 and the image pickup means 41. It further has an image extraction unit 433 that generates an extracted image that extracts the inside of the included contour, and the communication unit 45 transmits the extracted image to the server 2. As a result, the identity verification can be completed online, and the image obtained by extracting the part necessary for the identity verification is transmitted, so that the amount of transmission data can be reduced.

(3) The display unit 42 displays the guide frame F in a random display mode based on the random value transmitted from the server 2. As a result, the guide frame F is displayed on the screen of the display unit 42 based on the random value obtained from the server 2 of the device other than the terminal device 4, so that the reliability of the identity verification can be improved. That is, when the guide frame F is randomly displayed by a random value generated on the terminal device 4 side without being caused by the server 2 which is an external device, the randomness may not be guaranteed due to falsification of the terminal device 4 or the like. Since the guide frame F is displayed on the screen of the terminal device 4 due to the random value obtained from the server 2 of another device, an invalid image of the identification card that matches the guide frame F is prepared in advance. Since it is difficult to do so, the reliability of identity verification can be improved.

(4) In the random display mode, at least one of the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F is randomly determined. As a result, it is possible to prevent the person from passing through the identity verification procedure due to an invalid image of the identification card.

(5) The display unit 42 displays the guidance G that guides the user so that the outline of the identification card can be matched with the guide frame F when the shooting conditions are not satisfied. Specifically, the processing unit 43 guides the user so that the contour of the identification card can be matched with the guide frame F when the determination unit 431 determines that the shooting conditions are not satisfied. A display control unit for displaying G on the display unit 42 is provided. As a result, the convenience of the user can be improved.

(6) The image extraction unit 433 masks the predetermined area R in which the sensitive information of the identification card is described in the extracted image. This makes it possible to reduce the time and effort of masking. That is, the image of the identification card may be stored in an external device such as the server 2. In that case, mask the relevant part before storing it so that unnecessary information on the ID card is not stored. The sensitive information is unnecessary information for identity verification, and since the area in which the information is described is masked in advance, masking on the external device side can be eliminated.

[Other embodiments]
The present invention is not limited to the above embodiment, but also includes other embodiments shown below. The present invention also includes a combination of all or any of the above embodiments and the following other embodiments. Further, various omissions, replacements, and changes can be made to these embodiments without departing from the scope of the invention, and modifications thereof are also included in the present invention.

For example, although the image extraction unit 433 is provided in the processing unit 43, it is not always necessary to provide the image extraction unit 433. That is, the image automatically taken by the image pickup means 41 and the shooting control unit 432 may be transmitted to the server 2 via the communication unit 45. Further, although the predetermined area R is masked by the image extraction unit 433, it is not always necessary to mask the predetermined area R.

The display control unit 434 uses a table stored in the storage unit 44 to determine the guide frame F, but a function that inputs a random value from the server 2 and determines the guide frame F of any one pattern. May be used. Further, the server 2 may transmit the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F as random values to the terminal device 4. In this case, the terminal device 4 specifies the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F from the received random values, and determines the size of the display unit 42 of the terminal device 4. The guide frame F is displayed at the corresponding relative position. As a result, the guide frame F corresponding to the display unit 42 of various sizes can be displayed, and it is possible to easily secure the live performance.

1 System 2 Server 3 Network 4 Terminal device 41 Imaging means 42 Display unit 43 Processing unit 431 Judgment unit 432 Imaging control unit 433 Image extraction unit 434 Display control unit 44 Storage unit 45 Communication unit C Identification card E Identification card contour F Guide frame F1 1st frame F2 2nd frame G Guidance M Margin area S1 Surface of ID card S2 Side of ID card (thickness)

Claims (12)

An imaging means for taking images including identification cards,
A display unit that displays an image of the photographing area of the imaging means on the screen, and a display unit.
A processing unit that controls the shooting of the image, and
Equipped with
The display unit displays a trapezoidal guide frame in the screen in a random display mode, and the random display mode is such that the size of the guide frame and the position of the guide frame are randomly determined. can be,
The processing unit
A determination unit for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control unit that causes the image pickup means to shoot the image when the determination unit determines that the shooting conditions are satisfied.
To have
A terminal device characterized by. Equipped with a communication unit that communicates with external devices via a network
The processing unit
Further having an image extraction unit for generating an extracted image obtained by extracting the inside of the contour including the thickness of the identification card from the image captured by the photographing control unit and the imaging means.
The communication unit
Sending the extracted image to the external device,
The terminal device according to claim 1. The external device is a server and
The display unit displays the guide frame in a random display mode based on a random value transmitted from the server.
2. The terminal device according to claim 2. In the random display mode, the size of the guide frame, the position of the guide frame, and the type of the trapezoidal shape of the guide frame are randomly determined .
The terminal device according to any one of claims 1 to 3. The display unit displays guidance for guiding the user so that the contour of the identification card can be matched with the guide frame when the shooting conditions are not satisfied.
The terminal device according to any one of claims 1 to 4. The processing unit
When the determination unit determines that the shooting conditions are not satisfied, the display unit displays guidance for guiding the user so that the contour of the identification card can be matched with the guide frame. Having a display control unit,
The terminal device according to any one of claims 1 to 5. The image extraction unit masks a predetermined area in the extracted image in which the sensitive information of the identification card is described.
The terminal device according to claim 2 or 3. It is a method of controlling a terminal device provided with an image pickup means for capturing an image including an identification card and a display unit for displaying an image of a photographing area of the image pickup means on a screen.
It is a guide frame display step in which a trapezoidal guide frame is displayed in a random display mode in the screen of the display unit, and in the random display mode, the size of the guide frame and the position of the guide frame are randomly displayed. The guide frame display step, which is defined ,
A determination step for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control step for causing the imaging means to shoot the image when it is determined by the determination step that the shooting conditions are satisfied.
To prepare for
A method of controlling a terminal device characterized by. A program executed on a terminal device including an imaging means for capturing an image including an identification card, a display unit for displaying an image of a photographing area of the imaging means on a screen, and a computer.
To the computer
It is a guide frame display step in which a trapezoidal guide frame is displayed in a random display mode in the screen of the display unit, and in the random display mode, the size of the guide frame and the position of the guide frame are randomly displayed. The guide frame display step, which is defined ,
A determination step for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control step for causing the imaging means to shoot the image when it is determined by the determination step that the shooting conditions are satisfied.
To execute,
A program featuring. An imaging means for taking images including identification cards,
A display unit that displays an image of the photographing area of the imaging means on the screen, and a display unit.
A processing unit that controls the shooting of the image, and
Equipped with
The display unit displays a guide frame having a shape obtained by viewing the identification card from a direction other than the vertical direction with respect to the identification card in the screen in a random display mode , and the random display mode is the above-mentioned. The size of the guide frame and the position of the guide frame are randomly determined.
The processing unit
A determination unit for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control unit that causes the image pickup means to shoot the image when the determination unit determines that the shooting conditions are satisfied.
To have
A terminal device characterized by. It is a method of controlling a terminal device provided with an image pickup means for capturing an image including an identification card and a display unit for displaying an image of a photographing area of the image pickup means on a screen.
It is a guide frame display step in which a guide frame having a shape viewed from a direction other than the vertical direction with respect to the identification card is displayed in a random display mode on the screen of the display unit. The display mode includes a guide frame display step in which the size of the guide frame and the position of the guide frame are randomly determined .
A determination step for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control step for causing the imaging means to shoot the image when it is determined by the determination step that the shooting conditions are satisfied.
To prepare for
A method of controlling a terminal device characterized by. A program executed on a terminal device including an imaging means for capturing an image including an identification card, a display unit for displaying an image of a photographing area of the imaging means on a screen, and a computer.
To the computer
It is a guide frame display step in which a guide frame having a shape viewed from a direction other than the vertical direction with respect to the identification card is displayed in a random display mode on the screen of the display unit. The display mode includes a guide frame display step in which the size of the guide frame and the position of the guide frame are randomly determined .
A determination step for determining whether or not the shooting condition that the contour including the thickness of the identification card is along the guide frame is satisfied.
A shooting control step for causing the imaging means to shoot the image when it is determined by the determination step that the shooting conditions are satisfied.
To execute,
A program featuring.

Images