KR20220026912A - Computing apparatus and method for authentication of pattern code including facial feature information - Google Patents

Abstract

The present invention relates to a computing apparatus for authenticating a pattern code including facial feature information to prevent stealing of pattern codes and a method thereof. According to the present invention, the computing apparatus comprises a communication device, a memory storing or loading a pattern code authentication program including facial feature point information, and a processor executing the pattern code authentication program. The pattern code verification program includes instructions described to use an AI (Artificial Intelligence) face recognition model to generate face vector data representing facial features from an image of an authentication requester's face; extract face vector data from scan data of the pattern code presented by the authentication requester; and when similarity between the generated face vector data and the extracted face vector data is equal to or greater than a preset threshold value, determine that the authentication requester is the same person as a user who registered the face image in the pattern code to processing pattern code authentication as a success.

Description

Computing device and method for performing pattern code authentication including facial feature point information

The present invention relates to a computing device and method for performing pattern code authentication including facial feature point information.

A QR (Quick Response) code is generally generated including information such as text and a URL (Uniform Resource Locator) link.

Although the QR code is used as an identity authentication method, it is difficult to confirm that the QR code user is the person because the QR code itself does not include the user's biometric information. Therefore, you cannot prevent users other than yourself from copying and using the QR code.

An object to be solved is to provide a computing device and method for performing pattern code authentication including facial feature point information for preventing the pattern code from being used by users other than the creator.

The task to be solved is to include face vector data in the pattern code, and compare the face vector data obtained from the face image of the authentication requester during authentication using the pattern code with the face vector data registered in the pattern code to request authentication with the pattern code generator. To provide a computing device and method for performing pattern code authentication including facial feature point information for confirming whether a person is the same person.

According to one aspect of the present invention, a computing device includes a communication device, a memory for storing or loading a pattern code authentication program including facial feature point information, and a processor executing the pattern code authentication program, the pattern code authentication program comprising: , using an AI (Artificial Intelligence) face recognition model to generate face vector data representing facial features from an image taken of the face of the authentication requester, and extracting face vector data from the scan data of the pattern code presented by the authentication requester, , if the degree of similarity between the generated face vector data and the extracted face vector data is greater than or equal to a preset threshold, it is determined that the authentication requester is the same person as the user who registered the face image in the pattern code, and the pattern code authentication is successfully processed. Includes instructions described.

The face vector data includes facial feature vectors, and additional face information including at least one of gender, mask wearing, glasses wearing, race, facial expression, age, attractiveness, and a smile value, and the pattern code authentication program may further include instructions described to determine whether or not the same person is the same based on whether the distance similarity between the facial feature vectors and the face additional information match.

The pattern code authentication program may further include instructions described to vary the threshold value by reflecting a similarity with the generated face vector data determined as the same person.

The memory further stores or loads a pattern code issuing program,

The pattern code issuing program is described to generate face vector data representing facial features from a registrant image photographing a registrant's face by using the AI face recognition model, and to generate a pattern code including the generated face vector data It may contain instructions.

The pattern code issuing program generates a pattern code for payment including the payment method information of the registrant and the face vector data, and the pattern code for payment is, if the authentication succeeds, payment using the payment method information It can be used for payment programs that go through the process.

The pattern code issuance program generates a pattern code for the electronic paperweight including the electronic paperweight data of the registrant and the face vector data, and the pattern code for the electronic paperweight is, if the pattern code authentication is successful, the electronic paperweight It can be used in programs that process data.

The pattern code may be used for an electronic access list service and used for access authentication, or may be printed on a resident registration card and used for identity authentication.

The memory further stores or loads an electronic signature service program, and the electronic signature service program extracts identification information by reading an identification image attached to an electronic document requiring an electronic signature, and extracts identification information from the extracted identification authority server. If valid by inquiring about the validity of the ID information, each face vector data is generated using the AI face recognition model from the face image obtained by photographing the face of the electronic signature requester and the self image registered in the self-certification authority server and, comparing the respective face vector data, if the similarity is greater than or equal to a threshold, generates a pattern code including the face vector data, issues a face electronic signature, and attaches the pattern code and the face electronic signature to the electronic document It includes instructions described to do so, and the electronic document may include an electronic contract.

According to another feature of the present invention, there is provided an operating method of a device operated by at least one processor, the steps of recognizing a pattern code to obtain scan data, capturing a user's face to obtain a face image, the scan data and transmitting a pattern code authentication request including the face image to a server device, receiving a pattern code authentication result from the server device, and if the pattern code authentication result is successful, using service data stored in the pattern code and performing a service procedure, wherein the pattern code authentication is performed between face vector data representing facial features generated from the face image using an artificial intelligence (AI) face recognition model and face vector data stored in the pattern code. It is determined whether the degree of similarity is equal to or greater than a preset threshold, and whether the user is the same as a registrant who has registered a face image in the pattern code.

The pattern code is generated in the server apparatus or the registrant device in response to a request of a registrant device connected to the server apparatus, and the server apparatus receives a face image photographed through the registrant device from the registrant device, A pattern code including face vector data generated using an AI face recognition model and service data provided through the registrant device is generated from a photographed face image and provided to the registrant device, wherein the registrant device includes the photographed face An image may be transmitted to the server device to receive the face vector data from the server device, and a pattern code including the received face vector data and service data input by the registrant may be generated.

The service procedure may include at least one of a payment service requiring identification, an electronic questionnaire service, identification using a resident registration card, and an electronic signature service.

According to the embodiment, the pattern code is generated by including vector data generated by AI face recognition when generating the pattern code, and face authentication is performed once more when the pattern code is authenticated to determine whether the person is the person, thereby preventing theft of the pattern code and preventing the identity of the user. can only be used

1 is a configuration diagram of a pattern code authentication system including facial feature point information according to an embodiment.
2 is a flowchart illustrating a pattern code generation method according to an embodiment.
3 is a flowchart illustrating a QR code generation method according to another embodiment.
4 is a flowchart illustrating a pattern code authentication method including facial feature point information according to an exemplary embodiment.
5 illustrates an electronic access list service flow as an embodiment of applying pattern code authentication including facial feature point information according to an embodiment.
6 illustrates a facial recognition electronic signature service procedure as an embodiment of applying pattern code authentication including facial feature point information according to another embodiment.
7 shows an example of a contract to which the facial recognition electronic signature of FIG. 6 is applied.
8 illustrates a payment service procedure as an embodiment of applying pattern code authentication including facial feature point information according to another embodiment.
9 illustrates a corona questionnaire procedure as an example of applying pattern code authentication including facial feature point information according to another embodiment.
10 is a structural diagram of a computing device according to another embodiment.
11 is a structural diagram of a computing device according to another embodiment.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, terms such as “…unit”, “…group”, “…module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. can

The devices described in the present invention are composed of hardware including at least one processor, a memory device, a communication device, and the like, and a program to be executed in combination with the hardware is stored in a designated place. The hardware has the configuration and capability to implement the method of the present invention. The program includes instructions for implementing the method of operation of the present invention described with reference to the drawings, and is combined with hardware such as a processor and a memory device to execute the present invention.

As used herein, "transmission or provision" may include not only direct transmission or provision, but also transmission or provision indirectly through another device or using a detour path.

In this specification, expressions described in the singular may be construed in the singular or plural unless an explicit expression such as “a” or “single” is used.

In this specification, regardless of the drawings, the same reference numbers refer to the same components, and "and/or" includes each and every combination of one or more of the referenced components.

In this specification, terms including an ordinal number such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In the embodiments described in this specification with reference to the drawings, an arbitrary embodiment may be implemented alone, various embodiments may be combined or divided, and a specific operation may not be performed in each embodiment.

Now, a computing device and method for performing pattern code authentication including facial feature point information according to an embodiment of the present invention will be described with reference to the drawings.

1 is a configuration diagram of a pattern code authentication system including facial feature point information according to an embodiment.

In this case, the configuration of the authentication system only shows a schematic configuration necessary for description according to an embodiment of the present invention, and is not limited thereto.

Referring to FIG. 1 , the pattern code authentication system including facial feature point information includes a server apparatus 100 , a registration device 200 , and an authenticator device 300 , and these components 100 , 200 , 300 are network ( 400) to transmit and receive data.

At this time, although the registration device 200 and the authenticator device 300 are described as separate devices to distinguish the roles of the devices 200 and 300 , the present invention is not necessarily limited thereto, and the registration device 200 is not limited thereto. and the authenticator device 300 may be implemented as one physical device.

The server apparatus 100 issues a pattern code including face vector data, which is feature information of a face image received from the registration device 200 .

The pattern code represents information in a black and white grid pattern, and a typical example is a QR code.

In this case, the pattern code may include an authentication key or service data selectively received from the registration device 200 . The service data is data related to a service that requires user authentication, and may include, for example, response data to a corona questionnaire, payment method information, and the like. The authentication key may be used in addition to the face image during authentication or as a means of replacing the face image.

The pattern code may be generated in the server apparatus 100 and transmitted to the registration device 200 . Alternatively, the server apparatus 100 may transmit only the face vector data to the registration device 200 , and a pattern code using the face vector data may be generated in the registration device 200 .

The pattern code may include a command requesting to take a face image or to input an authentication code when the pattern code is scanned. Also, the pattern code may include face vector data, service data, and a serial number. The service data may include text data, a URL link, and the like.

The server apparatus 100 performs pattern code authentication including facial feature point information according to a request from the authenticator device 300 . Pattern code authentication is a procedure for determining whether a registrant who has registered a face image in a pattern code and a requestor for authentication are the same person. According to this procedure, the server apparatus 100 determines in advance the degree of similarity between the face vector data extracted from the face image received from the authenticator device 300 and the face vector data read from the pattern code received from the authenticator device 300 . If the specified threshold is satisfied, pattern code authentication is treated as success, and if not satisfied, pattern code authentication is treated as failure.

Such pattern code authentication can be applied to corona questionnaires, payment using pattern codes, resident registration cards, and electronic signature services. However, it is not limited to these services, and may be applied to various services to which face recognition or face recognition may be applied.

The server device 100 uses an artificial intelligence (AI) face recognition model to generate face vector data representing facial features from an image of the face of the person requesting authentication. In this case, as the AI face recognition model, a known commercial face recognition engine may be used. The server device 100 may use a deep learning model as an AI face recognition model.

The face vector data generated by the server device 100 may be shown in Table 1 below.

{ "Vector": pwAAAAAAAAMAgAATEcXPVXKh71wl7s9VoYsPdxCBb6DQWG9S5m2vfEGdr3M7xW9Opvxur3QTD2Fu8u9oq8DvpZWLz6zI5W97Tq1O4FINj0NdRA99nJpOrG4hb24PcI9cjXRPaGMdbsHje872zNDvNX8Pr1eDIY9p / enveb727zFVCo6YhDHPHhqFz2gen89ClZovt4hQDzH6rc8NZ1hur69pD1oezS + tW8iPT9pFr2v1q + 8wb / AvE7iXLz0lCY82igzPeJq6j0vhjs8JBgqvp9DAD4dgXU9b / GHPPuZY72k8ko9ZEGyvVLGnT3QnPg9lglUPBHdrD2CgUG9zyHRvFzcO73WT + 87j9zcvV4m6L2rQwq + LZsZPku3173efHy7LnBuvaGBEL2Rau + 6fofvvdYD9L2AT5A9gI19PEIfjz2tUxE98 + KXvT2BND6KdHi + togNvqmxgD3PpSu + FLFCvBbxBr5eCue9WrGfvXyqRT37gy8 + AYImPfLaxb1RsjK + PgxEveLG / r2hfzI8ueFyPTYTajw6rwo8HJmEPCPx1z3WlBS + rpwdvtCkBz2zkca9dgizvEUjzz0j / dC8HRRTvR / Or724yqC9Y45EvVh4Mr4g4WQ8Q4v / vC22oD02v6y9eWWVPbwzUj6qVH27IpRnvXjgOb2W + Zu9KaPFvRnunj2oW8i9z8nyPLpJ + j0 = "," name": "Patricia Halford"}

The server device 100 may include a face image database for storing face data sets. The face image database stores image resolution (high, middle, low), ID, accessories (mask, glasses), lighting location and intensity (1000lux, 400Lux…), facial expression (smile, grimacing,..), pose direction (angle), etc. In consideration of this, it is possible to store learning data of image data taken from various directions. In addition, the training data of the face image includes 128 pieces of index information for the face, for example, the tip of the nose, the center of the right eye, the center of the left eye. Right corner of mouth, left corner of mouth, right ear center, left ear center, etc... It is labeled considering the location of each location based on .

The server device 100 may detect the face region by preferentially detecting the position of the head by driving the face tracking and detection algorithm in the face image (as a single static image, including several objects, that is, things). At this time, after detecting the face region, crop is called on the target image to obtain an image in which the face is automatically aligned to the center.

As such, the acquired face image includes a face quality score, and is scored in consideration of the size of the image, image generation information, image orientation (angle), and the ability to extract facial feature points. can do.

The server device 100 may determine whether the acquired face image is a biometric image by using the face biometric recognition interface, and in this case, the face biometric recognition interface may use a well-known application programming interface (API).

The server device 100 may extract face vector data including a facial feature vector value and face attribute information (eg, gender, age, expression, race, etc.) from the face image.

The server device 100 may extract all face information and the number of faces in the image by performing face tracking from the image. The returned face information includes a face quality score, which may be used as a criterion for performing face attribute and face feature extraction. there is. For example, when the facial quality score is greater than or equal to a threshold (eg, 0.6), it can be used as a facial attribute and facial feature extraction.

A method of extracting facial feature data may have various embodiments.

According to an embodiment, the server device 100 may perform face recognition (or face recognition) by using a feature vector for the entire face region.

The server device 100 may use an image structure, depth information, shape and color, etc. (etc), and a combination thereof, to separate a face region from an image. In general, face regions can be separated based on template matching and region protection division using ellipse information of the face or using horizontal and vertical values.

The PCA (Principle Component Analysis) method is mainly used for feature vectors for the entire face region. According to this method, the entire face region is divided as a whole and expressed as an encoded graph, fixed at a position similar to the face region, and modeled. derive Then, by calculating the positions, lighting, angles, etc. of pixels, the average and variance values of the feature vector values are stored, and the stored average/dispersion values are compared with the average/dispersion values derived from the comparison target image to compare the stored image and the comparison target image. to determine whether there is a similarity between

According to another embodiment, the server device 100 may perform face recognition (or face recognition) by using a feature vector for a partial face region. The feature vector for the face partial region extracts parts with a large amount of information as features, and the parts with a large amount of information are called landmarks. For landmarks used for face recognition, 68 points are used. When a landmark is used, it is effective for recognition because it is geometrically stable even in rotation, movement, scale, lighting, and viewpoint change.

The server device 100 may detect a face region from an image by using the MCT-Adaboost algorithm, and may detect a plurality of predefined facial landmarks from the detected face region. The server device 100 may normalize a plurality of detected facial landmarks and generate face vector data, which is feature information on the plurality of normalized facial landmarks, using a Convolutional Neural Network (CNN). The server device 100 stores landmarks as eigenvalues of face regions, and extracts feature vectors, ie, face vector data, from each landmark.

The server device 100 recognizes that the face vector data are identical if the distance similarity between the face vector data stored in the pattern code and the face vector data generated from the photographed image (face recognition target image) does not exceed a threshold. And if the number of face vector data recognized as identical as above meets the threshold number, it is determined that the two images (face vector data registered in the pattern code and face vector data extracted from the photographed image) are identical, and pattern code authentication is successful. handle

In this case, since various known algorithms may be used, the distance similarity is not limited to a specific algorithm.

According to another embodiment, the server device 100 may perform face recognition (or face recognition) using infrared technology. In this case, the server device 100 projects more than tens of thousands of dots on the face with a dot projector using infrared technology, extracts face vector data from the projected dot image pattern, and compares it with the face vector data registered in the pattern code. Thus, it can be determined whether or not they match.

In addition, the server device 100 may use an AI face recognition model generated by finding a point in the face image where the contrast varies in units of pixels and learning it. In this case, the face vector data generated through learning by finding a point where the contrast varies in units of pixels may be included in the pattern code.

The face vector data representing the facial features extracted by the server device 100 includes feature values for the entire face or feature values for a plurality of face landmarks, and gender, whether to wear a mask, whether to wear glasses, race, facial expression, age, and attractiveness. It may include additional face information including at least one of a figure and a smile figure.

In this case, in the case of face additional information, it can be defined as shown in Table 2 below.

item value gender 0: female, 1: male wearing a mask 0: not worn, 1: worn Wearing glasses (glass) 0: not worn, 1: regular glasses, 2: sunglasses race 0: yellow, 1: black, 2: white facial expression 0: angry, 1: calm, 2: depressed, 3: happy, 4: sad, 5: fear, 6: surprise age predicted age attractive Prediction attractiveness (0-100) smile Prediction smile information (0-100)

The server device 100 performs face recognition by comparing the similarity of the feature values of the entire face or the face landmarks, and may additionally compare items of additional face information. In the case of comparison between items of additional face information, if the value is clear as a natural number such as 0, 1, 2, it compares whether or not they match. It can be compared whether the value difference is less than or equal to a threshold value. And when both conditions of feature value similarity and additional information match/similarity are satisfied, pattern code authentication can be processed as a success.

In this case, the server apparatus 100 may output some information on the screen of the authenticator device 300 according to an instruction registered in the pattern code in advance only for face additional information among the face vector data registered in the pattern code. For example, gender, race, age, etc. may be output to the screen.

Also, the server device 100 may vary a threshold used when evaluating the similarity of the face vector data. The threshold set in advance through AI learning may be updated while repeating pattern code authentication including facial feature point information. When the face vector data extracted from the photographed image satisfies a similarity condition with the face vector data registered in the pattern code, the server device 100 may change the threshold value by reflecting the similarity values.

According to an embodiment, the threshold value is set by extracting face vector data from images obtained by photographing the same person's face image under various conditions, and from images obtained by photographing another person's face image, and using a similarity distribution between them. can be In this case, the threshold value may be varied by reflecting the similarity value when the pattern code authentication is successful in the similarity distribution.

The registration device 200 and the authenticator device 300 may execute a client program corresponding to the server apparatus 100 to perform a pattern code authentication operation including facial feature point information.

In this case, the client program may be a smartphone application or an application built into the device itself. The client program operates in conjunction with the server device 100 .

In addition, in a state connected to the server device 100 without a separate client program, the web browser or mobile browser driven in the registration device 200 and the authenticator device 300 performs a facial feature point according to a control command of the server device 100 . A pattern code authentication operation including information can be performed.

The registration device 200 is a device for registering a face image to be used for authentication, and is a terminal provided by a registration requester. The registration device 200 may transmit a face image while being connected to the server device 100 to receive a pattern code including face vector data extracted from the face image, or may generate a pattern code using the face vector data. . The registration device 200 is a tablet PC (Tablet PC), a laptop (Laptop), a personal computer (PC, Personal Computer), a smart phone (Smart Phone), a personal information terminal (PDA, Personal Digital Assistant), a mobile communication terminal (Mobile) Communication Terminal) and may include various terminals such as a notebook (NoteBook).

The authenticator device 300 is a device that requests authentication of a pattern code including facial feature point information. The authenticator device 300 transmits the scan data of the pattern code and the face image of the authentication requester while connected to the server device 100 so that the face image registered in the pattern code and the transmitted face image from the server device 100 are Receives an authentication result that is determined to match.

The authenticator device 300 may be a user terminal provided by the authentication requester. The authenticator device 300 may be a terminal of a provider (hereinafter, collectively referred to as a service provider) that provides a service or task requiring identity authentication. The authenticator device 300 may include a tablet PC, a laptop, a personal computer, a smartphone, a personal information terminal, a mobile communication terminal, a notebook computer, a kiosk, a POS (Point of Sale) terminal, a terminal dedicated to electronic access list management, etc. there is.

The network 400 may include a wired network, a mobile communication network such as a long term evolution (LTE) network/5G network, various types of wireless networks such as a Wi-Fi network, and combinations thereof.

2 is a flowchart illustrating a method for generating a pattern code according to an exemplary embodiment, in which the pattern code is generated by the server device 100 .

Referring to FIG. 2 , when a user input signal requesting generation of a pattern code is generated in the registrant device 200 ( S101 ), a face image (image including the registrant’s face) captured by the camera is generated ( S103 ) .

The registrant device 200 determines whether or not the authentication key is registered (S105), and when it is determined that the authentication key is registered, generates an authentication key according to the user's input or the user's request (S107).

The registrant device 200 determines whether additional data is registered (S109), and when it is determined that additional data is registered, additional data is generated according to the user's input or the user's request (S111).

The registrant device 200 transmits a pattern code generation request including the face image generated in step S103, the authentication key generated in step S107 and/or additional data generated in step S111 to the server device 100 (S113). ).

The server device 100 generates face vector data representing facial features or facial attributes from the face image received in step S113 by using the AI face recognition model (S115).

The server device 100 generates a pattern code including the face vector data generated in step S115 and the authentication key and/or additional data received in step S113 (S117). Then, the generated pattern code is transmitted to the registrant device 200 (S119).

3 is a flowchart illustrating a pattern code generation method according to another exemplary embodiment, in which the pattern code is generated by the registrant device 200 .

Referring to FIG. 3 , when a user input signal requesting generation of a pattern code is generated in the registrant device 200 (S201), a face image (image including the registrant's face) captured through the camera is generated (S203) .

The registrant device 200 transmits a face vector data generation request including the generated face image (S203) to the server apparatus 100 (S205).

The server device 100 generates face vector data from the face image received in step S205 by using the AI face recognition model (S207). Then, the generated (S207) face vector data is transmitted to the registrant device 200 (S209).

The registrant device 200 determines whether or not the authentication key is registered (S211), and when it is determined that the authentication key is registered, generates an authentication key according to the user's input or the user's request (S213).

The registrant device 200 determines whether additional data is registered (S215), and when it is determined that additional data is registered, additional data is generated according to the user's input or the user's request (S217).

The registrant device 200 generates a pattern code including the face vector data received in step S209, the authentication key received in step S213, and/or additional data generated in step S217 (S219).

2 and 3 may be launched in various services providing pattern codes. For example, it can be applied to various services such as pattern code generation for KakaoTalk payment, electronic access list service, electronic signature service, and resident registration card. It may be provided through a site/mobile site.

According to an embodiment, after the client program executed in the registrant device 200 is connected to the server apparatus 100 , it may be performed. In this case, whether or not to register the authentication key or not to register additional data depends on the settings of the client program. For example, when the client program is a payment program, it may request to register an authentication key when generating payment method information as a pattern code. In this case, the additional data becomes payment method information.

According to another embodiment, the pattern code generation may be performed after the registrant device 200 accesses a service web site or a mobile site. In this case, whether or not to register the authentication key or not to register additional data depends on the settings of the website or mobile site. For example, if you click the pattern code issuance item on the portal site, you can request to register an authentication key as well.

In addition, in the embodiments of FIGS. 2 and 3 , authentication key registration and additional data registration may be selectively performed according to a user's selection.

4 is a flowchart illustrating a pattern code authentication method including facial feature point information according to an exemplary embodiment.

Referring to FIG. 4 , the authenticator device 300 may be a terminal of a service provider requiring face authentication or a terminal of a service user requiring face authentication. For example, the authenticator device 300 may be a payment terminal of a store, an identity authentication terminal, an electronic access list service terminal, and the like.

The authenticator device 300 scans a pattern code presented by a service user or an authentication requester requiring authentication (S301). The authenticator device 300 creates a face image by photographing the face of the authentication requester through camera photography (S303).

The authenticator device 300 transmits the pattern code authentication request including the scan data of the pattern code and the face image to the server apparatus 100 (S305).

The server device 100 extracts face vector data from the pattern code scan data (S307).

The server device 100 generates face vector data from the face image received in step S305 by using the AI face recognition model (S309).

The server device 100 calculates a similarity by comparing the face vector data extracted in step S307 with the face vector data generated in step S309 ( S311 ).

The server device 100 determines whether the similarity is greater than the threshold value (313), and if so, determines that the registrant who has registered the face in the pattern code and the authentication requester are the same person, and processes the pattern code authentication as a success (S315). Then, the authentication success response is transmitted to the authenticator device 300 (S317).

If the degree of similarity is not greater than the threshold value in step S313, the server device 100 determines that the registrant who has registered a face in the pattern code and the authentication requester are not the same person, and processes it as authentication failure (S319). And transmits an authentication failure response to the authenticator device 300 (S321).

At this time, if the pattern code received in step S305 includes the authentication key and the pattern code includes a command instructing to determine whether the authentication key matches or not, after step S315, the authenticator device 300 to input the authentication key If the authentication key requested and received from the authenticator device 300 matches the authentication key included in the pattern code, it may be treated as authentication failure. Of course, the authentication key may be input in the authenticator device 300 and transmitted together in step S305.

Alternatively, when the authenticator device 300 does not have a pattern code scanner, the authenticator device 300 and the server apparatus 100 may perform authentication using an authentication key.

The pattern code authentication including the facial feature point information described above can be applied to various services, which will be described as follows.

5 is a flowchart illustrating an electronic access list service according to an embodiment of applying pattern code authentication including facial feature point information according to an embodiment.

Referring to FIG. 5 , a pattern code issuer provides a pattern code generation function to an app or an Internet site. A user (facility user) who uses a facility requiring an electronic access list uses the registration device 200 to access an app or internet site provided by a pattern code issuance provider and provides his or her face image to issue a pattern code (S1) receive When entering the facility, the facility user presents the pattern code to the facility manager and takes a picture of the face (S2). The facility manager scans the pattern code of the facility user through the manager app of the authentication device 300 and provides a facial image obtained by photographing the facility user to the pattern code authentication service provider to perform authentication (S3). At this time, the authentication operator compares the degree of similarity between the face vector data in the pattern code and the face vector data extracted from the photographed face image to authenticate (S3). If authentication is successful, the facility manager creates a visit record and facility information and provides it as a social security information source (S4). The social security information source stores the visit record and facility information provided by the facility manager, and provides it (S5) if requested by the Korea Centers for Disease Control and Prevention. The Korea Centers for Disease Control and Prevention identifies visitors based on the pattern code issuance information provided by the pattern code issuing company (S6) and the visit record/facility information provided by the social security information source, conducts an epidemiological investigation, and discards the information after a certain period of time. .

In this case, the facility user may use the registration device 200 and the authentication device 300 described with reference to FIGS. 1 to 4 . The facility manager may use the authentication device 300 described with reference to FIGS. 1 to 4 . If the facility manager does not have a pattern code scanner, the facility user's authentication device 300 may be used to check the pattern code authentication result through the screen of the authentication device 300 .

The pattern code issuing operator and the pattern code authentication operator may be the same, and may operate the server device 100 .

6 is an embodiment of applying pattern code authentication including facial feature point information according to another embodiment, illustrating a facial recognition electronic signature service procedure, and FIG. 7 shows an example of a contract to which the facial recognition electronic signature of FIG. 6 is applied.

Referring to FIG. 6 , an operation when the registrant device 200 and the authentication device 300 are implemented as one user device is shown, and such a user device may interwork with the server apparatus 100 . In this case, the server device 100 may be a server that generates an electronic contract according to the facial recognition electronic signature service. In this case, the flowchart of FIG. 6 will be described as an operation implemented in the user device while being connected to the server apparatus 100 .

The user device generates an electronic document type contract according to the electronic document format provided by the server device 100 and based on data input by the user (S401).

The user device reads the ID image attached to the contract with OCR (optical character recognition) technology and extracts ID information (eg, name, resident number, issue address, etc.) (S403).

The user device checks the authenticity of the ID information by requesting verification of whether the ID information is valid from the identity authentication authority (eg, the Road Traffic Authority server, the government 24 server, the National Police Agency server, the electronic list server, etc.) (S405).

As a result of checking the authenticity in step S405, the user device determines whether the ID information is valid (S407), and if it is determined that it is valid, drives the camera to photograph the face (S409).

A face image taken from a user device and a face image registered with a self-certification authority are provided, face vector data is extracted from each of them, and facial recognition is driven to compare the degree of similarity between the extracted face vector data (S411).

The user device receives the face recognition result from the server apparatus 100, and determines whether the face recognition result is successful (S413). If it is determined that the face recognition is successful, the server apparatus 100 or the user device generates a pattern code including the face vector data extracted from the photographed face image and identification information in step S411 (S415).

The server device 100 issues a face electronic signature ID according to the face recognition result (S417). The user device may attach the pattern code generated in step S415 and the electronic signature ID issued in step S417 to the electronic contract or print it on the offline contract document (S419).

Referring to FIG. 7 , a pattern code may be printed or attached to the upper part of the labor contract, and a face signature indicating a facial recognition authentication result may be printed or attached to the lower part of the labor contract. At this time, the face signature can be created as "2020.07.30. 12:30:39 000 signed with Face ID: aDWf45xGegor004ExTeact".

8 illustrates a payment service procedure as an embodiment of applying pattern code authentication including facial feature point information according to another embodiment.

Referring to FIG. 8 , an embodiment in which the authentication device 300 is implemented as a payment kiosk is shown. The payment requester scans the pattern code in the payment kiosk 300 and photographs a face with a camera. The payment kiosk 300 transmits the scan data of the pattern code and the face image captured by the camera to the server device ( 100 in FIGS. 1 to 4 ) to request authentication, and receives the authentication result. If the authentication result is successful, the payment kiosk 300 proceeds with the payment process using the payment method scanned from the pattern code.

The payment software provided to the store, including the payment kiosk 300 , may output the face vector data transmitted from the server device 100 to the screen, and in this case, additional information such as age and gender may be output to the screen. However, if personal privacy is taken into consideration, the data that can be viewed in the store is limited, and only the male and age information are shown roughly, so it can be briefly checked first.

9 illustrates a corona questionnaire procedure as an example of applying pattern code authentication including facial feature point information according to another embodiment.

Referring to FIG. 9 , a visitor to a facility requiring a corona examination, for example, a hospital, takes a face image through the registration device 200 and creates an electronic questionnaire. Then, the pattern code is issued by accessing the server device 100 of the pattern code authentication service provider and transmitting the photographed face image and the electronic questionnaire (S11). Of course, the pattern code may be generated through an app of the registration device 200 .

Visitors present the pattern code to security personnel and have their faces photographed. That is, the security officer recognizes the pattern code through the manager app of the authentication device 300 and obtains a face photographed image (S12). The security officer performs pattern code face authentication between the authentication device 300 and the server device 100 of the pattern code authentication service provider (S13). If authentication is successful, the security guard can scan the pattern code to check the electronic questionnaire data obtained and then permit access. If the security officer does not have pattern code scanners, after performing the pattern code face authentication (S13) through the visitor's authentication device 300, the pattern code authentication result is output on the screen of the authentication device 300 Afterwards, you can present it to security personnel. Then, the security personnel may check the pattern face authentication result and the electronic questionnaire data output on the screen of the authentication device 300 , and then allow the visitor to enter.

In addition to the above embodiments, ID theft can be prevented by printing a pattern code according to an embodiment of the present invention on a resident registration card when generating a resident registration, and performing pattern code authentication including printed facial feature point information during identification using the resident registration card. . For example, when a criminal group forges a picture of oneself in the resident registration of another person, the police officer scans the pattern code of the resident registration card, takes a picture of the presenter of the resident registration card at the scene, and sends it to the server device 100 to check whether the identity is the same. In addition, the same can be applied to cases where a minor steals another person's resident registration card.

In addition, pattern code authentication including facial feature point information can be applied to a distance education system or to a self-authentication process in a test center.

Meanwhile, FIG. 10 is a structural diagram of a computing device according to another embodiment. The server device 100 described in FIGS. 1 to 9 may be implemented as a computing device, for example, a computer-readable medium.

Referring to FIG. 10 , the hardware of the computing device 500 may include at least one processor 501 , a memory 503 , a storage 505 , and a communication interface 507 , to be connected through a bus. can In addition, hardware such as an input device and an output device may be included. The computing device 500 may be loaded with various software including an operating system capable of driving a program. The computing device 500 operated by at least one processor executes a program including instructions described to execute the operations of the present invention.

The processor 501 is a device for controlling the operation of the computing device 500 and may be various types of processors 501 that process instructions included in a program, for example, a central processing unit (CPU), an MPU ( It may be a micro processor unit), a micro controller unit (MCU), a graphic processing unit (GPU), or the like. The memory 503 loads the corresponding program so that the instructions described to execute the operation of the present invention are processed by the processor 501 . The memory 503 may be, for example, read only memory (ROM), random access memory (RAM), or the like. The storage 505 stores various data, programs, etc. required for executing the operation of the present invention. The communication interface 507 may be a wired/wireless communication module.

11 is a structural diagram of a computing device according to another embodiment. The devices 200 and 300 described in FIGS. 1 to 9 may be implemented as a computing device, for example, a computer-readable medium.

11 , the computing device 600 includes a communication device 601 , a display device 603 , an input device 605 , a camera module 607 , a pattern code scanner 609 , a memory 611 , and at least one and a processor 613 of

The communication device 601 allows the memory 611 and/or the processor 613 to access the network 400 . The display device 603 may use a liquid crystal display (LCD) technology or a light emitting polymer display (LPD) technology, and the display device 603 may be a capacitive, resistive, or infrared touch display. The display device 603 outputs an operation result of the program executed by the processor 613 on the screen. The input device 605 is a means for receiving a user command, and may include various input devices such as a touch sensor, a keypad, a button, a mouse, a keyboard, and a pointing device. At least one camera module 607 may be automatically driven by the processor 613 or driven by the input device 605 .

The pattern code scanner 609 is a scanner that recognizes a QR code or the like. The memory 611 may include high-speed random access memory, and may also include one or more magnetic disk storage devices, nonvolatile memory such as flash memory devices, or other nonvolatile semiconductor memory devices. The memory 611 is accessed by the processor 613 , and reading/writing/modification/deletion/update of data by the processor 613 may be performed. The memory 611 may include a ROM (not shown), a RAM (not shown), and a memory card (not shown) (eg, micro SD card, memory stick) in the processor 613 . can The memory 611 stores various programs and data necessary for the operation of the computing device 600 .

The at least one processor 613 may be implemented as a central processing unit (CPU) or other chipsets, microprocessors, or the like. The processor 613 is a processor configured to perform an operation of the computing device 600 and perform instructions, and executes a program stored in the memory 611 .

The embodiment of the present invention described above is not implemented only through the apparatus and method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also provided. is within the scope of the right.

Claims (11)

communication device,
A memory for storing or loading a pattern code authentication program including facial feature point information, and
A processor executing the pattern code authentication program,
The pattern code authentication program,
Using an AI (Artificial Intelligence) face recognition model to generate face vector data representing facial features from an image taken of the face of the authentication requester, extracting face vector data from the scan data of the pattern code presented by the authentication requester,
If the degree of similarity between the generated face vector data and the extracted face vector data is greater than or equal to a preset threshold, it is determined that the authentication requester is the same person as the user who registered the face image in the pattern code, and the pattern code authentication is successfully processed. Computing device comprising the instructions (Instructions). In claim 1,
The face vector data is
a facial feature vector, and additional face information including at least one of gender, whether or not to wear a mask, whether or not to wear glasses, race, facial expression, age, attractiveness, and a smile value;
The pattern code authentication program,
The computing device further comprising instructions to determine whether the same person is the same based on whether the distance similarity between the facial feature vectors and the face additional information match. In claim 1,
The pattern code authentication program,
The computing device further comprising instructions to vary the threshold value by reflecting the similarity with the generated face vector data determined to be the same person. In claim 1,
The memory further stores or loads the pattern code issuing program,
The pattern code issuing program,
Computing device comprising instructions described to generate face vector data representing facial features from a registrant image photographing the face of a registrant by using the AI face recognition model, and to generate a pattern code including the generated face vector data . In claim 4,
The pattern code issuing program,
generating a pattern code for payment including the registrant's payment method information and the face vector data;
The pattern code for the payment is,
When the authentication is successful, the computing device is used in a payment program that proceeds a payment process using the payment method information. In claim 4,
The pattern code issuing program,
Generating a pattern code for the electronic medical examination including the registrant's electronic questionnaire data and the face vector data,
The pattern code for the electronic paperweight is,
If the pattern code authentication is successful, the computing device is used in a program processing the electronic paperweight data. In claim 4,
The pattern code is
A computing device that is used in the electronic access directory service to authenticate access, or is printed on a resident registration card and used for identity authentication. In claim 4,
The memory further stores or loads the electronic signature service program,
The electronic signature service program,
If it is determined that it is valid by reading the ID image attached to the electronic document requiring an electronic signature, extracting ID information, and inquiring about the validity of the extracted ID information with the self-certification authority server,
Each face vector data is generated by using the AI face recognition model from the face image obtained by photographing the face of the electronic signature requester and the self image registered in the self-certification authority server,
Comparing each of the face vector data and generating a pattern code including the face vector data if the similarity is greater than or equal to a threshold, issuing a face electronic signature,
instructions described to attach the pattern code and the facial electronic signature to the electronic document;
The electronic document includes an electronic contract. A method of operating a device operated by at least one processor, comprising:
Recognizing the pattern code to obtain scan data;
acquiring a face image by photographing the user's face;
transmitting a pattern code authentication request including the scan data and the face image to a server device, and receiving a pattern code authentication result from the server device; and
If the pattern code authentication result is successful, comprising the step of performing a service procedure using the service data stored in the pattern code,
The pattern code authentication is
Using an AI (Artificial Intelligence) face recognition model, it is determined whether the similarity between the face vector data representing the facial features generated from the face image and the face vector data stored in the pattern code is equal to or greater than a preset threshold value, and the user selects the pattern code Determining whether the registrant is the same as the registrant who has registered the face image, the operation method. In claim 9,
The pattern code is
generated in the server apparatus or the registrant device by a request of the registrant device connected to the server apparatus,
The server device,
A face image photographed through the registrant device is received from the registrant device, and a pattern code including face vector data generated using an AI face recognition model from the photographed face image and service data provided through the registrant device is generated. to provide it to the registrant device,
The registrant device,
An operation method of transmitting the photographed face image to the server device, receiving the face vector data from the server device, and generating a pattern code including the received face vector data and service data input by a registrant. In claim 9,
The service procedure is
An operating method comprising at least one of a payment service requiring identity verification, an electronic questionnaire service, an identity verification using a resident registration card, and an electronic signature service.

Images