WO2023128345A1

WO2023128345A1 - Personal identification method and system using homomorphically encrypted image

Info

Publication number: WO2023128345A1
Application number: PCT/KR2022/019485
Authority: WO
Inventors: 안용대; 박준홍
Original assignee: 주식회사 디사일로
Priority date: 2021-12-30
Filing date: 2022-12-02
Publication date: 2023-07-06
Also published as: KR102404763B1

Abstract

The present invention relates to a personal identification method using a homomorphically encrypted image, the method comprising the steps of: obtaining first image data of a user; homomorphically encrypting the first image data; transmitting the homomorphically encrypted first image data to an image calculation server; receiving a homomorphically encrypted identification result calculated on the basis of the homomorphically encrypted first image data and pre-stored second image data of another user from the image calculation server; and decrypting the homomorphically encrypted identification result.

Description

Personal identification method and system using homomorphic encrypted image

The present invention relates to a method and system for identifying individuals using homomorphic encrypted images.

Three types of personal identification methods are typically used to prove the user's identity in using computers, laptops, KIOSK terminals, access control facilities, bank terminals (ATM), websites, and Internet banking.

First, a user inputs a password using alphabets and numbers or a personal identification number consisting only of numbers for authentication, thereby identifying whether or not the user is a correct user. The second is a method of recognizing the user's unique biometric information such as the user's fingerprint, iris, face, and voice to identify whether the user is a valid user. Third, a user carries an additional device only for authentication, such as an OTP (One-Time Password) generation device of Internet banking and an employee ID card, and uses the device when authentication is requested to identify the correct user.

The first method is the most used among them, but because users designate different passwords for each system and it is difficult to remember them, many people use short and common passwords for convenience, which is weak in terms of security.

In addition, the third method has a disadvantage in that the user must always carry the device for authentication, and the process of reissuing the device is cumbersome if the device is lost while in possession.

The background description of the invention has been prepared to facilitate understanding of the present invention. It should not be construed as an admission that matters described in the background art of the invention exist as prior art.

Accordingly, the second identification method using the user's unique biometric information, which has no risk of loss and does not change, is a safe method. Significant time and cost for building data is consumed, and there is a problem in that a user's image (unique bio information) is exposed.

Accordingly, a new method for quickly and safely identifying a user using a user's image in a public space is required.

As a result, the inventors of the present invention have attempted to develop a method and a system for safely identifying a user without fear of exposure of the user's image using only a device capable of acquiring the user's image.

In particular, the inventors of the present invention configured a method so that biometric information unique to the user is not exposed by homomorphically encrypting image data acquired from the user and then obtaining an operation result obtained by homomorphically encrypting the user identification result.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a personal identification method using a homomorphic encrypted image according to an embodiment of the present invention is provided. The method includes obtaining first image data of a user, homomorphically encrypting the first image data, transmitting the homomorphically encrypted first image data to an image processing server, and performing the homomorphic encryption from the image processing server. and receiving a homomorphically encrypted identification result calculated on the basis of the first image data and pre-stored second image data of another user, and decrypting the homomorphically encrypted identification result.

According to a feature of the present invention, the transmitting of the homomorphically encrypted first image data includes transmitting a parameter for a homomorphic encryption operation used to homomorphically encrypt the first image data to the image operation server. can include more.

According to another feature of the present invention, the homomorphically encrypted identification result may be an identification result calculated based on the homomorphically encrypted first image data and the homomorphically encrypted second image data based on the parameters.

According to another feature of the present invention, the second image data is image data of a plurality of different users pre-stored in the image calculation server, and the decoding may include other image data matching the user among the plurality of other users. The method may further include obtaining an identification result for the user.

According to another feature of the present invention, the first image data and the second image data may include at least one of an image captured through an identifier device, a plurality of feature coordinate values extracted from the image, and RGB values for each pixel of the image. may contain one.

According to another feature of the present invention, the homomorphic encryption step uses any one of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. It may be a step of performing homomorphic encryption.

In order to solve the above problems, a personal identification method using a homomorphic encrypted image according to another embodiment of the present invention is provided. The method may include receiving an operation request including homomorphically encrypted first image data of a user from an identifier device, obtaining previously stored second image data of another user according to the operation request, the homomorphically encrypted first image data. Calculating a homomorphically encrypted identification result based on one image data and the second image data, and sending the homomorphically encrypted identification result to the identifier device.

According to a feature of the present invention, the receiving of the operation request further comprises receiving parameters for a homomorphic encryption operation, used to homomorphically encrypt the first image data, from the identifier device, wherein the obtaining The performing may further include performing homomorphic encryption on the second image data based on the parameter.

According to another feature of the present invention, the calculating of the homomorphically encrypted identification result may include determining a first location corresponding to the homomorphically encrypted first image data and a second location corresponding to the second image data. and calculating a distance value between the first location and the second location corresponding to the identification result.

According to another feature of the present invention, the calculating of the homomorphically encrypted identification result may include the second image data of the plurality of other users and the homomorphically encrypted first image data according to the type of the received operation request. It may be a step of calculating an encrypted identification result based on.

Other embodiment specifics are included in the detailed description and drawings.

According to the present invention, a user can be identified without sharing user-specific bio information (face image) to an external server for user identification in a public space. In particular, the present invention can identify a user or determine whether a user is the same as another user.

In addition, in the present invention, the user's face image can be safely protected by calculating the user's image data in a state of homomorphic encryption and decrypting and confirming only the calculation result in the device that acquired the user's image.

In addition, the present invention does not need to possess a separate device or memorize a unique identification number for user identification and user authentication, so user convenience can be improved.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present invention.

1 is a schematic diagram of a personal identification system according to an embodiment of the present invention.

2 is a block diagram showing the configuration of an identifier device according to an embodiment of the present invention.

3 is a flowchart of a personal identification method of an identifier device according to an embodiment of the present invention.

4 and 5 are schematic diagrams for explaining a personal identification interface screen output to an identifier device according to an embodiment of the present invention.

6 is a block diagram showing the configuration of an image calculation server that performs a homomorphic encryption calculation according to an embodiment of the present invention.

7 is a flowchart of a personal identification method of an image calculation server according to an embodiment of the present invention.

8 and 9 are schematic flowcharts of a data identification method according to an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like elements.

In this document, expressions such as "has," "may have," "includes," or "may include" indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document may modify various elements, regardless of order and/or importance, and refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, without departing from the scope of rights described in this document, a first element may be named a second element, and similarly, the second element may also be renamed to the first element.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component). On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, third components) do not exist between the other components.

As used in this document, the expression "configured to" means "suitable for," "having the capacity to," depending on the circumstances. ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware. Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (e.g., embedded processor) to perform those operations, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe a specific embodiment, and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted as having the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, an ideal or excessively formal meaning. not be interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude the embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as those skilled in the art can fully understand, various interlocking and driving operations are possible, and each embodiment can be implemented independently of each other. It may be possible to implement together in an association relationship.

For clarity of interpretation of this specification, terms used in this specification will be defined below.

Referring to FIG. 1 , a personal identification system 1000 may include an identifier device 100 displaying a user's personal identification result and an image calculation server 200 calculating a user's personal identification result.

The personal identification system 1000 may be a system capable of identifying a user using a user's image. In the present invention, identifying a user can be understood as recognizing what kind of user a user is or determining whether a user is the same as another user by comparing unique bio information (image of a user) between two users. .

In the present invention, other users as comparison targets for user identification may be users of a group to which the user belongs. For example, users registered in a DB server (not shown) (or image calculation server 200) of a company, school, or region to which the user belongs or users registered in a conference in which the user participates are other users to be compared. can be The identifier device 100 may pre-designate a DB server to be used to use the personal identification service in order to increase accuracy of personal identification results and improve identification speed.

In the personal identification system 1000, the identifier device 100 and the image calculation server 200 can send and receive all data in an encrypted state, and the identifier device 100 and the image calculation server 200 in an encrypted state Data can be encrypted through a homomorphic encryption technique to enable data operation.

That is, data exchanged between the identifier device 100 and the image processing server 200 in the personal identification system 1000 is homomorphic encrypted data, not the original data, and the original data can be stored in each device.

The identifier device 100 and the image calculation server 200 may homomorphically encrypt data through a web page or application/program capable of processing homomorphically encrypted data, and may perform calculations between homomorphically encrypted data. The identifier device 100 and the image calculation server 200 may perform operations between homomorphic ciphertext or between homomorphic ciphertext and plaintext, and may homomorphically encrypt image data using various homomorphic encryption algorithms. For example, the identifier device 100 and the image calculation server 200 use any one encryption method of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. Homomorphic encryption can be used to image data.

The identifier device 100 is a device capable of acquiring a user's image and outputting an image identification result, and may be implemented as a PC, tablet PC, smart phone, wearable device, or the like. Here, the user's unique bio information refers to images of various regions of the user's body that can be captured, and may include images of a face, vein, iris, fingerprint, and the like.

In the present invention, for convenience of description, a method of identifying a user using a face image of the user will be described.

The identifier device 100 may transmit parameters for homomorphic encryption operation to the image operation server 200 so as to obtain an identification result for a corresponding user based on the homomorphically encrypted first image data (image of the user). Specifically, the parameters may be the polynomial degree of a function used for homomorphic encryption operation, scale bits and coefficients specified for homomorphic encryption operation, and attribute information (resolution, size) of an image. can

The identifier device 100 may receive an identification result subjected to a homomorphic encryption operation using a parameter, and may obtain an identification result for a user by decrypting it. For example, when the identifier device 100 is installed in a certain space, the identifier device 100 may take a user's image, encrypt it, and transmit the same to the image calculation server 200, and the image calculation server The isomorphically encrypted operation result received from step 200 is decrypted, and whether or not the corresponding user is registered as an accessible user, that is, whether or not the corresponding user is allowed to enter can be output to each user according to the user identification result.

In various embodiments, the identifier device 100 may directly perform the calculation without receiving the isomorphically encrypted calculation result from the image calculation server 200 . In this case, the identifier device 100 may receive second image data of another user that is homomorphically encrypted from the image calculation server 200, perform an operation between the two homomorphically encrypted data, and operate between the homomorphically encrypted data. The method will be described later.

In various embodiments, the identifier device 100 may homomorphically encrypt the directly captured image itself, but may extract feature points (feature data) from the image and homomorphically encrypt them according to a manager's setting. In this case, the identifier device 100 may transmit metadata about the feature data to the image calculation server 200 as the first image data that is homomorphically encrypted.

The image calculation server 200 is a server capable of performing calculations between homomorphically encrypted data using pre-stored image data according to the calculation request of the identifier device 100, and includes PCs, tablet PCs, smart phones, general-purpose computers, It can be implemented with a laptop and a cloud server.

The image calculation server 200 may store a plurality of second image data (images of other users), perform a homomorphic encryption operation with one user's image data, or perform a plurality of user image data according to the type of operation request. It is possible to perform a plurality of homomorphic encryption operations using

The image calculation server 200 may calculate a homomorphic encrypted calculation result, and decryption of the calculation result may be performed by the identifier device 100 . That is, since the image calculation server 200 receives the homomorphically encrypted first image data from the identifier device 100, performs calculation, and transmits the result without decryption, the image calculation server 200 It is not possible to confirm an identification result of whether the user matches user A previously stored or whether the user is one of a plurality of users.

In various embodiments, the image calculation server 200 performs an operation based on the homomorphically encrypted first image data and the plaintext second image data, or uses an encryption key in which parameters received from the identifier device 100 are reflected. Thus, an operation may be performed based on the homomorphically encrypted second image data.

In various embodiments, the image calculation server 200 may provide a web page or application for isomorphic data encryption and identification result decryption to the identifier device 100 .

In various embodiments, the identifier device 100 and the image calculation server 200 may pre-process image data stored in their respective devices to reduce the burden of homomorphic encryption calculation before performing homomorphic encryption. For example, the identifier device 100 and the image calculation server 200 may convert image data into locations in order to calculate a similarity to the image data. That is, each image data can be converted to a designated position of a discretized grid system.

So far, the personal identification system 1000 according to an embodiment of the present invention has been described. According to the present invention, all data transmitted and received between the identifier device 100 and the image calculation server 200 is in a state of homomorphic encryption, so there is no possibility of exposure of a user image while personal identification service is provided, and personal information is infringed. situation can be prevented.

Hereinafter, the identifier device 100 receiving the personal identification service will be described.

Referring to FIG. 2 , the identifier device 100 may include a memory interface 110 , one or more processors 120 and a peripheral interface 130 . The various components within identifier device 100 may be connected by one or more communication buses or signal lines.

The memory interface 110 may be connected to the memory 150 and transfer various data to the processor 120 . Here, the memory 150 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage, cloud , It may include at least one type of storage medium among blockchain databases.

In various embodiments, the memory 150 acquires an image of a user, and converts data constituting a personal identification interface screen for outputting a personal identification result, preprocessed first image data, and first image data in a form capable of homomorphic encryption. A function for conversion, an algorithm for homomorphic encryption, homomorphically encrypted first image data, parameters for homomorphic encryption operation, and the like may be stored.

In various embodiments, memory 150 includes operating system 151 , communication module 152 , graphical user interface module (GUI) 153 , sensor processing module 154 , telephony module 155 , and application module 156 . ) At least one or more of them may be stored. Specifically, the operating system 151 may include instructions for processing basic system services and instructions for performing hardware tasks. The communication module 152 may communicate with at least one of one or more other devices, computers, and servers. A graphical user interface module (GUI) 153 may process a graphical user interface. Sensor processing module 154 may process sensor-related functions (eg, process voice input received through one or more microphones 192 ). The phone module 155 may process phone-related functions. The application module 156 may perform various functions of a user application, such as electronic messaging, web browsing, media processing, navigation, imaging, and other processing functions. In addition, the identifier device 100 may store one or more software applications 156 - 1 and 156 - 2 (eg, a personal identification service application) associated with any one type of service in the memory 150 .

In various embodiments, the memory 150 may store a digital assistant client module 157 (hereinafter referred to as a DA client module), thereby storing instructions and various user data 158 for performing client-side functions of the digital assistant. (eg user-customized vocabulary data, preference data, other data such as the user's electronic address book, etc.).

Meanwhile, the DA client module 157 receives a user's voice input, text input, touch input, and/or gesture input through various user interfaces (eg, the I/O subsystem 140) provided in the identifier device 100. can be obtained

In addition, the DA client module 157 may output audio-visual and tactile data. For example, the DA client module 157 may output data consisting of a combination of at least two of voice, sound, notification, text message, menu, graphic, video, animation, and vibration. In addition, the DA client module 157 may communicate with a digital assistant server (not shown) using the communication subsystem 180 .

In various embodiments, DA client module 157 may collect additional information about the surrounding environment of identifier device 100 from various sensors, subsystems, and peripheral devices to construct a context associated with user input. . For example, the DA client module 157 may infer the user's intention by providing context information together with the user's input to the digital assistant server. Here, the situational information that may accompany the user input may include sensor information, eg, lighting, ambient noise, ambient temperature, image of the surrounding environment, video, and the like. For another example, the contextual information may include the physical state of the identifier device 100 (eg, device orientation, device location, device temperature, power level, speed, acceleration, motion pattern, cellular signal strength, etc.). For another example, the context information is information related to the state of the software of the identifier device 100 (eg, processes running on the identifier device 100, installed programs, past and present network activity, background services, error logs, resource usage). etc.) may be included.

In various embodiments, the memory 150 may include added or deleted commands, and the identifier device 100 may also include additional components other than those shown in FIG. 2 or may exclude some components.

The processor 120 may control the overall operation of the identifier device 100, and may execute various commands for implementing an interface for personal identification service by driving an application or program stored in the memory 150.

The processor 120 may correspond to an arithmetic device such as a central processing unit (CPU) or an application processor (AP). In addition, the processor 120 may be implemented in the form of an integrated chip (IC) such as a System on Chip (SoC) in which various computing devices performing machine learning, such as a Neural Processing Unit (NPU), are integrated. .

In various embodiments, the processor 120 may homomorphically encrypt an image and obtain an identification result of a homomorphically encrypted user based on the homomorphic encryption, which will be described below with reference to FIGS. 3 to 5 .

3 is a flowchart of a personal identification method of an identifier device according to an embodiment of the present invention, and FIGS. 4 and 5 are schematic diagrams for explaining a personal identification interface screen output to an identifier device according to an embodiment of the present invention. am.

Referring to FIG. 3 , the processor 120 may acquire first image data of the user (S110). For example, the processor 120 may acquire a user's image captured through the camera subsystem 170, and when using a vein distribution pattern rather than the user's image for personal identification, the communication module 152 Images of a finger and a wrist may be obtained from a wearable device (not shown) possessed by an individual user.

In this regard, referring to FIG. 4 , the processor 120 of the identifier device 100 may provide an interface screen for acquiring the user's first image data as shown in (a). Specifically, the interface screen may include a guideline 10 for obtaining a user's face image through the camera subsystem 170 and a user guide text 11 .

In addition to this, information on the location where the identifier device 100 is placed and time information for performing personal identification may be displayed together on the interface screen.

When acquiring the user image 12 within the guideline 10, the processor 120 may obtain the user image through the camera subsystem 170 as shown in (b).

In various embodiments, the processor 120 may homomorphically encrypt the entire image, but may also homomorphically encrypt feature points of the image according to a manager's setting. To this end, the processor 120 may extract feature data from the image. Specifically, the processor 120 calculates the rate of change of pixel values based on the horizontal/vertical position of the image (image differentiation), performs edge detection and corner detection using these values, Acquiring feature data (coordinate values of feature points), image histogram, gradient histogram descriptor, FAST (Features from Accelerated Segment Test), SIFT (Scale-Invariant Feature Transform), SURF (Speed- Feature data (coordinate values of feature points) may be acquired through a method such as Up Robust Features. Accordingly, for example, the processor 120 may extract coordinate values of the user's eyes, nose, and mouth from the image, and may also extract RGB values for each pixel of the image.

That is, the processor 120 uses an image captured through the camera subsystem 170 and a plurality of feature coordinate values extracted from the image ((User1=(E ₁ ,N ₁ , L ₁ ))), and any one of RGB values for each pixel of the image may be used as the user's first image data requiring personal identification. Among them, since the data size of the RGB values for each pixel of the image may be excessively large depending on the size or resolution of the image, the processor 120, for example, converts the RBG values at pixels corresponding to a plurality of feature coordinate values to the user's It can be used as the first image data.

In various embodiments, the processor 120 may pre-process the user's first image data in order to reduce the burden of the homomorphic encryption operation. For example, the processor 120 may convert image data into positions using a pre-stored function in order to calculate a similarity between the first image data and the second image data to be compared.

Referring back to FIG. 3 , after step S110, the processor 120 may homomorphically encrypt the first image data (S120). Specifically, the processor 120 may homomorphically encrypt the first image data using an encryption key in which parameters for homomorphic encryption operation are reflected.

In various embodiments, the processor 120 may encrypt the first image data using any one of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. Homomorphic encryption is possible.

After step S120, the processor 120 may transmit the homomorphically encrypted first image data to the image calculation server 200 through the communication module 152 (S130). The processor 120 may transmit to the image calculation server 200 an operation request including the homomorphically encrypted first image data and parameters used in the process of homomorphically encrypting the first image data. For example, parameters include the polynomial degree of a function used for homomorphic encryption operation, scale bits and coefficients specified for homomorphic encryption operation, and image attribute information (resolution, size). can be

After step S130, the processor 120 may receive a homomorphically encrypted identification result calculated based on the homomorphically encrypted first image data and pre-stored second image data of another user from the image calculation server 200 (S140). ). Here, the homomorphically encrypted identification result may be an identification result calculated based on the homomorphically encrypted first image data and the homomorphically encrypted second image data based on the parameters provided in step S130 .

That is, the first and second image data may be computed in a homomorphic encrypted state using an encryption key to which the same parameters are reflected, and accordingly, the identification result computed by the image computation server 200 may be correctly decrypted. can

Meanwhile, the processor 120 may receive a pre-stored identification result with image data of one other user or identification result with image data of a plurality of other users according to an operation request.

After step S140, the processor 120 may decrypt the homomorphically encrypted identification result (S150). Specifically, the processor 120 may output decoded results of different types to the touch screen 143 according to an operation request. For example, the processor 120 may check an identification result of whether the user matches another designated user or an identification result of whether the user is one of a plurality of users.

In this regard, referring to FIG. 5 , the processor 120 of the identifier device 100 determines whether or not the user is allowed access according to whether the user matches or does not match any one of the plurality of access users, as shown in (a). It is possible to provide a notification 13 indicating.

Referring back to FIG. 2 , the peripheral interface 130 may be connected to various sensors, subsystems, and peripheral devices to provide data so that the identifier device 100 can perform various functions. Here, that the identifier device 100 performs a certain function may be understood as being performed by the processor 120 .

Perimeter interface 130 may receive data from motion sensor 160, light sensor (light sensor) 161, and proximity sensor 162, through which identifier device 100 may receive orientation, light, and proximity. sensing function, etc. As another example, the peripheral interface 130 may receive data from other sensors 163 (positioning system-GPS receiver, temperature sensor, biometric sensor) through which the identifier device 100 may receive data from the other sensors. It can perform functions related to (163).

In various embodiments, the identifier device 100 may include a camera subsystem 170 coupled to the peripheral interface 130 and an optical sensor 171 coupled thereto, through which the identifier device 100 may take pictures and video Various shooting functions such as clip recording can be performed.

In various embodiments, identifier device 100 may include a communication subsystem 180 coupled with peripheral interface 130 . The communication subsystem 180 is composed of one or more wired/wireless networks, and may include various communication ports, radio frequency transceivers, and optical transceivers.

In various embodiments, identifier device 100 includes an audio subsystem 190 coupled to peripheral interface 130, which audio subsystem 190 includes one or more speakers 191 and one or more microphones 192. By including, the identifier device 100 can perform voice-activated functions, such as voice recognition, voice replication, digital recording, and telephony functions.

In various embodiments, identifier device 100 may include I/O subsystem 140 coupled with peripheral interface 130 . For example, the I/O subsystem 140 may control the touch screen 143 included in the identifier device 100 through the touch screen controller 141 . For example, the touch screen controller 141 uses any one of a plurality of touch sensing technologies such as capacitive, resistive, infrared, surface acoustic wave technology, proximity sensor array, and the like to provide a user's touch and motion or touch. and cessation of movement. For another example, I/O subsystem 140 may control other input/control devices 144 included in identifier device 100 via other input controller(s) 142 . As an example, other input controller(s) 142 may control one or more buttons, rocker switches, thumb-wheels, infrared ports, USB ports, and pointer devices such as styluses and the like.

So far, the identifier device 100 according to an embodiment of the present invention has been described. According to the present invention, the identifier device 100 may request an operation using homomorphically encrypted image data to the image operation server 200 in order to compare its own image data with image data of another user, and accordingly Accordingly, the identity of the user can be quickly confirmed while protecting the portrait right of the user.

Hereinafter, the image calculation server 200 providing a personal identification service will be described.

Referring to FIG. 6 , the image processing server 200 may include a communication interface 210, a memory 220, an I/O interface 230, and a processor 240, each component comprising one or more communication buses or They can communicate with each other through signal lines.

The communication interface 210 may be connected to a plurality of identifier devices 100 through a wired/wireless communication network to exchange data. For example, the communication interface 210 may receive an operation request including homomorphically encrypted first image data and parameters for a homomorphic encryption operation from the identifier device 100, and may receive the homomorphically encrypted first image data from the identifier device 100. Identification results can be transmitted.

On the other hand, the communication interface 210 enabling the transmission and reception of such data includes a communication pod 211 and a wireless circuit 212, where the wired communication port 211 is one or more wired interfaces, for example, Ethernet, This may include Universal Serial Bus (USB), FireWire, and the like. Also, the wireless circuit 212 may transmit/receive data with an external device through an RF signal or an optical signal. In addition, wireless communication may use at least one of a plurality of communication standards, protocols and technologies, such as GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

The memory 220 may store various data used in the image calculation server 200 . For example, the memory 220 may be configured to convert the second image data (user image, feature data (coordinate values, RGB values) extracted from the user image) for a plurality of users into a form capable of isomorphic encryption. It can store functions, algorithms for homomorphic encryption, etc.

In various embodiments, the memory 220 may include volatile or non-volatile recording media capable of storing various data, commands, and information. For example, the memory 220 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage storage , Cloud, and a blockchain database may include at least one type of storage medium.

In various embodiments, the memory 220 may store configurations of at least one of the operating system 221 , the communication module 222 , the user interface module 223 , and one or more applications 224 .

Operating system 221 (e.g. embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) is a variety of software for controlling and managing general system tasks (e.g. memory management, storage device control, power management, etc.) components and drivers, and may support communication between various hardware, firmware, and software components.

The communication module 223 may support communication with other devices through the communication interface 210 . The communication module 220 may include various software components for processing data received by the wired communication port 211 or the wireless circuit 212 of the communication interface 210 .

The user interface module 223 may receive a user's request or input from a keyboard, touch screen, microphone, etc. through the I/O interface 230 and provide a user interface on a display.

Applications 224 may include programs or modules configured to be executed by one or more processors 230 . Here, an application for computing image data may be implemented on a server farm.

The I/O interface 230 may connect at least one of an input/output device (not shown) of the image calculation server 200, for example, a display, a keyboard, a touch screen, and a microphone, to the user interface module 223. The I/O interface 230 may receive user input (eg, voice input, keyboard input, touch input, etc.) together with the user interface module 223 and process a command according to the received input.

The processor 240 is connected to the communication interface 210, the memory 220, and the I/O interface 230 to control overall operations of the image calculation server 200, and applications or programs stored in the memory 220. It is possible to perform various commands for processing homomorphically encrypted data through

The processor 240 may correspond to an arithmetic device such as a central processing unit (CPU) or an application processor (AP). In addition, the processor 240 may be implemented in the form of an integrated chip (IC) such as a System on Chip (SoC) in which various computing devices are integrated. Alternatively, the processor 240 may include a module for calculating an artificial neural network model, such as a Neural Processing Unit (NPU).

In various embodiments, the processor 240 may provide a service for identifying a user in a state in which the user's personal information is not exposed, which will be described below with reference to FIG. 7 .

Referring to FIG. 7 , the processor 240 may receive an operation request including homomorphically encrypted first image data of the user from the identifier device 100 through the communication interface 210 (S210). The operation request may include a parameter for a homomorphic encryption operation used to homomorphically encrypt the first image data and feature data extracted from the first image.

After step S210, the processor 240 may acquire pre-stored second image data of another user according to the operation request (S220). The processor 240 determines whether the operation request is for one other user's second image data or a plurality of other users' second image data, and the user's second image data stored in the memory 220 Image data can be loaded.

In addition, the processor 240 converts the first image data into an image captured through the identifier device 100, a plurality of feature coordinate values ((User1=(E ₁ ,N ₁ , L ₁ ))) and RGB values (feature data) for each pixel of the image may be checked, and second image data suitable thereto may be obtained.

The processor 240 may homomorphically encrypt another user's second image data stored in the memory 220 using an encryption key in which the same parameter of the homomorphically encrypted first image data is reflected. In addition, the processor 240 may homomorphically encrypt second image data of one other user stored in the memory 220 or homomorphically encrypt second image data of a plurality of other users stored in the memory 220 according to an operation request. can

In various embodiments, the processor 240 may perform homomorphic encryption using any one of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. .

Meanwhile, the processor 240 may acquire the previously stored second image data of another user and may not perform homomorphic encryption.

After step S220, the processor 240 may calculate a homomorphically encrypted identification result based on the homomorphically encrypted first image data and the second image data (S230). Specifically, the processor 240 converts the two image data to a distance similarity calculation method (eg, Euclidean distance measurement method, Minkowski distance measurement method, Cosine similarity calculation method, The similarity between image data (image identification result) may be calculated by calculating using a mean squared difference similarity calculation method, a Pearson similarity calculation method, or the like.

To this end, the processor 240 may determine a first position corresponding to the homomorphically encrypted first image data and a second position corresponding to the second image data. For example, the processor 240 may determine a location corresponding to each image data as a real number value or a location vector using a hexagonal hierarchical spatial index (H3) system.

The processor 240 may obtain a value corresponding to an identification result by calculating a distance value between the first location and the second location using the aforementioned distance similarity calculation method. For example, when the calculated distance value is included in a predetermined distance range, the processor 240 may calculate a homomorphic encrypted identification result including a result that the two image data are similar, and the calculated distance value is If it is not included in the predetermined distance range, a homomorphic encrypted identification result including a result indicating that the two image data are not similar may be calculated.

In various embodiments, the processor 240 may calculate an encrypted identification result based on the first image data that is homomorphically encrypted with the second image data of one or more other users, according to the type of operation request.

Meanwhile, since the above identification result is homomorphically encrypted, the processor 240 cannot determine a similarity result between the two image data.

After step S230, the processor 240 may transmit the homomorphically encrypted identification result to the identifier device 100 (S240). The homomorphic encrypted identification result can be decrypted by the identifier device 100, and accordingly, the processor 240 does not acquire a user's image in a format such as jpg, png, or pdf, and stores a plurality of images stored in the memory 220. Comparison with other users' images and identification results may be calculated and provided to the identifier device 100.

So far, the image calculation server 200 according to an embodiment of the present invention has been described. According to the present invention, a user's own image is homomorphically encrypted and decrypted in a state stored in each safe device, and the image calculation server 200 delivers only the homomorphically encrypted calculation result, thereby minimizing the risk of processing sensitive information. can

Hereinafter, a personal identification method between the identifier device 100 and the image calculation server 200 will be briefly described.

Referring to FIG. 8 , the identifier device 100 may acquire an image of a user (S10) or, in addition, extract feature data from the image (S11), and homomorphically encrypt the image or feature data (first image data). It can (S12).

The identifier device 100 may transmit parameters for homomorphic encryption calculation to the image calculation server 200 together with the homomorphically encrypted first image data. Here, the parameter for the homomorphic encryption operation may be a parameter applied to an encryption key of the homomorphically encrypted first image data.

The image operation server 200 may homomorphically encrypt previously stored second image data using parameters (S14), and calculate a homomorphically encrypted identification result based on the homomorphically encrypted first and second image data (S15). ) (that is, it can operate on homomorphic encrypted data). Specifically, the image calculation server 200 may determine locations corresponding to the homomorphically encrypted image data, calculate a distance between the locations, perform calculations between the homomorphically encrypted data, and generate a value corresponding to the identification result. can be obtained

In various embodiments, the image calculation server 200 may perform a comparison operation between the plaintext second image data and the homomorphically encrypted first image data without encrypting the previously stored second image data.

The image calculation server 200 transmits the encrypted calculation result to the identifier device 100 (S16), the identifier device 100 decrypts the calculation result (S17), and outputs the decryption result on the display screen. (S18).

Meanwhile, calculation between homomorphic encrypted data may be performed in the identifier device 100 in the same manner as in the image calculation server 200 .

In relation to this, referring to FIG. 9 , steps S20 to S21 are the same as before, but the identifier device 100 may selectively perform homomorphic encryption on the image or feature data (S22).

Thereafter, the identifier device 100 may transmit a data identification request including parameters for isomorphic encryption operation to the image operation server 200 (S23).

The image calculation server 200 may homomorphically encrypt a plurality of pre-stored second image data using the same parameters as previously received parameters and transmit the same to the identifier device 100 according to the data identification request.

That is, since the image calculation server 200 provides the second image data that is homomorphically encrypted, the identifier device 100 may calculate a homomorphically encrypted identification result based on the homomorphically encrypted first and second image data ( S25) (that is, homomorphic encrypted data can be calculated).

The identifier device 100 may transmit the encrypted calculation result to the image calculation server 200 (S26), and the image calculation server 200 may decrypt and transmit the calculation result again (S27).

Finally, the identifier device 100 may output a decoded result (S28), and the result may be, for example, whether or not the image of the user and another user matches, and the user's identification information.

Although one embodiment of the present invention has been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be variously modified and implemented without departing from the technical spirit of the present invention. there is. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

Claims

obtaining first image data of a user;

Homomorphically encrypting the first image data;

Transmitting homomorphically encrypted first image data to an image calculation server;

receiving a homomorphically encrypted identification result calculated based on the homomorphically encrypted first image data and previously stored second image data of another user from the image calculation server; and

decrypting the homomorphically encrypted identification result; Personal identification method using a homomorphic encrypted image comprising a.
According to claim 1,

In the step of transmitting the homomorphically encrypted first image data,

The personal identification method using a homomorphic encrypted image, further comprising: transmitting a parameter for homomorphic encryption operation, used to homomorphically encrypt the first image data, to the image operation server.
According to claim 2,

The homomorphic encrypted identification result,

wherein the identification result is calculated based on the homomorphically encrypted first image data and the second image data homomorphically encrypted based on the parameters.
According to claim 1,

The second image data,

Image data of a plurality of other users pre-stored in the image calculation server;

The decryption step is

Acquiring an identification result for another user matched with the user among the plurality of other users;
According to claim 1,

The first image data and the second image data,

A personal identification method using a homomorphic encrypted image, comprising at least one of an image captured through an identifier device, a plurality of feature coordinate values extracted from the image, and RGB values for each pixel of the image.
According to claim 1,

In the homomorphic encryption step,

Personal identification method using a homomorphic encrypted image, which is a step of homomorphic encryption using any one of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. .
Receiving an operation request including homomorphically encrypted first image data of a user from an identifier device;

obtaining pre-stored second image data of another user according to the operation request;

calculating a homomorphically encrypted identification result based on the homomorphically encrypted first image data and the second image data; and

sending the homomorphically encrypted identification result to the identifier device; Personal identification method using a homomorphic encrypted image comprising a.
According to claim 7,

Receiving the operation request,

Receiving parameters for a homomorphic encryption operation used to homomorphically encrypt the first image data from the identifier device;

The obtaining step is

Homomorphically encrypting the second image data based on the parameter; The personal identification method using a homomorphically encrypted image, further comprising:
According to claim 7,

Calculating the homomorphic encrypted identification result,

determining a first position corresponding to the homomorphically encrypted first image data and a second position corresponding to the second image data; and

Calculating a distance value between the first location and the second location corresponding to the identification result;
According to claim 7,

The second image data,

Second image data of a plurality of other users stored in advance;

Calculating the homomorphic encrypted identification result,

calculating an encrypted identification result based on the second image data of the plurality of other users and the homomorphically encrypted first image data according to the type of the received operation request; method.
According to claim 7,

The first image data and the second image data,

An individual identification method using a homomorphic encrypted image, comprising at least one of an image captured through the identifier device, a plurality of feature points extracted from the image, and RGB values for each pixel of the image.
According to claim 8,

In the homomorphic encryption step,

Personal identification method using a homomorphic encrypted image, which is a step of homomorphic encryption using any one of partial homomorphic encryption, somewhat homomorphic encryption, and fully homomorphic encryption. .