KR20210110765A - Method for providing ai-based big data de-identification solution - Google Patents

Abstract

Disclosed is a method for providing an artificial intelligence-based big data de-identification solution comprising: a step of collecting big data; a step of identifying personal information included in the big data; and a step of de-identifying the identified personal information. Therefore, the present invention is capable of having an effect of being able to safely process atypical/semi-structured personal information.

Description

{METHOD FOR PROVIDING AI-BASED BIG DATA DE-IDENTIFICATION SOLUTION}

The present invention relates to a method of providing an artificial intelligence-based big data de-identification solution.

With the passage of the Data 3 Act on January 9, 2020, measures to de-identify personal information for the use of big data are emerging as essential, and de-identified personal information must be provided.

According to the recent trend toward the data economy, the use of big data is inevitable, and the problems of personal information infringement that may occur as a result are continuously increasing. For this, it is necessary to provide a personal information de-identification solution, and the manual processing method provided by existing solutions has limitations, so an improved intelligent processing process is needed.

Currently, the security market including personal information de-identification solutions can be viewed as an early stage, and furthermore, as a solution that can be applied to all fields that use data, the value that can be used in the market is expected to increase continuously.

A personal information de-identification solution is an essential requirement for the development of the big data industry, and personal information de-identification means de-identification processing such as pseudonymization, total processing, data deletion, data categorization, and data masking when using personal information. do.

The personal information de-identification solution aims to de-identify such personal information safely and quickly with an automated process procedure equipped with artificial intelligence in accordance with the contents of the personal information de-identification guideline.

The problem to be solved by the present invention is to provide an artificial intelligence-based big data de-identification solution.

It aims to present a safe use method through de-identification of personal information in situations where there is concern about personal information exposure due to the use of big data due to the revision of the Data 3 Act.

Specifically, in the step of collecting data in an atypical format, morpheme classification is performed through pre-processing, and personal information is identified in real time using the processed data. and securely de-identify, store and utilize.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Personal information that is easily omitted from structured, unstructured, and semi-structured data, which is a big data collection target, is accurately identified and de-identified through AI machine learning through natural language processing. By using memory-based Spark distributed processing, CDC technology, and KorBERT module, which is the core of natural language processing elements, it can present superior performance compared to existing competitor products.

In addition, by developing a personal information de-identification adequacy evaluation module, it is possible to evaluate the adequacy by simulating personal information inference attacks such as connection attack, homogeneity attack, background knowledge attack, concentration attack, and similarity attack. Restful API for interworking with other solutions Through linkage, stable linkage can be achieved without changing the existing environment.

Other specific details of the invention are included in the detailed description and drawings.

According to the disclosed embodiment, there is an effect that not only standardized personal information but also atypical/semi-structured personal information can be identified and safely processed.

In addition, it is possible to quickly and safely de-identify personal information based on an automated system, which has the advantage of not only protecting personal information but also expecting high economic effects.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 to 3 are diagrams illustrating a big data de-identification solution according to the disclosed embodiment.
4 is a diagram illustrating a configuration of a non-identification model according to an exemplary embodiment.
5 is a diagram illustrating an appropriateness determination model according to an exemplary embodiment.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

In this specification, a computer means all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 to 3 are diagrams illustrating a big data de-identification solution according to the disclosed embodiment.

The method of providing a big data de-identification solution according to the disclosed embodiment may include various technical elements to be described later. Each technical element may operate individually or as a combined one, and the specific combination method is not limited.

In an embodiment, the computer may perform a natural language processing-based personal information identification operation. For example, a computer identifies personal information through learning based on existing detection patterns and natural language processing.

In the case of existing products, it is common to find and de-identify data records stored in the DB or in a standardized format. In contrast, in the case of the solution according to the disclosed embodiment, primary personal information detection is performed by personal information detection using a regular expression pattern engine. Personal information is identified, and in the case of an image, a character is recognized through OCR (Optical Character Recognition) and then analyzed.

In the case of primary personal information detection, a logic to detect known standardized information such as resident number, passport number, driver's license number, alien registration number, phone number, card number, and account number is used.

In the case of secondary personal information detection, a method of detecting atypically generated personal information other than predetermined personal information through context analysis through AI-based data mining is used.

For example, personal information can be identified by analyzing the classification and frequency of phrases subjected to natural language processing such as TDM and IF-IDF. Examples of personal information items that are difficult to find in a formal pattern include, but are not limited to, personal information in various forms such as ideology, religion, hobbies, habits, sex life, location, life flow, relationships, consumption patterns, etc. Even if it is not legally specified, de-identification is necessary if it is information that can identify an individual.

In one embodiment, the computer performs personal information detection to which artificial intelligence AI learning techniques are applied.

For example, the computer may perform personal information pattern analysis through a data mining technique. Specifically, in the case of unstructured data, personal information that is difficult to find with a specific pattern can be distinguished through data mining, and phrases parsed by natural language processing can be processed into meaningful forms to extract patterns according to the flow of context. In addition, a generative adversarial network (GAN), which is a representative method of unsupervised learning, may be used as the analysis algorithm, but is not limited thereto.

In one embodiment, the computer may perform real-time personal information detection and event processing to which CDC (Change Data Capture) technology is applied.

For example, the computer may capture the collected event and match the changed content with the analysis engine in real time to determine whether to process it. Real-time processing is an important factor influencing performance in a large-capacity big data environment, and the solution according to the disclosed embodiment can be designed with a high throughput compared to previously released personal information de-identification solutions. In addition, the event processing status can be reflected and displayed in real time through a dashboard or a visibility tool.

With the CDC technology, when collecting large or new personal information, personal information is detected by a regular expression method, and data that is changed or generated afterward can be detected and analyzed by capturing only the changed contents.

Continuously searching for personal information for a large amount of data is inefficient and takes a lot of performance or time, so you can apply CDC technology to check only the changed event log and identify it in real time.

In an embodiment, a differential privacy model is added in addition to the existing k-anonymity, L-diversity, and T-accessibility privacy models to differentiate the personal information non-identification model.

4 is a diagram illustrating a configuration of a non-identification model according to an exemplary embodiment.

Currently, many companies perform de-identification processing by applying only representative de-identification privacy models of k-anonymity, L- diversity, and T-accessibility. apply the treatment.

Specifically, secure de-identification is difficult with only the k·L·T privacy model, and an additional method is required. Accordingly, a technique of protecting by applying differential privacy and adding noise to each response value so that the distribution of response values has a difference of less than a certain standard is applied.

For example, the existing privacy model may be vulnerable to a connection attack that analyzes different data by linking it (eg, a connection attack that extracts personal information by connecting public medical data and electoral lists).

A k-anonymity vulnerability can occur due to homogeneity attacks and attacks based on background knowledge, and L-diversity is used to solve this. Records that are de-identified together in a data set to become a homogeneous set (a set grouped with the same value by k-anonymity) have at least one different sensitive information.

However, even in the case of L-diversity, it is vulnerable to concentration and similarity attacks, and T-proximity is applied to defend against L-diversity attacks. This means that the distribution of specific information must show a difference of less than t within the homogeneous set and within the entire data set.

Furthermore, the solution according to the disclosed embodiment applies a differential privacy model, which may use the techniques shown in the table below.

type technique Explanation noise
additional technique PCA Bayesian Estimation 1. Extract original data X
2. PCA application
3. Convert to Xp
4. Add Laplace Noise
5. Create disturbance data Y Noise leveling Measuring the distance to the average value of a set of entries rather than calculating the distance to each entry in the perturbed data Area-based Disturbance Techniques Divide the time series data into congruent and inconsistent areas and add more noise for congruent areas compressor board
Disturbance Techniques Discrete Fourier Transform (DFT) technique Data mining using only a few Fourier coefficients that reflect the characteristics of the original data rather than the entire data DWT (Discrete Wavelet Transform) technique Only significant coefficients with amplitudes greater than a set threshold perturb geometric
Disturbance Techniques rotational disturbance technique Data perturbation through rotation, translation, etc. Multiple Rotation Techniques Original normalization -> Generate n random seeds -> Orthogonal matrix -> Rotate original using orthogonal matrix -> Generate perturbed data Y Condensation Disturbance Techniques The original data is divided into k condensed groups, and the central object of each group is selected, and the remaining k-1 objects are newly created so that the distribution is similar to the original data.

In one embodiment, the computer secures traceability of personal information usage history. For example, in order to track the usage history of personal information to which privacy is applied, an audit trail of the provided details may be performed when de-identified personal information is used, and various statistical information may be provided by logging related contents. In addition, it is possible to provide a GUI screen that is intuitive and capable of multidimensional analysis through the use of visibility tools.

In addition, according to the disclosed embodiment, a function for examining whether de-identification is appropriate may be provided.

5 is a diagram illustrating an appropriateness determination model according to an exemplary embodiment.

For example, after taking measures to de-identify personal information, sample data can be delivered to the adequacy engine. If the de-identification action of the transmitted data is appropriate, it can be used immediately, and if it is not appropriate, re-identification processing can be performed. By applying this adequacy verification logic to the system, adequacy can be automatically verified, and it can be identified by determining whether it is vulnerable to attacks such as non-identifying data attacks such as connection attack, homogeneity attack, background attack, concentration attack, and similarity attack. have.

In addition, an export function according to the disclosed embodiment may be provided. The export function means to transfer the identified personal information to the target data after de-identification such as total processing, average value, categorization, anonymization, deletion, and masking, and the delivery method is API linkage, FTP, SFTP, DB migration method, etc. It can be delivered in a way that suits the environment of the time.

In addition, a complete deletion function may be provided according to the disclosed embodiment. The complete deletion function is an optional function to delete the log after providing de-identified personal information, and the administrator can choose whether to completely delete it. For example, it may be necessary to completely delete personal information when it is destroyed due to the expiration of the life cycle of personal information.

In addition, a management screen according to the disclosed embodiment may be developed and provided.

For example, the basic design intent of the admin screen is to develop a HTML5-based UI that is concise, intuitive, easy to use, and emphasizes UI unity and maintenance convenience. Features of HTML5 include compatibility (guaranteeing compatibility with existing HTML, compatibility with RESTful API), practical design (introducing safe security and useful new functions), and optimization of mobile environment (no need to install unnecessary agents because activeX or plug-in is not installed) ), multidimensional graphing and visibility support (best for presentation of analytics and visibility). In addition, a policy setting function may be provided through the administrator screen.

For example, the functions shown in the table below may be provided as functions of the manager screen.

division Detailed function characteristic management function Administrator authentication - ID/PW, certificate, biometric authentication, etc. Grant Administrator Privileges - Divided into super administrator, advanced, intermediate, beginner, etc. to grant different privileges Settings backup function - Setting backup function such as set policy and IP information Account and password rules - Reflecting the regulations in accordance with the certification guidelines access control function - Only authorized accounts and IPs, macs, and devices can be accessed Policy setting Schedule function for de-identification - Giving selection of work type (reflected immediately, performed at a specific time) Choosing a De-Identification Method - Select methods such as totals, categorization, anonymity, pseudonym, masking, deletion, etc. Non-identifying adequacy verification function - Validation of de-identified data - Re-performed depending on whether safety is verified data delivery schedule - Provision of delivery function for de-identified data - immediate delivery, scheduled delivery Instant de-identification processing capabilities - Immediate de-identification processing function only for un-identified unprocessed contents Complete deletion function - Due to the nature of personal information, it is necessary to completely delete personal information whose life cycle and purpose of use have expired.
- Use degaussing or wiping tools etc audit logging - Administrator behavior log, system log, de-identification processing log, data provision log, etc. Link setting - Linking restful API with other solutions Statistics and Reporting - Development of statistics and reporting functions for various analysis

In addition, a monitoring function according to the disclosed embodiment may be provided. As the monitoring function, various types of monitoring functions such as server status monitoring, policy violation monitoring, de-identification processing monitoring, and data provision status monitoring can be provided. may be

In addition, a Restful API linkage function may be provided. It aims to use the existing infrastructure as it is by using the HTTP protocol, and the roles of the server and the client are clearly separated. In addition, by applying the caching function, which is the strongest feature of HTTP, the overall response time, performance, and resource utilization with the server are improved. In addition, structural flexibility can be secured.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. The components of the present invention may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , Java, assembler, etc. may be implemented in a programming or scripting language. Functional aspects may be implemented in an algorithm running on one or more processors.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (3)

collecting big data;
identifying personal information included in the big data; and
de-identifying the identified personal information; containing,
A method of providing an artificial intelligence-based big data de-identification solution. According to claim 1,
The step of identifying the personal information,
using natural language processing to identify not only standardized personal information but also unstructured/semi-structured personal information; containing,
A method of providing an artificial intelligence-based big data de-identification solution. According to claim 1,
The step of de-identifying the personal information,
de-identifying the personal information using a k-anonymity model, L-diversity model, T-proximity model and differential privacy model; containing,
A method of providing an artificial intelligence-based big data de-identification solution.

Images