CN112163214A - Data access method and device - Google Patents
Data access method and device Download PDFInfo
- Publication number
- CN112163214A CN112163214A CN202011005438.0A CN202011005438A CN112163214A CN 112163214 A CN112163214 A CN 112163214A CN 202011005438 A CN202011005438 A CN 202011005438A CN 112163214 A CN112163214 A CN 112163214A
- Authority
- CN
- China
- Prior art keywords
- data
- storage area
- development
- desensitization
- production
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000000586 desensitisation Methods 0.000 claims abstract description 180
- 238000011161 development Methods 0.000 claims abstract description 178
- 238000004519 manufacturing process Methods 0.000 claims abstract description 143
- 238000012545 processing Methods 0.000 claims description 77
- 230000004044 response Effects 0.000 claims description 18
- 238000012795 verification Methods 0.000 claims description 8
- 238000010200 validation analysis Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 24
- 238000007726 management method Methods 0.000 description 20
- 230000001360 synchronised effect Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 230000001960 triggered effect Effects 0.000 description 7
- 238000013523 data management Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/45—Structures or tools for the administration of authentication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
- G06F21/78—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer to assure secure storage of data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The specification discloses a method and apparatus for data access. In the embodiment of the specification, a production storage area for storing original business data and a development storage area for storing desensitization data are divided from a storage area of a data platform. And simultaneously, determining two user roles of a production tenant and a development tenant, wherein the production tenant can access the production storage area, and the development tenant can access the development storage area. The original service data and the desensitization data are respectively stored in two storage areas which are isolated from each other, and the access authority of the user can be determined based on the user role, so that the development tenant can only access the data in the development storage area without contacting the original service data, and the safety of the sensitive data of the user is ensured.
Description
Technical Field
The present specification relates to the field of big data, and in particular, to a method and an apparatus for data access.
Background
In the internet era, sensitive data of users is collected by a large number of organizations, such as a personalized certificate number, a contact address, etc., which are filled in when an account number is registered. For an organization, how to ensure the security of user sensitive data becomes a problem to be solved urgently while realizing services based on the user sensitive data.
Disclosure of Invention
In view of the above, the present specification provides a method and apparatus for data access.
Specifically, the description is realized by the following technical scheme:
according to a first aspect of the present application, there is provided a method for data access, applied to a data platform, the method including:
dividing a production storage area and a development storage area from a storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
determining a user role for a user accessing the data platform;
if the user role of the user is a production tenant, determining that the user has the authority of accessing a production storage area;
and if the user role of the user is a development tenant, determining that the user has the authority of accessing a development storage area.
According to a second aspect of the present application, there is provided an apparatus for data access, which is applied to a data platform, the apparatus comprising:
a storage area dividing unit for dividing a production storage area and a development storage area from the storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
the user role determining unit is used for determining the user role of the user accessing the data platform;
the permission determining unit is used for determining that the user has the permission to access the production storage area under the condition that the user role of the user is a production tenant; and determining that the user has the authority to access the development storage area under the condition that the user role of the user is a development tenant. According to a third aspect of the present application, there is provided an electronic device comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of the first aspect by executing the executable instructions.
According to a fourth aspect of the present application, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of the first aspect described above.
One embodiment of the specification realizes that a production storage area for storing original service data and a development storage area for storing desensitization data are divided from a storage area of a data platform. And simultaneously, determining two user roles of a production tenant and a development tenant, wherein the production tenant can access the production storage area, and the development tenant can access the development storage area.
The method of the specification stores original service data and desensitization data in two storage areas which are isolated from each other respectively, and can determine the access authority of a user based on the role of the user. Therefore, the development tenant can only access the data in the development storage area and can not contact the original service data, and the safety of the user sensitive data is guaranteed.
Drawings
FIG. 1 is a flow chart diagram illustrating a method of data access in an exemplary embodiment of the present description.
FIG. 2 is a schematic diagram of a data platform shown in an exemplary embodiment of the present description.
FIG. 3 is a flow diagram illustrating another method of data access in an exemplary embodiment of the present description.
Fig. 4 is a schematic structural diagram of a data platform according to an exemplary embodiment of the present disclosure.
Fig. 5 is a schematic structural diagram of an apparatus for data access according to an exemplary embodiment of the present disclosure.
Fig. 6 is a block diagram illustrating an apparatus for data access in an exemplary embodiment of the present description.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.
The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
For some management departments, a large amount of data information is generally required to be processed, for example, for a human department of an enterprise, a large amount of employee attendance information is required to be processed; for the financial department, a large amount of receipt and payment information needs to be processed.
To improve work efficiency, these authorities may use big data technology to assist in data processing. In general, a management department may first establish a data platform and then store the collected original service data in the data platform. Based on the stored raw business data, the managing department may delegate other departments or companies to develop the specified software logic and then use that software logic for business processing. For example, the human department may entrust the research and development department to develop a set of software logic to assess the attendance of employees.
However, the raw service data stored by the data platform typically includes a large amount of sensitive data, such as certificate numbers, contact addresses, and the like. For the management department, other departments or companies are entrusted to develop software logic aiming at the original business data, and sensitive data are guaranteed not to be leaked. Usually, a data desensitization technology can be used to desensitize original service data by a certain method, thereby protecting the security of sensitive data.
In a related technology, a data platform performs desensitization processing on acquired original service data and then stores the desensitization processing, so that all people cannot touch sensitive data. Although the method can well ensure the security of sensitive data, even a management department cannot perform business processing based on real original business data, and deviation of processing results is easily caused.
In another related art, the collected original service data may be directly stored in a data platform. Subsequently, when a developer inquires data, the data platform judges whether the inquired data needs desensitization according to desensitization rules, if so, the original service data is desensitized and then returned, so that the developer can not contact the sensitive data; when the management department inquires the data, the data platform directly returns the original service data, so that the management department can perform service processing based on the original service data. However, the security of this method is low, because the original service data is still stored in the data platform, only the desensitization processing is performed according to the desensitization rule when the data is returned, and the developer can perform preset processing (for example, encoding or encryption processing) on the stored original service data, so that the data platform determines that the data does not need to be desensitized, and thus the developer can bypass the desensitization policy and obtain the original service data after the preset processing. Therefore, developers can restore the original service data after the preset processing to obtain the original service data without desensitization.
In view of this, in order to ensure the security of sensitive data while processing the service normally, the present application provides a method for data access.
Referring to fig. 1, fig. 1 is a flow chart illustrating a method for data access, which is applied to a data platform, according to an exemplary embodiment of the present disclosure.
As shown in fig. 1, the method may include the steps of:
step S102: dividing a production storage area and a development storage area from a storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data.
In this embodiment, the storage area of the data platform is at least divided into two storage areas, which are a production storage area and a development storage area. As shown in fig. 2, the production storage area and the development storage area are isolated from each other, wherein the production storage area is used for storing original business data, and the development storage area is used for storing desensitization data.
In this embodiment, the original service data in the production storage area usually contains a large amount of sensitive data.
In one example, desensitization processing may be performed on original service data in the production storage area, and after desensitization data is obtained, the desensitization data is synchronized to the development storage area.
In one example, for the collected original service data, on one hand, the original service data may be saved in a production storage area, and on the other hand, after desensitization processing is performed on the original service data, the obtained desensitization data may be saved in a development storage area.
Data desensitization is a technology of converting real data into virtual data and then storing or displaying the virtual data, and can prevent user sensitive data from being abused. The present specification will specifically describe the method for desensitizing data in conjunction with the flow shown in fig. 3, which is not repeated here.
In this embodiment, other storage areas, such as a shared storage area, may also be divided from the storage area of the data platform, which is not limited herein.
Step S104: a user role of a user accessing the data platform is determined.
In this embodiment, the user role may be predetermined for the user. In one example, user roles can include production tenants and development tenants.
Production tenants are usually owners of original business data, and development tenants are usually developers. For example, for an attendance management platform, a production tenant may be a human department that collects attendance data, and a development tenant may be a research and development department that is entrusted with developing software logic.
Of course, other user roles may also be included, such as managing tenants, etc., which are not limited herein.
Step S106: and if the user role of the user is a production tenant, determining that the user has the authority of accessing a production storage area.
Step S108: and if the user role of the user is a development tenant, determining that the user has the authority of accessing a development storage area.
In this embodiment, for different user roles, access permissions for different storage regions are determined.
As shown in fig. 2, for a development tenant, it may be determined that it has an access right to access a development storage area; for a production tenant, it may be determined that it has access to a production storage area.
When the method is implemented, the data platform can establish the corresponding relation between the user role and the storage area in advance. Further, upon receiving a user access, a user role of a user accessing the data platform may be first determined, and then based on the user role, it may be determined that it has a right to access the corresponding storage area.
Of course, the data platform may also determine correspondence between other user roles and the storage areas, for example, for a management tenant, it may be determined that it has access rights to all storage areas. Or, the data platform may further determine a correspondence between the role and another storage area, for example, for a shared storage area, it may be determined that both the development tenant and the shared tenant have an access right to the storage area. And will not be described in detail herein.
The method of the specification stores original service data and desensitization data in two storage areas which are isolated from each other respectively, and can determine the access authority of a user based on the role of the user. Therefore, the development tenant can only access the data in the development storage area and can not contact the original service data, and the safety of the user sensitive data is guaranteed.
In addition, in the method of the present specification, the production storage area and the development storage area are two storage areas isolated from each other in the data platform, and are not physically isolated. Therefore, when the data in the production storage area is synchronized to the development storage area, the data transmission efficiency is high and the safety is high.
In the method of this specification, desensitization data in the development storage area may be obtained by desensitizing original business data in the production storage area, and then, the development tenant may perform software logic development based on the desensitization data, which is specifically described below with reference to fig. 3.
Referring to fig. 3, fig. 3 is a flow chart illustrating another method of data access according to an exemplary embodiment of the present disclosure.
As shown in fig. 3, the method may include the steps of:
step S302: and desensitizing the original service data in the production storage area by adopting a preset desensitizing rule to obtain desensitized data.
In this embodiment, the data platform may be as shown in fig. 2. The storage area of the data platform is divided into at least two storage areas, namely a production storage area and a development storage area. The production storage area and the development storage area are isolated from each other, wherein the production storage area stores original service data, and the development storage area is used for storing desensitization data.
In this embodiment, the production storage area may store original service data. Optionally, first, metadata of the data table may be created, where the metadata may include data table attributes of the original service data, such as a data table name, a column name, description information, and the like. Optionally, the metadata may also include a data table format or the like. And then, based on the original service data acquired by the production tenant, and according to the corresponding relation between the metadata specified by the production tenant and the original service data, the metadata and the original service data are stored in an associated manner. For example, a production tenant may import data tables of raw business data, as well as metadata, into the data platform.
For example, for an attendance management platform, the raw business data stored in association therewith may be as shown in table 1. The metadata corresponding to table 1 may include a table name, a column name, etc., for example, the table name of table 1 is an attendance data table, etc., and the column name of table 1 includes a name, a contact address, a time on duty, and a time off duty.
| Name (I) | Contact means | Working hours | Off duty time |
| Zhang three | 13500010001 | 9:00 | 17:00 |
| Li four | 13500010002 | 10:00 | 17:00 |
TABLE 1
As shown in table 1, the original service data includes attendance information of zhang san and lie san, for example, the contact address of zhang san is 13500010001, and the attendance time is 9:00 and 17:00 respectively.
In this embodiment, the preset desensitization rule is to process the designated data by using a corresponding desensitization method. Common desensitization methods include encryption processing, obfuscation processing, and the like. For example, the encryption process may include encryption using the AES algorithm, or the like; the obfuscation process may include hiding a number of bits of the original character, etc. Optionally, for different types of original service data, the same desensitization method may be used for desensitization; or different desensitization methods can be adopted to perform desensitization processing, for example, a preset encryption algorithm is used for encrypting the Chinese characters, and the digital characters are hidden.
Taking the name "zhang san" in table 1 as an example, if the encryption is performed by using the AES algorithm, desensitization data may be obtained as "ezuxiq 6Yh +90F1 iefthka ═ or"; if a blurring process is used, desensitization data can be obtained as "flare".
In this embodiment, desensitization processing may be performed on all original service data, or only desensitization processing may be performed on part of the original service data, which is not limited in this embodiment.
Taking desensitization processing on the original business data shown in table 1 as an example, if fuzzy processing is performed on both names and contact addresses, desensitization data shown in table 2 can be obtained.
| Name (I) | Contact means | Working hours | Off duty time |
| Zhang | 135****0001 | 9:00 | 17:00 |
| Plum | 135****0002 | 10:00 | 17:00 |
TABLE 2
In one example, the desensitization rule may also include conditions that trigger desensitization processing, such as desensitization cycles. The data platform can perform desensitization processing on the original service data according to the desensitization period.
Of course, other contents may be included in the desensitization rule according to actual requirements, which are only exemplary and not limiting.
In this embodiment, in addition to the condition for triggering desensitization processing included in the desensitization rule, desensitization operation of the data platform on the original service data may also be triggered by various other methods.
In one example, the desensitization processing operation may be triggered by desensitization instructions issued by the production tenant. In response to the desensitization instruction of the production tenant, the data platform performs desensitization processing on the original service data in the production storage area by adopting a preset desensitization rule to obtain desensitization data.
For example, the data platform may provide a visualization interface with controls exposed therein for indicating desensitization processing. Through the control, the production tenant can trigger the data platform to perform desensitization processing on the original service data.
Optionally, the data platform may allow the production tenant to issue desensitization instructions, while forbidding the development tenant to issue desensitization instructions. In implementation, the data platform may determine whether to expose the control according to the user authority of the login platform, for example, when a production tenant logs in the data platform, the control for instructing desensitization processing is exposed; when a development tenant logs on to the data platform, the control for indicating desensitization processing is not exposed. Or the data platform may show the control for all users, but when the development tenant issues an instruction through the control, information such as no operation authority is prompted.
In another example, the desensitization instruction of the production tenant may also be a preset periodic desensitization instruction. For example, the production tenant may set a cycle time of the desensitization process through a visual interface of the data platform, for example, 1 day, 10 days, and the like, and the data platform periodically performs an operation of the desensitization process according to the cycle time set by the production tenant.
In another example, the data platform may also be independent of instructions of the production tenant and automatically perform desensitization processing. For example, the data platform automatically performs the desensitization processing operation when detecting the original service data update.
Of course, the above embodiments are only exemplary, and the operation of the desensitization process may be triggered by other ways in practical applications, which is not limited herein.
Step S304: and synchronizing the desensitization data to a development storage area so that a development tenant performs software logic development based on the desensitization data stored in the development storage area.
Alternatively, the desensitization process in step S302 and the desensitization data synchronization in this step may be performed in association. That is to say, when a desensitization operation is triggered, the data platform performs desensitization processing on original service data, and then directly synchronizes the desensitization data to the development storage area. Alternatively, the desensitization process in step S302 and the desensitization data synchronization in this step may be performed separately. That is, the data platform performs desensitization processing on original service data in advance, stores desensitization data, and then synchronizes the desensitization data to the development storage area when synchronization operation is triggered.
For the situation that desensitization processing and desensitization data are executed synchronously and respectively, the synchronization operation of the data platform on desensitization data may be triggered in various ways, for example, the synchronization operation may be triggered according to a synchronization instruction issued by a production tenant, or the data platform may automatically execute, for example, when desensitization data update is detected, the synchronization operation is automatically executed. In this embodiment, the method for triggering the synchronous operation is similar to the method for triggering the desensitization operation, and is not described here again.
The synchronization operation in this embodiment may be a synchronization operation on incremental data, for example, the data platform may first search for data updated compared to the last synchronization operation, and then synchronize the updated data to the development storage area; alternatively, the data platform may synchronize the entire amount of data, for example, synchronizing all data to the development storage area each time the data platform processes the data.
Of course, in actual practice, the incremental processing and the full processing may be combined, for example, the incremental processing may be performed every day, and the full processing may be performed every month. This is by way of example only and not by way of limitation.
In this embodiment, the production storage area further stores metadata corresponding to the original service data. When the data platform executes synchronous operation, determining metadata corresponding to desensitization data based on the corresponding relation between original service data and the metadata, and then storing the metadata and the corresponding desensitization data in the development storage area in an associated manner.
Taking tables 1 and 2 as examples, the metadata corresponding to the original service data shown in table 1 may include a table name, a column name, and the like, and according to the corresponding relationship, the metadata corresponding to the desensitization data shown in table 2 may be determined, for example, the table name is an attendance data table, and the column name includes a name, a contact address, an on-duty time, and an off-duty time.
The data platform synchronizes the metadata first, and after the synchronization of the metadata is completed, it is equivalent to establish a data table with the same attribute as the original service data in the development storage area, and then the data platform can store the metadata and the desensitization data into the development storage area in an associated manner according to the corresponding relationship between the metadata and the desensitization data, for example, as shown in table 2. It should be noted that the metadata shown in table 2 only includes column names, and as for other metadata such as table names and description information, corresponding correspondence may also be established, which is not described herein again. Based on the production storage area and the development storage area, the data platform can execute the instruction issued by the user. In this embodiment, when receiving a user instruction, the data platform first determines a user role of the user, for example, a production tenant or a development tenant, and then determines a storage area to which the user has an authority to access. For example, if the user is a production tenant, it is determined that the user has the right to access the production storage area, and if the user is a development tenant, it is determined that the user has the right to access the development storage area. Therefore, the development tenant can perform operations such as data query and data modification on desensitized data in the development storage area through the data platform, and perform software logic development.
At this point, the flow shown in fig. 3 is completed. According to the process, after desensitization processing is carried out on original service data in a production storage area by a data platform, the obtained desensitization data are synchronized to a development storage area, and the original service data and the desensitization data are isolated from each other. The development tenant can perform software logic development based on desensitization data in the development storage area without contacting original service data, and the security of user sensitive data is guaranteed. Meanwhile, when the metadata is synchronized, the metadata is not desensitized and is synchronized to the development storage area in association with the desensitized data, so that the development tenant can inquire the original metadata and establish a table structure based on the metadata so as to develop software logic.
In this embodiment, the development tenant may also verify the developed software logic based on desensitization data. The software logic may be processing of data in the data platform, such as SQL processing, spark processing, or other algorithm processing.
In one example, in response to a verification instruction of a development tenant, the data platform verifies the developed software logic with desensitized data in the development storage area. After receiving the software logic issued by the development tenant, the data platform determines that the development tenant has the authority of accessing the development storage area, and then executes the software logic based on desensitization data in the development storage area. If the execution is normal, the software logic is considered to pass the verification; if the execution is in error, the software logic is considered to be unverified.
The method of verifying software logic is described below by way of a specific embodiment.
Still taking the above-mentioned attendance management platform as an example, for example, the software logic is to merge data in different data tables. For example, the attendance management platform stores a staff position and position table in addition to the data tables shown in tables 1 and 2. For this post job level table, the raw business data stored in the production storage area is shown in table 3 and the desensitization data stored in the development storage area is shown in table 4.
| Name (I) | Position and position |
| Zhang three | Primary main pipe |
| Li four | Three-level staff member |
In table 3, the position of Zhang III is the first level supervisor, and the position of Li IV is the third level clerk.
| Name (I) | Position and position |
| Zhang | **** |
| Plum | **** |
TABLE 4
Wherein, Zhang and Li are desensitization data corresponding to name and position respectively.
For example, the software logic is to combine attendance data of the employee and the post position of the employee into one table, and if the execution is normal, the software logic is executed based on the instruction of the development tenant, and the execution result shown in table 5 can be obtained.
| Name (I) | Contact means | Working hours | Off duty time | Position and position |
| Zhang | 135****0001 | 9:00 | 17:00 | **** |
| Plum | 135****0002 | 10:00 | 17:00 | **** |
TABLE 5
According to the embodiment, when the development tenant verifies the software logic, the data processing is also carried out based on desensitization data, the original business data cannot be contacted, and the security of the sensitive data is protected.
In this embodiment, after the development tenant completes development and verification of the software logic, the development tenant may synchronize the software logic to the production tenant, so that the production tenant may use the software logic to process the service. For example, software logic may be synchronized directly to a production storage area; or the data platform may further include a shared storage area, both the production tenant and the development tenant may access the shared storage area, the development tenant may issue the software logic to the shared storage area, and the production tenant may acquire the software logic from the shared storage area.
In this embodiment, after obtaining the verified software logic, the production tenant may execute the software logic based on the original service data. In one example, in response to a processing instruction of a production tenant, the data platform executes validated software logic based on raw business data in the production storage area. For example, the data platform executes software logic specified by the production tenant based on the raw business data and feeds back the execution results to the production tenant.
Still take the software logic of combining the attendance data of the employee with the post level of the employee in the attendance management platform as an example. Based on the raw service data shown in tables 1 and 3, the execution results shown in table 6 can be obtained.
| Name (I) | Contact means | Working hours | Off duty time | Position and position |
| Zhang three | 13500010001 | 9:00 | 17:00 | Primary main pipe |
| Li four | 13500010002 | 10:00 | 17:00 | Three-level staff member |
TABLE 6
According to the embodiment, when the production tenant processes the service by using the software logic, the production tenant processes data based on the original service data, and the service execution is ensured not to have deviation.
In one example, the production tenant may also configure an execution cycle of the software logic, and the data platform may periodically execute the software logic to process the raw business data in the production storage area according to the time of the production tenant device.
It can be understood that the data platform and the software logic shown in the above embodiments are only exemplary illustrations, and in practical applications, a development tenant may develop corresponding software logic according to the requirements of a production tenant, and details are not described here.
According to the embodiment, the development tenant can verify the software logic based on desensitization data in the development storage area, and the original service data cannot be contacted while verification is completed, so that the safety of user sensitive data is guaranteed. The production tenant can execute the software logic based on the original service data in the production storage area to process the service corresponding to the software logic, and the service processing is executed based on the original service data, so that the accuracy of service execution is ensured, and deviation is avoided.
Corresponding to the foregoing embodiment of the data access method, the present specification further provides a structure diagram of a data platform, as shown in fig. 4, where the data platform includes a first module corresponding to a production tenant and a first module corresponding to a development tenant.
Based on the first module, the production tenant can process the production storage area. The first module includes:
and the metadata management unit is used for synchronously creating metadata of the data table according to the external data source.
The system comprises a service data management unit, a production storage area and a data processing unit, wherein the service data management unit is used for acquiring original service data according to an external data source and storing the original service data into the production storage area; and the data processing module is also used for carrying out desensitization processing on the original service data according to a preset desensitization rule to obtain desensitization data.
In this embodiment, the external data source is a source of original service data in the data platform, and for example, the metadata and the service data may be synchronized by importing a data table.
In this embodiment, the desensitization processing method may refer to the embodiment shown in fig. 1, and is not described here again.
In this embodiment, in response to the desensitization instruction of the production tenant, the metadata obtained by the metadata management unit and the desensitization data obtained by the service data management unit may also be synchronized to the development storage area.
And the service node management unit is used for executing the service node of the software logic.
And the business process management unit is used for executing the whole process of the software logic and processing the real data in the production storage area.
Based on the second module, the development tenant can process the development storage area. The second module includes:
and the metadata management unit is used for viewing the metadata in the development storage area.
And the business data management unit is used for viewing desensitization data in the development storage area.
In this embodiment, the development tenant may view the metadata and desensitization data having the correspondence based on the metadata management unit and the business data management unit.
And the service node management unit is used for carrying out service processing on the desensitization data.
In this embodiment, the business processing may include performing SQL processing, spark processing, algorithm processing, and the like on the desensitized data.
The business process management unit is used for enabling the development tenant to serially connect the business nodes subjected to business processing; and also for verifying software logic based on desensitization data.
In this embodiment, in response to an operation instruction of the development tenant, the verified software logic may be further synchronized to the production storage area.
To this end, the apparatus shown in fig. 3 is completed. In the method, the production storage area and the development storage area are two storage areas isolated from each other in the data platform, and are not isolated in physical space. When the data in the production storage area is synchronized to the development storage area, the data transmission efficiency is high and the safety is high. Meanwhile, development tenants can perform software logic development based on desensitized data in the development storage area, so that data development work can be completed, and the security of sensitive data is guaranteed. In addition, in the development storage area, metadata corresponding to desensitization data is also stored, and the metadata is not desensitized, so that development tenants can establish a table structure based on the metadata, and the development tenants can not have deviation in understanding of the desensitization data, and the accuracy of software logic developed by the development tenants is ensured.
Corresponding to the embodiments of the data access method, the present specification also provides embodiments of an apparatus for data access.
Embodiments of the data access device of the present description may be applied to a data platform. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. Taking a software implementation as an example, as a device in a logical sense, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory for operation through a processor of the data platform where the device is located. From a hardware aspect, as shown in fig. 5, the data access apparatus in this specification is a hardware structure diagram of a data platform, where the data access apparatus is located, and except for the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 5, the data platform where the apparatus is located in the embodiment may also include other hardware according to an actual function of the data platform, which is not described again.
Fig. 6 is a block diagram of a data access device shown in an exemplary embodiment of the present description.
Referring to fig. 6, the data access apparatus 600 can be applied to the data platform shown in fig. 5, and the apparatus includes:
a storage area dividing unit 601 for dividing a production storage area and a development storage area from the storage areas of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
a user role determination unit 602, configured to determine a user role of a user accessing the data platform;
an authority determining unit 603, configured to determine that the user has an authority to access a production storage area when the user role of the user is a production tenant; and determining that the user has the authority to access the development storage area under the condition that the user role of the user is a development tenant.
Optionally, the apparatus further comprises:
a desensitization unit 604, configured to perform desensitization processing on original service data in the production storage area by using a preset desensitization rule, to obtain desensitization data;
a synchronizing unit 605, configured to synchronize the desensitization data to the development storage area, so that the development tenant performs software logic development based on the desensitization data stored in the development storage area.
Optionally, performing desensitization processing on original service data in the production storage area by using a preset desensitization rule, including:
and in response to a desensitization instruction of the production tenant, desensitizing the original service data in the production storage area by adopting a preset desensitization rule.
Optionally, the production storage area further stores metadata corresponding to original business data, and the synchronizing the desensitization data to the development storage area includes:
determining metadata corresponding to desensitized data based on the corresponding relation between the original service data and the metadata;
storing the metadata in association with corresponding desensitization data to the development storage area.
Optionally, the apparatus further comprises:
a verification unit 606 for verifying the developed software logic with desensitization data in the development storage area in response to a verification instruction of the development tenant.
Optionally, the apparatus further comprises:
and the execution unit 607 is used for responding to the processing instruction of the production tenant and executing the verified software logic based on the original business data in the production storage area.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement it without inventive effort.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.
Corresponding to the foregoing embodiments of the data access method, the present specification also provides an electronic device, including: a processor and a memory for storing machine executable instructions. Wherein the processor and the memory are typically interconnected by means of an internal bus. In other possible implementations, the device may also include an external interface to enable communication with other devices or components.
In this embodiment, the processor is caused to:
dividing a production storage area and a development storage area from a storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
determining a user role for a user accessing the data platform;
if the user role of the user is a production tenant, determining that the user has the authority of accessing a production storage area;
and if the user role of the user is a development tenant, determining that the user has the authority of accessing a development storage area.
Optionally, the processor is further caused to:
desensitizing original service data in a production storage area by adopting a preset desensitizing rule to obtain desensitizing data;
and synchronizing the desensitization data to a development storage area so that a development tenant performs software logic development based on the desensitization data stored in the development storage area.
Optionally, performing desensitization processing on original service data in the production storage area by using a preset desensitization rule, including: and in response to a desensitization instruction of the production tenant, desensitizing the original service data in the production storage area by adopting a preset desensitization rule.
Optionally, metadata corresponding to the original service data is further stored in the production storage area;
the synchronizing the desensitization data to a development storage area includes:
determining metadata corresponding to desensitized data based on the corresponding relation between the original service data and the metadata;
and storing the metadata and the corresponding desensitization data in the development storage area in an associated manner so that the development tenant can perform software logic development based on the desensitization data stored in the development storage area and the metadata corresponding to the desensitization data.
Optionally, the processor is further caused to: the developed software logic is validated with desensitization data in the development storage area in response to a validation instruction of a development tenant.
Optionally, the processor is further caused to: and executing the verified software logic based on the original business data in the production storage area in response to the processing instruction of the production tenant.
In correspondence with the aforementioned embodiments of the data access method, the present specification also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of:
dividing a production storage area and a development storage area from a storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
determining a user role for a user accessing the data platform;
if the user role of the user is a production tenant, determining that the user has the authority of accessing a production storage area;
and if the user role of the user is a development tenant, determining that the user has the authority of accessing a development storage area.
Optionally, the steps further comprise:
desensitizing original service data in a production storage area by adopting a preset desensitizing rule to obtain desensitizing data;
and synchronizing the desensitization data to a development storage area so that a development tenant performs software logic development based on the desensitization data stored in the development storage area.
Optionally, performing desensitization processing on original service data in the production storage area by using a preset desensitization rule, including: and in response to a desensitization instruction of the production tenant, desensitizing the original service data in the production storage area by adopting a preset desensitization rule.
Optionally, the production storage area further stores metadata corresponding to original business data, and the synchronizing the desensitization data to the development storage area includes:
determining metadata corresponding to desensitized data based on the corresponding relation between the original service data and the metadata;
and storing the metadata and the corresponding desensitization data in the development storage area in an associated manner so that the development tenant can perform software logic development based on the desensitization data stored in the development storage area and the metadata corresponding to the desensitization data.
Optionally, the steps further comprise: the developed software logic is validated with desensitization data in the development storage area in response to a validation instruction of a development tenant.
Optionally, the steps further comprise: and executing the verified software logic based on the original business data in the production storage area in response to the processing instruction of the production tenant.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (14)
1. A method for data access, the method being applied to a data platform, the method comprising:
dividing a production storage area and a development storage area from a storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
determining a user role for a user accessing the data platform;
if the user role of the user is a production tenant, determining that the user has the authority of accessing a production storage area;
and if the user role of the user is a development tenant, determining that the user has the authority of accessing a development storage area.
2. The method of claim 1, further comprising:
desensitizing original service data in a production storage area by adopting a preset desensitizing rule to obtain desensitizing data;
and synchronizing the desensitization data to a development storage area so that a development tenant performs software logic development based on the desensitization data stored in the development storage area.
3. The method of claim 2, wherein the desensitizing the original service data in the production storage area using the preset desensitizing rule comprises:
and in response to the desensitization instruction of the production tenant, desensitizing the original service data in the production storage area by adopting a preset desensitization rule.
4. The method of claim 2, wherein the production storage area further stores metadata corresponding to raw business data;
the synchronizing the desensitization data to a development storage area includes:
determining metadata corresponding to desensitized data based on the corresponding relation between the original service data and the metadata;
and storing the metadata and the corresponding desensitization data in the development storage area in an associated manner so that the development tenant can perform software logic development based on the desensitization data stored in the development storage area and the metadata corresponding to the desensitization data.
5. The method according to claim 2 or 4, characterized in that the method further comprises:
the developed software logic is validated with desensitization data in the development storage area in response to a validation instruction of a development tenant.
6. The method of claim 5, further comprising:
and executing the verified software logic based on the original business data in the production storage area in response to the processing instruction of the production tenant.
7. An apparatus for data access, the apparatus being applied to a data platform, the apparatus comprising:
a storage area dividing unit for dividing a production storage area and a development storage area from the storage area of the data platform; the production storage area stores original service data; the development storage area is used for storing desensitization data obtained after desensitization processing is carried out on the original service data;
the user role determining unit is used for determining the user role of the user accessing the data platform;
the permission determining unit is used for determining that the user has the permission to access the production storage area under the condition that the user role of the user is a production tenant; and determining that the user has the authority to access the development storage area under the condition that the user role of the user is a development tenant.
8. The apparatus of claim 7, further comprising:
the desensitization unit is used for performing desensitization treatment on original service data in the production storage area by adopting a preset desensitization rule to obtain desensitization data;
and the synchronization unit is used for synchronizing the desensitization data to the development storage area so that the development tenant can perform software logic development based on the desensitization data stored in the development storage area.
9. The apparatus of claim 8, wherein the desensitizing processing on the original service data in the production storage area by using the preset desensitizing rule comprises:
and in response to a desensitization instruction of the production tenant, desensitizing the original service data in the production storage area by adopting a preset desensitization rule.
10. The apparatus of claim 8, wherein the production storage area further stores metadata corresponding to raw business data, and wherein synchronizing the desensitization data to a development storage area comprises:
determining metadata corresponding to desensitized data based on the corresponding relation between the original service data and the metadata;
storing the metadata in association with corresponding desensitization data to the development storage area.
11. The apparatus of claim 8 or 10, further comprising:
and the verification unit is used for responding to a verification instruction of the development tenant and verifying the developed software logic by using desensitization data in the development storage area.
12. The apparatus of claim 11, further comprising:
and the execution unit is used for responding to the processing instruction of the production tenant and executing the verified software logic based on the original service data in the production storage area.
13. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor implements the method of any one of claims 1-6 by executing the executable instructions.
14. A computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011005438.0A CN112163214A (en) | 2020-09-22 | 2020-09-22 | Data access method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011005438.0A CN112163214A (en) | 2020-09-22 | 2020-09-22 | Data access method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112163214A true CN112163214A (en) | 2021-01-01 |
Family
ID=73864362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011005438.0A Pending CN112163214A (en) | 2020-09-22 | 2020-09-22 | Data access method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112163214A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112765658A (en) * | 2021-01-15 | 2021-05-07 | 杭州数梦工场科技有限公司 | Data desensitization method and device, electronic equipment and storage medium |
| CN114048511A (en) * | 2022-01-12 | 2022-02-15 | 树根互联股份有限公司 | Cross-data-center data desensitization method and device and electronic equipment |
| CN115080950A (en) * | 2022-07-20 | 2022-09-20 | 深圳竹云科技股份有限公司 | Identity information processing method and device, computer equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307185A (en) * | 2011-06-27 | 2012-01-04 | 北京大学 | Data isolation method used in storage cloud |
| CN102708316A (en) * | 2012-04-19 | 2012-10-03 | 北京华胜天成科技股份有限公司 | Method for isolating data in multi-tenant architecture |
| CN104767745A (en) * | 2015-03-26 | 2015-07-08 | 浪潮集团有限公司 | Cloud data security protection method |
| CN106446704A (en) * | 2015-08-10 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Method and device for data processing |
| CN107766741A (en) * | 2017-10-23 | 2018-03-06 | 中恒华瑞（北京）信息技术有限公司 | Data desensitization system and method |
| CN108268558A (en) * | 2017-01-03 | 2018-07-10 | 中移(苏州)软件技术有限公司 | A kind of method and apparatus of data analysis |
| CN108460287A (en) * | 2018-03-21 | 2018-08-28 | 南通大学 | The division methods in user's control region and memory protect system in memory protection location |
| CN109558746A (en) * | 2018-11-06 | 2019-04-02 | 泰康保险集团股份有限公司 | Data desensitization method, device, electronic equipment and storage medium |
| CN110443048A (en) * | 2019-07-04 | 2019-11-12 | 广州海颐信息安全技术有限公司 | Data center looks into number system |
-
2020
- 2020-09-22 CN CN202011005438.0A patent/CN112163214A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307185A (en) * | 2011-06-27 | 2012-01-04 | 北京大学 | Data isolation method used in storage cloud |
| CN102708316A (en) * | 2012-04-19 | 2012-10-03 | 北京华胜天成科技股份有限公司 | Method for isolating data in multi-tenant architecture |
| CN104767745A (en) * | 2015-03-26 | 2015-07-08 | 浪潮集团有限公司 | Cloud data security protection method |
| CN106446704A (en) * | 2015-08-10 | 2017-02-22 | 阿里巴巴集团控股有限公司 | Method and device for data processing |
| CN108268558A (en) * | 2017-01-03 | 2018-07-10 | 中移(苏州)软件技术有限公司 | A kind of method and apparatus of data analysis |
| CN107766741A (en) * | 2017-10-23 | 2018-03-06 | 中恒华瑞（北京）信息技术有限公司 | Data desensitization system and method |
| CN108460287A (en) * | 2018-03-21 | 2018-08-28 | 南通大学 | The division methods in user's control region and memory protect system in memory protection location |
| CN109558746A (en) * | 2018-11-06 | 2019-04-02 | 泰康保险集团股份有限公司 | Data desensitization method, device, electronic equipment and storage medium |
| CN110443048A (en) * | 2019-07-04 | 2019-11-12 | 广州海颐信息安全技术有限公司 | Data center looks into number system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112765658A (en) * | 2021-01-15 | 2021-05-07 | 杭州数梦工场科技有限公司 | Data desensitization method and device, electronic equipment and storage medium |
| CN114048511A (en) * | 2022-01-12 | 2022-02-15 | 树根互联股份有限公司 | Cross-data-center data desensitization method and device and electronic equipment |
| CN115080950A (en) * | 2022-07-20 | 2022-09-20 | 深圳竹云科技股份有限公司 | Identity information processing method and device, computer equipment and storage medium |
| CN115080950B (en) * | 2022-07-20 | 2022-11-15 | 深圳竹云科技股份有限公司 | Identity information processing method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10489608B2 (en) | Automatic masking of sensitive data | |
| US9652512B2 (en) | Secure matching supporting fuzzy data | |
| CN112163214A (en) | Data access method and device | |
| US9514169B2 (en) | Columnar table data protection | |
| US9268947B1 (en) | Method and system for managing information associated with sensitive information in an enterprise | |
| CN104657665B (en) | A kind of document handling method | |
| US11354448B2 (en) | Demand trusted device-based data acquisition methods, apparatuses, and devices | |
| US11003653B2 (en) | Method and system for secure digital documentation of subjects using hash chains | |
| CN112333196A (en) | Attack event tracing method and device, electronic equipment and storage medium | |
| EP3095084A1 (en) | System and methods for location based management of cloud platform data | |
| CN111914029A (en) | Block chain-based medical data calling method and device, electronic equipment and medium | |
| WO2020160072A1 (en) | Auditing system using a trusted and cryptographically secure database | |
| CN115329177A (en) | Data processing method, device, storage medium and program product | |
| WO2021118413A2 (en) | Data processing method, comprising secure multilateral computing and data analysis methods | |
| CN112417406A (en) | Data desensitization method and device, readable storage medium and electronic equipment | |
| CN109729076B (en) | Data desensitization and inverse desensitization method and device, storage medium and terminal | |
| WO2022233236A1 (en) | Secure data analytics | |
| CN115878655A (en) | Data operation method and device, computer equipment and storage medium | |
| CN115185946A (en) | Multi-tenant system, multi-tenant management method, computer device, and storage medium | |
| CN114238273A (en) | Database management method, device, equipment and storage medium | |
| US20200151753A1 (en) | Virtual id and methods of use thereof | |
| US20200117822A1 (en) | Dynamic authorization matrix and related systems, methods, and computer program products | |
| CN112256688B (en) | Service data cleaning method and device and electronic equipment | |
| JP6130888B2 (en) | Personal information protection sales support system | |
| Javed et al. | Blockchain-Based Logging to Defeat Malicious Insiders: The Case of Remote Health Monitoring Systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210101 |
|
| RJ01 | Rejection of invention patent application after publication |