CN111782648B - Structured data processing method, device, computer equipment and storage medium - Google Patents
Structured data processing method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111782648B CN111782648B CN202010583553.XA CN202010583553A CN111782648B CN 111782648 B CN111782648 B CN 111782648B CN 202010583553 A CN202010583553 A CN 202010583553A CN 111782648 B CN111782648 B CN 111782648B
- Authority
- CN
- China
- Prior art keywords
- data
- user
- structured data
- index
- fingerprint
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/151—Transformation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Bioethics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data processing, and provides a structured data processing method, a device, computer equipment and a storage medium, wherein the structured data processing method comprises the following steps: receiving structured data uploaded by a user through an uploading interface; an index algorithm is called to map the structured data to obtain a mapping value, and a total index of the structured data is obtained according to the mapping value; acquiring a user ID of the user and carrying out serialization processing on the user ID to obtain a serialized user ID; determining the data type of the structured data, and calling a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file; the total index of the structured data and the fingerprint file are stored and managed. The invention can effectively manage, track and trace the structured data by generating the total index and the fingerprint file for the structured data, thereby realizing the safe transmission of the structured data.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a structured data processing method, a structured data processing device, a computer device, and a storage medium.
Background
The loss caused by data leakage is trillion worldwide, and for unstructured types of data such as video, pictures and the like, the data leakage source can be positioned by adding watermarks, and responsibility tracking is performed by tracing the leakage source.
However, for structured data such as data in a database, for example, data in a form type, although the security of the data can be ensured by an encryption mode, the data cannot be quickly positioned to a data leakage source under the condition of not damaging the complete expression of original data information, and thus, a responsible party cannot be determined.
Disclosure of Invention
In view of the foregoing, it is necessary to provide a structured data processing method, apparatus, computer device, and storage medium, which can implement security management of structured data and quickly locate a data source.
A first aspect of the present invention provides a structured data processing method, the method comprising:
receiving structured data uploaded by a user through an uploading interface;
an index algorithm is called to map the structured data to obtain a mapping value, and a total index of the structured data is obtained according to the mapping value;
acquiring a user ID of the user and carrying out serialization processing on the user ID to obtain a serialized user ID;
Determining the data type of the structured data, and calling a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file;
the total index of the structured data and the fingerprint file are stored and managed.
In an alternative embodiment, before the receiving the structured data uploaded by the user through the uploading interface, the method further comprises:
receiving a login request of the user and verifying the login request;
and responding to the login request verification, and determining that the user account carried in the login request is the user ID of the user.
In an alternative embodiment, the invoking the indexing algorithm maps the structured data to obtain a mapped value, and obtaining the total index of the structured data according to the mapped value includes:
traversing a plurality of row labels of the structured data;
selecting an index algorithm and mapping the traversed row labels through the selected index algorithm to obtain a plurality of mapping values;
converting each mapping value into a binary value, and determining the binary value as an index of data corresponding to a row label;
And splicing all indexes according to row marks to obtain a total index.
In an alternative embodiment, the determining the data type of the structured data, and invoking a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file includes:
traversing a plurality of column data of the structured data;
identifying the data type of each column of data through an intelligent semantic identification algorithm;
matching a fingerprint algorithm corresponding to the data type of each column of data;
and calling the fingerprint algorithm to insert the serialized user ID into the data of the corresponding data type to obtain a fingerprint file.
In an optional embodiment, the invoking the fingerprint algorithm to insert the serialized user ID into data of a corresponding data type, and obtaining the fingerprint file includes:
calculating the total number of data of each data type;
calculating the number of the serialized user IDs to be inserted in each data type according to a preset proportion and the total number;
randomly selecting target data corresponding to the number from each data type according to the number of the serialization user IDs to be inserted;
And calling the fingerprint algorithm to insert the serialized user ID into the target data according to the index sequence of the target data to obtain a fingerprint file.
In an optional embodiment, the invoking the fingerprint algorithm to insert the serialized user ID into data of a corresponding data type, and obtaining the fingerprint file includes:
receiving external parameters input by the user, wherein the external parameters comprise coverage areas of data to be encrypted and line labels to be encrypted;
determining an index corresponding to the line mark number to be encrypted;
and calling the fingerprint algorithm to insert the serialized user ID into the data corresponding to the index according to the coverage area to obtain a fingerprint file.
In an alternative embodiment, the method further comprises:
acquiring a plurality of first information of the structured data and a plurality of second information of the fingerprint file;
and calling a metadata management tool to store the first information and the second information in an associated mode.
A second aspect of the invention provides a structured data processing apparatus, said apparatus comprising:
the receiving module is used for receiving the structured data uploaded by the user through the uploading interface;
The mapping module is used for calling an index algorithm to map the structured data to obtain a mapping value, and obtaining the total index of the structured data according to the mapping value;
the processing module is used for acquiring the user ID of the user and carrying out serialization processing on the user ID to obtain a serialized user ID;
the generation module is used for determining the data type of the structured data, calling a fingerprint algorithm corresponding to the data type, and embedding the serialized user ID into the structured data to generate a fingerprint file;
and the storage module is used for storing and managing the total index of the structured data and the fingerprint file.
A third aspect of the invention provides a computer device comprising a processor for implementing the structured data processing method when executing a computer program stored in a memory.
A fourth aspect of the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the structured data processing method.
In summary, the method, the device, the computer equipment and the storage medium for processing the structured data of the invention embed the user ID into the structured data through the fingerprint algorithm, namely, the user marks the structured data in a way of embedding watermark-like information through the user authorization credential; and generating a total index for the structured data through an index algorithm, and storing the total index and the generated fingerprint file in an associated manner, so that effective management, tracking and tracing of the structured data are realized, and safe transmission of the structured data is realized.
Drawings
Fig. 1 is a flowchart of a structured data processing method according to an embodiment of the present invention.
Fig. 2 is a block diagram of a structured data processing apparatus according to a second embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that, without conflict, the embodiments of the present invention and features in the embodiments may be combined with each other.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
Fig. 1 is a flowchart of a structured data processing method according to an embodiment of the present invention. The structured data processing method specifically comprises the following steps, the order of the steps in the flowchart may be changed according to different requirements, and some may be omitted.
S1, receiving structured data uploaded by a user through an uploading interface.
The computer equipment is internally provided with a structured data fingerprint management platform in advance, and the structured data fingerprint management platform comprises: an application layer, a service layer and a management layer. The application layer is an exposed interface layer, and the service layer is arranged between the application layer and the management layer.
The application layer is provided with an uploading interface, and the uploading interface can be a structured data uploading interface for uploading structured data by a user.
S2, an index algorithm is called to map the structured data to obtain a mapping value, and a total index of the structured data is obtained according to the mapping value.
The service layer is at least provided with an index generator and a fingerprint file generator.
The management layer is at least provided with an index algorithm configuration module and a fingerprint algorithm configuration module, wherein the index algorithm configuration module is used for pre-configuring at least one index algorithm, and the fingerprint algorithm configuration module is used for pre-configuring a plurality of fingerprint algorithms.
When the computer equipment monitors that the structured data fingerprint management platform receives structured data uploaded by a user, at least one index algorithm pre-configured in an index algorithm configuration module is called by the index generator, and a total index is generated for the structured data. When a plurality of index algorithms are preconfigured in the index algorithm configuration module, the computer equipment randomly selects one index algorithm from the plurality of index algorithms to generate a total index for the structured data.
In an optional embodiment, the invoking the indexing algorithm maps the structured data to obtain a mapped value, and obtaining the total index of the structured data according to the mapped value specifically includes:
s21, traversing a plurality of row labels of the structured data;
s22, selecting an index algorithm, and mapping the traversed row labels through the selected index algorithm to obtain a plurality of mapping values;
s23, converting each mapping value into a binary value, and determining the binary value as an index of data corresponding to the row label;
and S24, splicing all indexes according to row labels to obtain a total index.
Illustratively, traverse to three row labels: 1,2,3, mapping row index 1 to binary 00000001, mapping row index 2 to binary 00000010, and mapping row index 3 to binary 00000011, then binary 00000001 is the index of row 1 data, binary 00000010 is the index of row 2 data, binary 00000011 is the index of row 3 data, and concatenating these three indices to get the total index 000000010000001000000011.
In the alternative embodiment, the line mark can be regarded as index information, but the index information is obvious, and a unique mapping value can be obtained after the line mark is mapped by an index algorithm, so that the explicit index of the line mark is mapped and converted into the invisible index of the total index, and the safety of the index of the structured data is improved.
S3, obtaining the user ID of the user and carrying out serialization processing on the user ID to obtain the serialized user ID.
The user ID represents the user's authorization credentials.
In an alternative embodiment, the serializing the user ID includes: and converting each character in the user ID into a corresponding ASCII code value, wherein the ASCII code value is the serialized user ID.
The computer equipment performs serialization processing on the user ID, so that the user information can be hidden to a certain extent, and the privacy security of the user is ensured.
S4, determining the data type of the structured data, and calling a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file.
The computer equipment calls a plurality of fingerprint algorithms which are pre-configured in a fingerprint algorithm configuration module through the fingerprint file generator, and selects a fingerprint algorithm corresponding to the data type from the plurality of fingerprint algorithms to generate a fingerprint file for the structured data.
In specific implementation, the fingerprint file generator determines the data type of the structured data, then matches the fingerprint algorithm corresponding to the data type, and embeds the user ID into the structured data by adopting the matched fingerprint algorithm to generate the fingerprint file.
In an optional embodiment, the determining the data type of the structured data, and invoking a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file specifically includes:
s41, traversing a plurality of column data of the structured data;
s42, recognizing the data type of each column of data through an intelligent semantic recognition algorithm;
s43, matching a fingerprint algorithm corresponding to the data type of each column of data;
s44, invoking the fingerprint algorithm to insert the serialized user ID into the data of the corresponding data type to obtain a fingerprint file.
Illustratively, the structured data includes 3 columns of data, wherein the data type of the 1 st column of data is a floating point type, the data type of the 2 nd column of data is an integer type, and the data type of the 3 rd column of data is a string type. The computer equipment generates a fingerprint file for the 1 st data by matching a fingerprint algorithm corresponding to a floating point type from a plurality of fingerprint algorithms, generates a fingerprint file for the 2 nd data by matching a fingerprint algorithm corresponding to an integer type, and generates a fingerprint file for the 3 rd data by matching a fingerprint algorithm corresponding to a character string type.
In the alternative embodiment, because the data types of the structured data are different, the effect of encrypting the data of each data type by using the same fingerprint algorithm is not very good, and the effect of encrypting the data by calling the fingerprint algorithm matched with the data type is better by determining the data type, so that the integral structure of the data can be maintained to the greatest extent, and the complete expression of the original data information is not destroyed.
In an optional embodiment, the invoking the fingerprint algorithm inserts the serialized user ID into the data of the corresponding data type, and obtaining the fingerprint file specifically includes:
s441, calculating the total number of data of each data type;
s442, calculating the number of the serialized user IDs to be inserted in each data type according to the preset proportion and the total number;
s443, randomly selecting target data corresponding to the number from each data type according to the number of the serialized user IDs to be inserted;
s444, invoking the fingerprint algorithm to insert the serialized user ID into the target data according to the index sequence of the target data to obtain a fingerprint file.
In this alternative embodiment, the preset ratio is set lower.
Because the preset proportion is lower, the quantity of the target data of each data type is smaller, namely the overall data modification proportion is smaller, and the effect that a user does not feel can be achieved by comparing the data before and after the serial user ID is inserted.
In an optional embodiment, the invoking the fingerprint algorithm inserts the serialized user ID into the data of the corresponding data type, and obtaining the fingerprint file specifically includes:
s441, receiving external parameters input by a user, wherein the external parameters comprise coverage areas of data to be encrypted and line labels to be encrypted;
s442, determining an index corresponding to the line mark number to be encrypted;
s443, invoking the fingerprint algorithm to insert the serialized user ID into the data corresponding to the index according to the coverage area to obtain a fingerprint file.
In this alternative embodiment, the user selects which data to encrypt, how much to encrypt, i.e. the user participates in the encryption process, so that the autonomy is strong, and it is more clear for the user to know which data is important, so that the directionality of inserting the serialized user ID is stronger.
S5, storing and managing the total index of the structured data and the fingerprint file.
And a metadata management module is also configured in a management layer in the structured data fingerprint management platform, and the metadata management module is a metadata management tool implanted in the structured data fingerprint management platform.
The computer equipment stores the total index of the structured data and the fingerprint file in a metadata management module in a correlated way, and provides inquiry, retrieval, authority management and the like for the total index of the structured data and the fingerprint file through one-stop management of the metadata management module.
In the embodiment, an intelligent structured data fingerprint management platform is installed in the computer equipment, a fingerprint algorithm is called to embed a user ID into the structured data, namely, the user mark is carried out on the structured data in a mode of embedding watermark-like information through a user authorization credential; and generating a total index for the structured data through an index algorithm, and storing the total index and the generated fingerprint file in an associated manner, so that effective management, tracking and tracing of the structured data are realized, and safe transmission of the structured data is realized.
In an alternative embodiment, prior to S1, the method further comprises:
receiving a login request of a user, verifying the login request, and determining a user account carried in the login request as a user ID of the user in response to the verification of the login request.
In this optional embodiment, when the user logs in the structured data fingerprint management platform for the first time, the registration signing process needs to be completed, and after the user successfully registers and signs up, the structured data fingerprint management platform records the user account and the user password of the user, and when the subsequent user requests to log in the structured data fingerprint management platform again, verification is performed according to the recorded user account and user password and the user account and password carried in the login request of the user. And when the recorded user account is consistent with the user account carried in the login request of the user, and the recorded user password is consistent with the user password carried in the login request of the user, determining that the login request passes verification. And when the recorded user account is inconsistent with the user account carried in the login request of the user and/or the recorded user password is inconsistent with the user password carried in the login request of the user, determining that the login request is not verified.
And whether the user is a subscriber or not is determined by verifying the user account number and the user password of the user, and an uploading interface is provided for the user to upload the structured data only if the user is determined to be the subscriber, so that the safety and the reliability of the source of the structured data are ensured from the source.
In an alternative embodiment, the method further comprises:
acquiring a plurality of first information of the structured data and a plurality of second information of the fingerprint file;
and calling a metadata management tool to store the first information and the second information in an associated mode.
The plurality of first information includes: a lot number, the user ID, a time at which the structured data was received, a storage location of the structured data, a total index of the structured data.
And after the fingerprint file is sent to a receiver, acquiring a plurality of second information. The plurality of second information includes: a lot number, the user ID, a recipient ID, a time to receive and a time to transmit the structured data, a storage location of the structured data and a location to transmit the structured data, an index of the data inserted into the serialized user ID.
In this embodiment, by storing the related information of the uploaded structured data and storing the related information of the transmitted fingerprint file, when a leakage accident occurs in the fingerprint file, the leakage source can be determined through the related information, and the responsible party can be locked.
Fig. 2 is a block diagram of a structured data processing apparatus according to a second embodiment of the present invention.
In some embodiments, the structured data processing apparatus 20 may comprise a plurality of functional modules comprising computer program segments. The computer program of the individual program segments in the structured data processing arrangement 20 can be stored in a memory of a computer device and executed by the at least one processor for performing (see fig. 1 for details) the functions of structured data processing.
In this embodiment, the structured data processing apparatus 20 may be divided into a plurality of functional modules according to the functions performed by the structured data processing apparatus. The functional module may include: the device comprises a receiving module 201, a mapping module 202, a processing module 203, a generating module 204, a storage module 205, a verification module 206, an obtaining module 207 and an associating module 208. The module referred to in the present invention refers to a series of computer program segments capable of being executed by at least one processor and of performing a fixed function, stored in a memory. In the present embodiment, the functions of the respective modules will be described in detail in the following embodiments.
The receiving module 201 is configured to receive structured data uploaded by a user through an uploading interface.
The computer equipment is internally provided with a structured data fingerprint management platform in advance, and the structured data fingerprint management platform comprises: an application layer, a service layer and a management layer. The application layer is an exposed interface layer, and the service layer is arranged between the application layer and the management layer.
The application layer is provided with an uploading interface, and the uploading interface can be a structured data uploading interface for uploading structured data by a user.
The mapping module 202 is configured to invoke an indexing algorithm to map the structured data to obtain a mapping value, and obtain a total index of the structured data according to the mapping value.
The service layer is at least provided with an index generator and a fingerprint file generator.
The management layer is at least provided with an index algorithm configuration module and a fingerprint algorithm configuration module, wherein the index algorithm configuration module is used for pre-configuring at least one index algorithm, and the fingerprint algorithm configuration module is used for pre-configuring a plurality of fingerprint algorithms.
When the computer equipment monitors that the structured data fingerprint management platform receives structured data uploaded by a user, at least one index algorithm pre-configured in an index algorithm configuration module is called by the index generator, and a total index is generated for the structured data. When a plurality of index algorithms are preconfigured in the index algorithm configuration module, the computer equipment randomly selects one index algorithm from the plurality of index algorithms to generate a total index for the structured data.
In an alternative embodiment, the mapping module 202 invokes an indexing algorithm to map the structured data to obtain a mapped value, and obtaining the total index of the structured data according to the mapped value specifically includes:
traversing a plurality of row labels of the structured data;
selecting an index algorithm and mapping the traversed row labels through the selected index algorithm to obtain a plurality of mapping values;
converting each mapping value into a binary value, and determining the binary value as an index of data corresponding to a row label;
and splicing all indexes according to row marks to obtain a total index.
Illustratively, traverse to three row labels: 1,2,3, mapping row index 1 to binary 00000001, mapping row index 2 to binary 00000010, and mapping row index 3 to binary 00000011, then binary 00000001 is the index of row 1 data, binary 00000010 is the index of row 2 data, binary 00000011 is the index of row 3 data, and concatenating these three indices to get the total index 000000010000001000000011.
In the alternative embodiment, the line mark can be regarded as index information, but the index information is obvious, and a unique mapping value can be obtained after the line mark is mapped by an index algorithm, so that the explicit index of the line mark is mapped and converted into the invisible index of the total index, and the safety of the index of the structured data is improved.
The processing module 203 is configured to obtain a user ID of the user and perform serialization processing on the user ID to obtain a serialized user ID.
The user ID represents the user's authorization credentials.
In an alternative embodiment, the processing module 203 performs serializing processing on the user ID includes: and converting each character in the user ID into a corresponding ASCII code value, wherein the ASCII code value is the serialized user ID.
The computer equipment performs serialization processing on the user ID, so that the user information can be hidden to a certain extent, and the privacy security of the user is ensured.
The generating module 204 is configured to determine a data type of the structured data, and invoke a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file.
The computer equipment calls a plurality of fingerprint algorithms which are pre-configured in a fingerprint algorithm configuration module through the fingerprint file generator, and selects a fingerprint algorithm corresponding to the data type from the plurality of fingerprint algorithms to generate a fingerprint file for the structured data.
In specific implementation, the fingerprint file generator determines the data type of the structured data, then matches the fingerprint algorithm corresponding to the data type, and embeds the user ID into the structured data by adopting the matched fingerprint algorithm to generate the fingerprint file.
In an alternative embodiment, the generating module 204 determines a data type of the structured data, and invoking a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file specifically includes:
traversing a plurality of column data of the structured data;
identifying the data type of each column of data through an intelligent semantic identification algorithm;
matching a fingerprint algorithm corresponding to the data type of each column of data;
and calling the fingerprint algorithm to insert the serialized user ID into the data of the corresponding data type to obtain a fingerprint file.
Illustratively, the structured data includes 3 columns of data, wherein the data type of the 1 st column of data is a floating point type, the data type of the 2 nd column of data is an integer type, and the data type of the 3 rd column of data is a string type. The computer equipment generates a fingerprint file for the 1 st data by matching a fingerprint algorithm corresponding to a floating point type from a plurality of fingerprint algorithms, generates a fingerprint file for the 2 nd data by matching a fingerprint algorithm corresponding to an integer type, and generates a fingerprint file for the 3 rd data by matching a fingerprint algorithm corresponding to a character string type.
In the alternative embodiment, because the data types of the structured data are different, the effect of encrypting the data of each data type by using the same fingerprint algorithm is not very good, and the effect of encrypting the data by calling the fingerprint algorithm matched with the data type is better by determining the data type, so that the integral structure of the data can be maintained to the greatest extent, and the complete expression of the original data information is not destroyed.
In an optional embodiment, the invoking the fingerprint algorithm inserts the serialized user ID into the data of the corresponding data type, and obtaining the fingerprint file specifically includes:
calculating the total number of data of each data type;
calculating the number of the serialized user IDs to be inserted in each data type according to a preset proportion and the total number;
randomly selecting target data corresponding to the number from each data type according to the number of the serialization user IDs to be inserted;
and calling the fingerprint algorithm to insert the serialized user ID into the target data according to the index sequence of the target data to obtain a fingerprint file.
In this alternative embodiment, the preset ratio is set lower.
Because the preset proportion is lower, the quantity of the target data of each data type is smaller, namely the overall data modification proportion is smaller, and the effect that a user does not feel can be achieved by comparing the data before and after the serial user ID is inserted.
In an optional embodiment, the invoking the fingerprint algorithm inserts the serialized user ID into the data of the corresponding data type, and obtaining the fingerprint file specifically includes:
receiving external parameters input by a user, wherein the external parameters comprise coverage areas of data to be encrypted and line labels to be encrypted;
determining an index corresponding to the line mark number to be encrypted;
and calling the fingerprint algorithm to insert the serialized user ID into the data corresponding to the index according to the coverage area to obtain a fingerprint file.
In this alternative embodiment, the user selects which data to encrypt, how much to encrypt, i.e. the user participates in the encryption process, so that the autonomy is strong, and it is more clear for the user to know which data is important, so that the directionality of inserting the serialized user ID is stronger.
The storage module 205 is configured to store and manage the total index of the structured data and the fingerprint file.
And a metadata management module is also configured in a management layer in the structured data fingerprint management platform, and the metadata management module is a metadata management tool implanted in the structured data fingerprint management platform.
The computer equipment stores the total index of the structured data and the fingerprint file in a metadata management module in a correlated way, and provides inquiry, retrieval, authority management and the like for the total index of the structured data and the fingerprint file through one-stop management of the metadata management module.
In the embodiment, an intelligent structured data fingerprint management platform is installed in the computer equipment, a fingerprint algorithm is called to embed a user ID into the structured data, namely, the user mark is carried out on the structured data in a mode of embedding watermark-like information through a user authorization credential; and generating a total index for the structured data through an index algorithm, and storing the total index and the generated fingerprint file in an associated manner, so that effective management, tracking and tracing of the structured data are realized, and safe transmission of the structured data is realized.
The verification module 206 is configured to receive a login request of a user, verify the login request, and determine that a user account carried in the login request is a user ID of the user in response to the login request passing the verification.
In this optional embodiment, when the user logs in the structured data fingerprint management platform for the first time, the registration signing process needs to be completed, and after the user successfully registers and signs up, the structured data fingerprint management platform records the user account and the user password of the user, and when the subsequent user requests to log in the structured data fingerprint management platform again, verification is performed according to the recorded user account and user password and the user account and password carried in the login request of the user. And when the recorded user account is consistent with the user account carried in the login request of the user, and the recorded user password is consistent with the user password carried in the login request of the user, determining that the login request passes verification. And when the recorded user account is inconsistent with the user account carried in the login request of the user and/or the recorded user password is inconsistent with the user password carried in the login request of the user, determining that the login request is not verified.
And whether the user is a subscriber or not is determined by verifying the user account number and the user password of the user, and an uploading interface is provided for the user to upload the structured data only if the user is determined to be the subscriber, so that the safety and the reliability of the source of the structured data are ensured from the source.
The obtaining module 207 is configured to obtain a plurality of first information of the structured data and obtain a plurality of second information of the fingerprint file.
The associating module 208 is configured to invoke a metadata management tool to store the plurality of first information in association with the plurality of second information in association with the plurality of first information.
The plurality of first information includes: a lot number, the user ID, a time at which the structured data was received, a storage location of the structured data, a total index of the structured data.
And after the fingerprint file is sent to a receiver, acquiring a plurality of second information. The plurality of second information includes: a lot number, the user ID, a recipient ID, a time to receive and a time to transmit the structured data, a storage location of the structured data and a location to transmit the structured data, an index of the data inserted into the serialized user ID.
In this embodiment, by storing the related information of the uploaded structured data and storing the related information of the transmitted fingerprint file, when a leakage accident occurs in the fingerprint file, the leakage source can be determined through the related information, and the responsible party can be locked.
Fig. 3 is a schematic structural diagram of a computer device according to a third embodiment of the present invention. In the preferred embodiment of the present invention, the computer device 30 includes a memory 31, at least one processor 32, at least one communication bus 33, and a transceiver 34.
It will be appreciated by those skilled in the art that the configuration of the computer device shown in fig. 3 is not limiting of the embodiments of the present invention, and that either a bus-type configuration or a star-type configuration is possible, and that the computer device 30 may include more or less other hardware or software than that shown, or a different arrangement of components.
In some embodiments, the computer device 30 is a computer device capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and its hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit, a programmable gate array, a digital processor, an embedded device, and the like. The computer device 30 may also include a client device, including but not limited to any electronic product that can interact with a client by way of a keyboard, mouse, remote control, touch pad, or voice control device, such as a personal computer, tablet, smart phone, digital camera, etc.
It should be noted that the computer device 30 is only used as an example, and other electronic products that may be present in the present invention or may be present in the future are also included in the scope of the present invention by way of reference.
In some embodiments, the memory 31 has a computer program stored therein, and the at least one processor 32 may call the computer program stored in the memory 31 to perform the relevant functions. For example, each of the modules described in the above embodiments is a computer program stored in the memory 31 and executed by the at least one processor 32, thereby realizing the functions of the respective modules. The Memory 31 includes Read-Only Memory (ROM), programmable Read-Only Memory (PROM), erasable programmable Read-Only Memory (EPROM), one-time programmable Read-Only Memory (One-time Programmable Read-Only Memory, OTPROM), electrically erasable rewritable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM) or other optical disc Memory, magnetic tape Memory, or any other medium that can be used for computer-readable carrying or storing data.
In some embodiments, the at least one processor 32 is a Control Unit (Control Unit) of the computer device 30, connects the various components of the entire computer device 30 using various interfaces and lines, and performs various functions and processes of the computer device 30 by running or executing programs or modules stored in the memory 31, and invoking data stored in the memory 31. For example, the at least one processor 32, when executing the computer programs stored in the memory, implements all or part of the steps of the structured data processing methods described in embodiments of the present invention. The at least one processor 32 may be comprised of integrated circuits, such as a single packaged integrated circuit, or may be comprised of multiple integrated circuits packaged with the same or different functionality, including one or more central processing units (Central Processing unit, CPU), microprocessors, digital processing chips, graphics processors, combinations of various control chips, and the like.
In some embodiments, the at least one communication bus 33 is arranged to enable connected communication between the memory 31 and the at least one processor 32 or the like.
Although not shown, the computer device 30 may also include a power source (e.g., a battery) for powering the various components, preferably the power source is logically connected to the at least one processor 32 via a power management device so as to perform functions such as managing charging, discharging, and power consumption via the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The computer device 30 may further include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described in detail herein.
The integrated units implemented in the form of software functional modules described above may be stored in a computer readable storage medium. The software functional modules described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, a computer device, or a network device, etc.) or processor (processor) to perform portions of the methods described in the various embodiments of the invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it will be obvious that the term "comprising" does not exclude other elements or that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.
Claims (7)
1. A structured data processing method, the structured data processing method comprising:
receiving structured data uploaded by a user through an uploading interface;
and calling an index algorithm to map the structured data to obtain a mapping value, and obtaining a total index of the structured data according to the mapping value, wherein the method comprises the following steps: traversing a plurality of row labels of the structured data; selecting an index algorithm and mapping the traversed row labels through the selected index algorithm to obtain a plurality of mapping values; converting each mapping value into a binary value, and determining the binary value as an index of data corresponding to a row label; splicing all indexes according to row marks to obtain a total index;
acquiring a user ID of the user and carrying out serialization processing on the user ID to obtain a serialized user ID;
Determining the data type of the structured data, and calling a fingerprint algorithm corresponding to the data type to embed the serialized user ID into the structured data to generate a fingerprint file, wherein the method comprises the following steps: traversing a plurality of column data of the structured data; identifying the data type of each column of data through an intelligent semantic identification algorithm; matching a fingerprint algorithm corresponding to the data type of each column of data; calculating the total number of data of each data type; calculating the number of the serialized user IDs to be inserted in each data type according to a preset proportion and the total number; randomly selecting target data corresponding to the number from each data type according to the number of the serialization user IDs to be inserted; invoking the fingerprint algorithm to insert the serialized user ID into the target data according to the index sequence of the target data to obtain a fingerprint file;
the total index of the structured data and the fingerprint file are stored and managed.
2. The structured data processing method of claim 1, wherein prior to said receiving structured data uploaded by a user via an upload interface, said structured data processing method further comprises:
Receiving a login request of the user and verifying the login request;
and responding to the login request verification, and determining that the user account carried in the login request is the user ID of the user.
3. The structured data processing method of claim 1 wherein said invoking said fingerprint algorithm to insert said serialized user ID into data of a corresponding data type, obtaining a fingerprint file comprises:
receiving external parameters input by the user, wherein the external parameters comprise coverage areas of data to be encrypted and line labels to be encrypted;
determining an index corresponding to the line mark number to be encrypted;
and calling the fingerprint algorithm to insert the serialized user ID into the data corresponding to the index according to the coverage area to obtain a fingerprint file.
4. A structured data processing method as claimed in any one of claims 1 to 3 wherein said structured data processing method further comprises:
acquiring a plurality of first information of the structured data and a plurality of second information of the fingerprint file;
and calling a metadata management tool to store the first information and the second information in an associated mode.
5. A structured data processing apparatus for implementing a structured data processing method as claimed in any one of claims 1 to 4, said structured data processing apparatus comprising:
the receiving module is used for receiving the structured data uploaded by the user through the uploading interface;
the mapping module is used for calling an index algorithm to map the structured data to obtain a mapping value, and obtaining the total index of the structured data according to the mapping value, and comprises the following steps: traversing a plurality of row labels of the structured data; selecting an index algorithm and mapping the traversed row labels through the selected index algorithm to obtain a plurality of mapping values; converting each mapping value into a binary value, and determining the binary value as an index of data corresponding to a row label; splicing all indexes according to row marks to obtain a total index;
the processing module is used for acquiring the user ID of the user and carrying out serialization processing on the user ID to obtain a serialized user ID;
the generating module is configured to determine a data type of the structured data, invoke a fingerprint algorithm corresponding to the data type, and embed the serialized user ID into the structured data to generate a fingerprint file, and includes: traversing a plurality of column data of the structured data; identifying the data type of each column of data through an intelligent semantic identification algorithm; matching a fingerprint algorithm corresponding to the data type of each column of data; calculating the total number of data of each data type; calculating the number of the serialized user IDs to be inserted in each data type according to a preset proportion and the total number; randomly selecting target data corresponding to the number from each data type according to the number of the serialization user IDs to be inserted; invoking the fingerprint algorithm to insert the serialized user ID into the target data according to the index sequence of the target data to obtain a fingerprint file;
And the storage module is used for storing and managing the total index of the structured data and the fingerprint file.
6. A computer device, characterized in that it comprises a processor for implementing the structured data processing method according to any of claims 1 to 4 when executing a computer program stored in a memory.
7. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the structured data processing method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583553.XA CN111782648B (en) | 2020-06-23 | 2020-06-23 | Structured data processing method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010583553.XA CN111782648B (en) | 2020-06-23 | 2020-06-23 | Structured data processing method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782648A CN111782648A (en) | 2020-10-16 |
| CN111782648B true CN111782648B (en) | 2023-08-18 |
Family
ID=72757226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010583553.XA Active CN111782648B (en) | 2020-06-23 | 2020-06-23 | Structured data processing method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782648B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999019817A2 (en) * | 1997-10-14 | 1999-04-22 | Bellsouth Intellectual Property Corporation | A system and method for processing a memory map to provide listing information representing data within a database |
| CN105095520A (en) * | 2015-09-23 | 2015-11-25 | 电子科技大学 | Distributed type in-memory database indexing method oriented to structural data |
| WO2018161881A1 (en) * | 2017-03-09 | 2018-09-13 | 腾讯科技（深圳）有限公司 | Structuralized data processing method, data storage medium, and computer apparatus |
| CN108701198A (en) * | 2016-02-17 | 2018-10-23 | 微软技术许可有限责任公司 | Conjunction SQL statement can search for encrypting |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080162526A1 (en) * | 2006-12-28 | 2008-07-03 | Uma Kant Singh | Method and system for managing unstructured data in a structured data environment |
-
2020
- 2020-06-23 CN CN202010583553.XA patent/CN111782648B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1999019817A2 (en) * | 1997-10-14 | 1999-04-22 | Bellsouth Intellectual Property Corporation | A system and method for processing a memory map to provide listing information representing data within a database |
| CN105095520A (en) * | 2015-09-23 | 2015-11-25 | 电子科技大学 | Distributed type in-memory database indexing method oriented to structural data |
| CN108701198A (en) * | 2016-02-17 | 2018-10-23 | 微软技术许可有限责任公司 | Conjunction SQL statement can search for encrypting |
| WO2018161881A1 (en) * | 2017-03-09 | 2018-09-13 | 腾讯科技（深圳）有限公司 | Structuralized data processing method, data storage medium, and computer apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782648A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110768968B (en) | Authorization method, device, equipment and system based on verifiable statement | |
| CN107196901B (en) | Identity registration and authentication method and device | |
| CN104320389B (en) | A kind of fusion identity protection system and method based on cloud computing | |
| CN113542288A (en) | Service authorization method, device, equipment and system | |
| US20220329446A1 (en) | Enhanced asset management using an electronic ledger | |
| US20190141048A1 (en) | Blockchain identification system | |
| EP3552131B1 (en) | Password security | |
| EP3206329B1 (en) | Security check method, device, terminal and server | |
| CN104506487A (en) | Credible execution method for privacy policy in cloud environment | |
| WO2009096999A9 (en) | Apparatus, and an associated methodology, for facilitating authentication using a digital music authentication token | |
| US20240039731A1 (en) | Authenticated Modification of Blockchain-Based Data | |
| US20160379328A1 (en) | Identification Authentication in a Communications Network | |
| JP2019028805A5 (en) | ||
| CN112637167A (en) | System login method and device, computer equipment and storage medium | |
| CN110717770B (en) | Anti-counterfeiting detection method, device, equipment and storage medium for vehicle parts | |
| CN114547701A (en) | Block chain-based tamper-proof identification chip information trusted storage system | |
| CN113158156A (en) | Service processing method, system, device, electronic equipment and storage medium | |
| CN111782648B (en) | Structured data processing method, device, computer equipment and storage medium | |
| CN111654591A (en) | Picture tamper-proofing method, computer equipment and storage medium | |
| CN108965335A (en) | Prevent method, electronic equipment and the computer media of malicious access login interface | |
| CN114422109A (en) | Information encryption method, device, server and medium | |
| CN102457484A (en) | Method for checking user information by combining user name/password authentication and check code | |
| CN115829186B (en) | ERP management method based on artificial intelligence and data processing AI system | |
| CN117176472B (en) | Data tamper-proof method, device and system based on intelligent password security equipment | |
| CN114238908B (en) | Page repeated login method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |