CN117290495A

CN117290495A - Data management method, device, storage medium and electronic equipment

Info

Publication number: CN117290495A
Application number: CN202311483916.2A
Authority: CN
Inventors: 潘学芳; 陈可心; 黄登玺; 冯鹤立; 王畅; 苏晓莹; 阮念军; 宁辛
Original assignee: China Everbright Bank Co Ltd
Current assignee: China Everbright Bank Co Ltd
Priority date: 2023-11-08
Filing date: 2023-11-08
Publication date: 2023-12-26

Abstract

The application discloses a data management method, a data management device, a storage medium and electronic equipment. Wherein the method comprises the following steps: obtaining external data provided by a target data source, wherein the external data is data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type mark and the attribution mark to perform summarization and arrangement on the external data, and storing the arranged external data into a target database. The application solves the technical problem of low data management efficiency.

Description

Data management method, device, storage medium and electronic equipment

Technical Field

The present invention relates to the field of computers, and in particular, to a data management method, apparatus, storage medium, and electronic device.

Background

In the related art, external data is usually stored in a database, and stored data in the database is uniformly processed by combining manual work, but the data management mode is low in efficiency and the cost of data management is increased. Therefore, there is a problem in that the data management efficiency is low.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the application provides a data management method, a data management device, a storage medium and electronic equipment, so as to at least solve the technical problem of low data management efficiency.

According to an aspect of the embodiments of the present application, there is provided a data management method, including: obtaining external data provided by a target data source, wherein the external data are data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type marks and the attribution marks to collect and sort the external data, and storing the sorted external data into the target database.

According to another aspect of the embodiments of the present application, there is also provided a data management apparatus, including: the first acquisition unit is used for acquiring external data provided by a target data source, wherein the external data are data to be stored in a target database; the first processing unit is used for classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; the second processing unit is used for carrying out attribution processing on the external data to obtain a second processing result, and carrying out attribution marking on each data in the external data by utilizing the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and the first sorting unit is used for summarizing and sorting the external data by combining the type marks and the attribution marks, and storing the sorted external data into the target database.

As an alternative, the first finishing unit includes: the first processing module is used for summarizing the external data by combining the type marks and the attribution marks to obtain a third processing result, and carrying out subdivision marks on each data in the external data by utilizing the third processing result, wherein the subdivision labels are subdivision labels of each data in the external data and are determined according to the data types and the attribution subjects; the first sorting module is used for sorting the external data by using the subdivision marks to obtain the sorted external data, and storing the sorted external data into the target database.

As an alternative, the first sorting module may sort the external data using the segment labels to obtain the sorted external data, and store the sorted external data in the target database, and then include: and the first display module is used for displaying a plurality of external data unified views according to the subdivision marks, wherein each external data unified view in the plurality of external data unified views is used for displaying the external data under the same subdivision mark.

As an alternative, the first processing unit includes: the first acquisition module is used for acquiring the data service type of the target data source, wherein the data service type is the type of the data service which the target data source is allowed to provide; and the second processing module is used for carrying out classification processing on the external data by combining the data service types to obtain the first processing result, and carrying out the type marking on the external data by using the first processing result.

As an alternative, the second processing module includes at least one of: the first marking submodule is used for marking the external data belonging to the wind control class in the external data by utilizing the first processing result; the second marking sub-module is used for marking the industry and commerce type of the external data belonging to the industry and commerce type in the external data by utilizing the first processing result; a third marking sub-module for marking the tax type of the external data belonging to the tax type in the external data by using the first processing result; a fourth marking sub-module for marking the external data belonging to the industry and commerce in the external data by using the first processing result; a fifth marking sub-module for marking the financial account number type of the external data belonging to the financial account number type in the external data by using the first processing result; a sixth marking sub-module for marking the personal basic information type of the external data belonging to the personal basic information type by using the first processing result; a seventh marking sub-module for marking judicial and punishment types of external data belonging to judicial and punishment types in the external data by using the first processing result; a ninth marking sub-module for marking the external data belonging to the credit category in the external data by using the first processing result; a tenth marking sub-module for marking the information class of the external data belonging to the information class in the external data by using the first processing result; an eleventh marking sub-module for marking the comprehensive class of the external data belonging to the comprehensive class in the external data by using the first processing result; and a twelfth marking sub-module for marking the external data belonging to the blacklist class in the external data by using the first processing result.

As an alternative, the second processing unit includes: the second acquisition module is used for acquiring target fields corresponding to each data in the external data; and the third processing module is used for determining the attribution subject corresponding to each data in the external data based on the target field and carrying out attribution marking on the external data with different attribution subjects.

As an optional solution, the second obtaining module includes, before the obtaining the target field corresponding to each data in the external data: a third obtaining module, configured to obtain original fields corresponding to respective data in the external data; a fourth processing module, configured to perform a screening process on the original resource to obtain the target field, where the screening process includes at least one of: screening out technical fields of a business process, screening out redundant fields, screening out fields with null values higher than a first threshold value, screening out fields with accuracy smaller than or equal to a second threshold value, screening out fields without clear business meanings, and screening out fields processed through basic fields.

According to yet another aspect of embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data management method as above.

According to still another aspect of the embodiments of the present application, there is further provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the data management method described above through the computer program.

In the embodiment of the application, external data provided by a target data source is obtained, wherein the external data is data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type marks and the attribution marks to collect and sort the external data, and storing the sorted external data into the target database. The data is subdivided, summarized and sorted through data classification and theme attribution, and the sorting result is visually displayed in a unified view mode of external data, so that the external data is grounded and unified multiplexing is realized, the external data is participated in data calculation and business logic calculation, the value of the external data is realized, the purposes of unified storage and unified sharing of the external data are further achieved, the technical effect of improving the efficiency of data management is realized, and the technical problem of lower data management efficiency is further solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:

FIG. 1 is a schematic illustration of an application environment of an alternative data management method according to an embodiment of the present application;

FIG. 2 is a schematic illustration of a flow of an alternative data management method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an alternative data management method according to an embodiment of the present application;

FIG. 4 is a schematic diagram of another alternative data management method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of another alternative data management method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of another alternative data management method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an alternative data management device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of an alternative electronic device according to an embodiment of the present application.

Detailed Description

In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to an aspect of the embodiments of the present application, there is provided a data management method, optionally, as an alternative implementation, the above data management method may be applied, but not limited to, in the environment shown in fig. 1. Which may include, but is not limited to, a user device 102 and a server 112, which may include, but is not limited to, a display 108, a processor 106, and a memory 1004, the server 112 including a database 114 and a processing engine 116.

The specific process comprises the following steps:

step S102, the user equipment 102 acquires target data 1002;

steps S104-S106, transmitting the target data 1002 to the server 112 via the network 110;

step S108, the server 112 determines, from the target data 1002, a classification result by the processing engine;

steps S110-S112, the data 1002 is sent to the user device 102 via the network 110, the user device 102 displays the processed data on the display 108 via the processor 106, and the processed data is stored in the memory 104.

In addition to the example shown in fig. 1, the above steps may be assisted by a server, that is, the steps of acquisition of external data, classification processing, attribution processing, and the like are performed by the server, thereby reducing the processing pressure of the server. The user device 102 includes, but is not limited to, a handheld device (e.g., a mobile phone), a notebook computer, a desktop computer, a vehicle-mounted device, etc., and the present application is not limited to a particular implementation of the user device 102.

Optionally, as an alternative embodiment, as shown in fig. 2, the data management method includes:

s202, obtaining external data provided by a target data source, wherein the external data is data to be stored in a target database;

S204, classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data;

s206, carrying out attribution processing on the external data to obtain a second processing result, and carrying out attribution marking on each data in the external data by utilizing the second processing result, wherein the attribution marking is used for indicating attribution subject of each data in the external data;

and S208, integrating the type marks and the attribution marks, sorting the external data, and storing the sorted external data into a target database.

Alternatively, in this embodiment, the above-mentioned data management method may be applied, but not limited to, in the scenario of a large data platform in the financial industry. The majority of financial institutions currently employ system acquisition program development to perform external data access operations in the form of API access or client database deployment. However, as financial institutions become intricate in systems and constantly updated, small institutions may have a system count on the order of tens, while larger institutions (e.g., commercial banks) may include hundreds of systems. In this case, both transaction-based and data-based systems are more or less dependent on data provided by external data providers and comprise hundreds of systems for larger institutions. Many organizations currently do not always have unified management of external data usage. Resulting in the problems of data redundancy and difficult arrangement.

For the problems, the prior art adopts multiple systems to automatically access and store external data, so that a large amount of repeated work is caused, and data among the systems cannot be shared because of inconsistent treatment standards, and the same type of data of the same provider cannot be shared and secondarily used among the systems. And the use cost of external data is higher, each system can be used and called without multiplexing historical receipts, thus the unnecessary cost is increased, and the technical problem of lower data management efficiency is generated.

According to the method and the device, the data are subdivided, summarized and sorted through data classification and subject attribution, and the unified view of the external data is displayed, so that unified storage and unified sharing of the external data are realized, key items such as code value conversion of valuable data unified standards, priority combination of data similar items, standardized processing of data formats, elimination of non-valuable interference data items and the like are realized, the external data are landed and unified multiplexing is realized, and the technical problem of low data management efficiency is solved.

Optionally, in this embodiment, the first processing result is a result obtained by performing a classification processing on the external data, the second processing result is a result obtained by performing a attribution processing on the external data, and the type label is used to indicate a data type to which each data in the external data belongs, which may be, but is not limited to, an air control class, an industrial class, a tax class, a financial account class, a personal basic information class, a judicial/punishment class, a credit class, an information class, a comprehensive class, a blacklist, and the like, and the attribution label is used to indicate an attribution subject of each data in the external data, which may be, but is not limited to, a principal, an event, a product, and a finance.

Alternatively, in the present embodiment, before the external data classification processing and the attribution processing, the following preparation work may be, but is not limited to, required: the main table service attribute, the service main key field, the table data range, the data updating mode, the physical deleting mode, the history data updating condition and the like are obtained from the external data provided by the target data source, and meanwhile, redundant fields, fields with poor data quality (high null value, low accuracy and the like), reserved fields without clear service meaning, I-shaped sections calculated through basic fields and the like are abandoned.

It should be noted that, external data provided by a target data source is obtained, wherein the external data is data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type mark and the attribution mark to perform summarization and arrangement on the external data, and storing the arranged external data into a target database. The data arrangement efficiency is improved through the technical means.

Further illustratively, as shown in fig. 3, an optional data service type 304 of external data provided by the external data 302 is obtained, where the data service type 304 is a type of data service allowed to be provided by a target data source, classification processing 308 is performed on the external data according to the data service type 304 to obtain a first processing result, the external data is marked with a type 312 according to the first processing result, a target field 306 corresponding to each data in the external data is obtained, a subject of attribution corresponding to each data is determined through the target field, attribution processing 310 is performed on the external data based on the subject of attribution corresponding to each data to obtain a second processing result, attribution marking 314 is performed on each data in the external data according to the second processing result, and summarizing and sorting 316 is performed on the external data by combining the type marking 312 and the attribution marking 314. And (3) cleaner data are obtained, so that the subsequent conversion of internal data is convenient.

According to the embodiment provided by the application, the external data provided by the target data source is obtained, wherein the external data are data to be stored in the target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type marks and the attribution marks to collect and sort the external data, and storing the sorted external data into the target database. The data is subdivided, summarized and sorted through data classification and theme attribution, and the sorting result is visually displayed in a unified view mode of external data, so that the external data is grounded and unified multiplexing is realized, the external data is participated in data calculation and business logic calculation, the value of the external data is realized, the purposes of unified storage and unified sharing of the external data are further achieved, the technical effect of improving the efficiency of data management is realized, and the technical problem of lower data management efficiency is further solved.

As an alternative, the method for summarizing and sorting the external data by combining the type mark and the attribution mark, and storing the sorted external data into the target database includes:

s1, summarizing external data by combining type marks and attribution marks to obtain a third processing result, and carrying out subdivision marks on all data in the external data by utilizing the third processing result, wherein subdivision marks are subdivision labels which are determined by all data in the external data according to data types and attribution subjects;

s2, sorting the external data by using the subdivision marks to obtain sorted external data, and storing the sorted external data into a target database.

Optionally, in this embodiment, the subdivision label is divided into each data in the external data, and the subdivision label is determined together according to the data type and the attribution theme, which may be, but is not limited to, enterprise basic information, external investment information, personnel information, associated party information, enterprise annual report information, enterprise own abnormal information, credit score, and the like, and is not limited in this regard;

it is to be noted that, summarizing the external data by combining the type mark and the attribution mark to obtain a third processing result, and carrying out subdivision marking on each data in the external data by utilizing the third processing result, wherein the subdivision mark is a subdivision label which is determined by each data in the external data according to the data type and attribution subject; and sorting the external data by using the subdivision labels to obtain sorted external data, and storing the sorted external data into a target database. Has the beneficial effect of optimizing the redundancy of the data.

Further by way of example, the external data may optionally be consolidated, for example, according to a type tag and a home tag, to obtain a third processing result. And the third processing result is utilized to carry out subdivision marking on the external data, and the subdivision marking is utilized to carry out induction finishing on the external data so as to obtain finer and tidier external data, thereby reducing the multiple mess of the data and enabling the data to be more in line with standardization.

According to the embodiment provided by the application, the external data is summarized by combining the type mark and the attribution mark to obtain a third processing result, and each piece of data in the external data is subdivided and marked by utilizing the third processing result, wherein the subdivision label is a subdivision label of each piece of data in the external data and is determined according to the data type and attribution subject; the external data is sorted by using the subdivision attribute marks to obtain sorted external data, and the sorted external data is stored in the target database, so that the aim of further subdividing the data is fulfilled, and the technical effect of greatly increasing the data processing efficiency is realized.

As an alternative, after sorting the external data using the subdivision index, obtaining sorted external data, and storing the sorted external data in the target database, the method includes:

And displaying a plurality of external data unified views according to the subdivision labels, wherein each external data unified view in the plurality of external data unified views is used for displaying the external data under the same subdivision label.

Optionally, in this embodiment, the external data unified view is divided into an enterprise external data unified view and a personal external data unified view, and is mainly used for displaying external data under the same mark, the external data is sorted according to the mark, so as to obtain sorted external data, and a plurality of external data unified views are displayed through the sorted external data.

It should be noted that, a plurality of external data unified views are displayed according to the subdivision label, wherein each external data unified view in the plurality of external data unified views is used for displaying the external data under the same subdivision label, which has the beneficial effect of clearly processing the data arrangement degree.

By way of further illustration, as shown in FIG. 4, an alternative example is to sort 404 external data 402 provided by an outsourcing company, determine that the type tag of the external data is business type data, and perform an attribution 406 on the external data, determine that the attribution tag of the external data is an event, aggregate 408 the attribution tag with the type tag, determine that the subdivision tag 410 of the data is administrative penalty information, and display a unified view of the administrative penalty information. In the unified view outside the company, the unified credit code registration number, the enterprise name and the organization code of the company are used for distinguishing the unique business body, the industrial and commercial data are used as total data, and the industrial and commercial data are arranged based on the basic data according to the integrity and the sequence of the data, so that the unified view 412 taking the industrial and commercial data as the total data is obtained. I.e., the consolidated data, as shown in fig. 5, including principals, events, products, etc. in the home topic; the data classification comprises industry and commerce classes, tax classes, risk classes and the like; the data subdivision comprises finer enterprise basic information, external investment information, personnel information and the like.

According to the embodiment provided by the application, the plurality of external data unified views are displayed according to the subdivision label, wherein each external data unified view in the plurality of external data unified views is used for displaying the external data under the same subdivision label, so that the aim of realizing access definition of the displayed external data is achieved, and the technical effects of landing and unified multiplexing of the external data are realized.

As an alternative, the classifying processing is performed on the external data to obtain a first processing result, and the type marking is performed on the external data by using the first processing result, including:

s1, acquiring a data service type of a target data source, wherein the data service type is a type of data service which the target data source is allowed to provide;

s2, classifying the external data by combining the data service types to obtain a first processing result, and marking the type of the external data by using the first processing result. ,

alternatively, in the present embodiment, the data service type is a form of data service provided by an external data provider.

It should be noted that, the data service type of the target data source is obtained, where the data service type is the type of data service that the target data source is allowed to provide; and classifying the external data by combining the data service type to obtain a first processing result, and marking the type of the external data by using the first processing result. Has the beneficial effect of improving the intuitiveness in processing data

Further by way of example, the method includes the steps of optionally acquiring a data service type of a target data source provided by an outsourcing company, classifying the external data to obtain a first processing result of the external data, determining that the external data is seal control type external data according to the first processing result of the external data, and marking the external data with seal control type.

According to the embodiment provided by the application, the data service type of the target data source is obtained, wherein the data service type is the type of the data service which the target data source is allowed to provide; the external data is classified according to the data service type to obtain a first processing result, and the type of the external data is marked by utilizing the first processing result, so that the aim of classifying and marking the data is fulfilled, and the technical effects of intuitiveness and cleanliness of data processing are realized.

As an alternative, the type of the external data is marked by using the first processing result, including at least one of the following:

s1, carrying out wind control type marking on external data belonging to the wind control type in the external data by utilizing a first processing result;

s2, carrying out industrial and commercial marking on the external data belonging to the industrial and commercial category in the external data by utilizing the first processing result;

S3, marking the external data belonging to the tax type in the external data by using the first processing result;

s4, carrying out industrial and commercial marking on the external data belonging to the industrial and commercial category in the external data by utilizing the first processing result;

s5, marking the external data belonging to the financial account number class in the external data by utilizing the first processing result;

s6, marking the external data belonging to the personal basic information class in the external data by using the first processing result;

s7, marking judicial and punishment types of external data belonging to judicial and punishment types in the external data by using the first processing result;

s8, carrying out credit sign type marking on the external data belonging to the credit sign type in the external data by utilizing the first processing result;

s9, marking the external data belonging to the information class in the external data by utilizing the first processing result;

s10, carrying out comprehensive class marking on external data belonging to comprehensive classes in the external data by utilizing a first processing result;

s11, carrying out blacklist type marking on external data belonging to the blacklist type in the external data by utilizing the first processing result.

According to the embodiment provided by the application, the external data belonging to the wind control class in the external data is marked by utilizing the first processing result; carrying out industrial and commercial marking on the external data belonging to the industrial and commercial category in the external data by utilizing the first processing result; carrying out tax type marking on external data belonging to tax types in the external data by utilizing the first processing result; carrying out industrial and commercial marking on the external data belonging to the industrial and commercial category in the external data by utilizing the first processing result; the first processing result is utilized to mark the external data belonging to the financial account number class in the external data; marking the external data belonging to the personal basic information class in the external data by using the first processing result; marking judicial and punishment types of external data belonging to judicial and punishment types in the external data by using a first processing result; carrying out credit sign type marking on external data belonging to the credit sign type in the external data by utilizing the first processing result; using the first processing result to mark the information class of the external data belonging to the information class; carrying out comprehensive class marking on external data belonging to comprehensive classes in the external data by utilizing the first processing result; and marking the external data belonging to the blacklist class in the external data by using the first processing result. And the aim of classifying and marking the data is fulfilled, so that the technical effect of cleanliness of data processing is realized.

Optionally, in this embodiment, performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, including:

s1, obtaining target fields corresponding to all data in external data;

s2, determining attribution subjects corresponding to each data in the external data based on the target field, and attributing and marking the external data of different attribution subjects.

By way of further illustration, the method includes the steps of optionally, for example, obtaining an original field corresponding to each data in the external data provided by the outsourcing company, and performing screening processing on the original field, including screening redundant fields, screening out fields without explicit meanings, removing useless spaces, and expressing dates in the same numerical form to obtain a target field corresponding to each data, wherein the target field is used for determining that the attribution subject of the external data is a financial subject, and marking the external data belonging to the financial subject as a financial class mark.

The process is that the external data provided for enterprise clients comprises unified social credit code, registration number, enterprise name and organization code (which can be intercepted from the unified social credit code), and the external data provided for individual clients comprises name and certificate number. The principal number is added in the external data service table uniformly for uniquely distinguishing the service main body, the enterprise unifies the social credit code/registration number/enterprise name/organization code and other relations, the individual passes through the certificate number (such as encryption of the certificate number, then the personal is associated with the internal data through data collision after encryption), the personal is associated with the internal data enterprise client to obtain the internal data unique identification code, the relation between the external principal number and the internal data unique identification code is maintained in the relation table, and the internal and external data communication is realized.

According to the embodiment provided by the application, the target field corresponding to each data in the external data is obtained; and determining attribution topics corresponding to each data in the external data based on the target field, and attributing and marking the external data of different attribution topics. And furthermore, the aim of standardized naming is fulfilled, and the technical effect of convenience in multi-source data merging is realized.

As an alternative, before acquiring the target field corresponding to each data in the external data, the method includes:

s1, acquiring original fields corresponding to all data in external data;

s2, screening the original resources to obtain target fields, wherein the screening comprises at least one of the following steps: screening out technical fields of a business process, screening out redundant fields, screening out fields with null values higher than a first threshold value, screening out fields with accuracy smaller than or equal to a second threshold value, screening out fields without clear business meanings, and screening out fields processed through basic fields.

According to the embodiment provided by the application, the original fields corresponding to the data in the external data are obtained; screening the original resources to obtain target fields, wherein the screening comprises at least one of the following steps: the technical fields of the business process are screened out, redundant fields are screened out, fields with null values higher than a first threshold value are screened out, fields with accuracy smaller than or equal to a second threshold value are screened out, fields without clear business meanings are screened out, fields processed through basic fields are screened out, and then the purpose of opening internal and external data is achieved, so that the technical effects of multiple types of accessed external data, clear and clear classification and convenience in asset management are achieved.

For easy understanding, the data management method is applied to a specific financial industry data processing scene:

for example, as shown in fig. 6, the external data model is mainly hierarchically divided into a paste source layer 602, a base layer 604 and a summary layer 606.

Pasting a source layer: unified storage, unified model template design, according to different interface storage, keep interface technical parameter, business attribute, permanent storage, facilitate business post analysis; the design details include:

s1, judging the data service form provided by an external data provider. When the real-time service is designed, recording input parameters and output results; when the batch service is designed, redundant input parameters are input; database service, which reserves original structured data and only processes the name of the name adding source system;

s2, judging the main data service table, and possibly existence of an auxiliary table, a father table, a relation table and the like. Mainly comprises one-to-one, one-to-many, one-to-null, many-to-many, master-slave relationship and the like;

s3, the data are connected in series by using a unique technical primary key (the code adopted is a transaction code + a timestamp + a 6-bit random code) and aiming at one query or one client main body, a plurality of data use position self-codes exist, and the data can be used for recovering the original semi-structured data;

S4, checking the necessary indispensable fields of the service field in storage, and providing key fields by a data provider, wherein the confidential data is encrypted by the irreversible encryption algorithm agreed by the two parties. For example, the business information is indispensable to unify social credit codes, legal names, legal identity information and the like;

s5, table naming references the custom transaction code plus the table name or label name of the data provider, the field name references the field name of the data provider when naming, renames the key words in the database, and the field length consults the rule of the provider on the code to perform expansion operation of 1, 2 or 3;

s6, classifying the source layer data from the data asset angle, wherein the main category refers to data services provided by a data provider, the data access categories are divided into ten major categories, different classifications are defined by the data access categories, and the current classifications are wind control categories, industrial and commercial categories, tax categories, financial account categories, personal basic information categories, judicial/punishment categories, credit investigation categories, information categories, comprehensive categories, blacklists and the like.

The main purpose of the basic layer is to divide the topic domain, filter the dirty and bad data, define the data cleaning rule, merge the multi-source data, define the data standard, normalize and name the entity attribute, and open the internal and external data, the technical details include:

S1, table analysis, namely determining whether a data table enters a model and a subject to which the data table belongs based on system level investigation and sample data analysis, and entering a model screening rule: the following type tables are not in the model, including but not limited to a control class table, a derivative table, an intermediate table, a temporary table, a reserved table, a redundant table, a useless table, an empty table, a backup table and the like, specifically whether a required service is reserved or not and whether a service is confirmed to have an actual application scene or not is judged, if a specific application scene is recommended to be reserved; the special description input parameter table is abandoned when the platform characteristic is put into the model, and the service field redundancy is concentrated in the service result; if the input result table only returns a state, discarding the state when no specific service field exists, and using the state as a service field;

s2, field level analysis, judging whether the fields enter a model or not based on sample data analysis and technical fields in the interface document, defining theme, entity and attribute names, and entering a model screening rule: the following type fields are not included in the model, and include, but are not limited to, fields for business process technology, redundant fields, fields with poor data quality (higher null value, lower accuracy, etc.), reserved fields without explicit business meaning, i-section addition calculated by basic fields, etc.;

S3, designing a logic model, and displaying a basic layer logic model through the topic-dividing ER diagram based on field level analysis;

s4, designing a physical model, wherein the physical model is based on a logic model design, the physical model is basically consistent with the logic model, and 2 technical fields including a data_src data source and a data_dt data date are added to external data during physics;

s5, code standardization, wherein the mapping of source codes and target codes is considered in the design, the standard condition of the target codes is judged through the association of a code table, and related code mapping is provided by contacting a data merchant as soon as possible;

s6, a mapping relation from a source to a target, recording a mapping rule of a target table and a source table, wherein one target table can have a plurality of groups and is sourced from different data providers or different interface service forms of the same provider; the external data mapping system not only comprises external data mapping provided by an external data provider, but also comprises external data and internal data mapping relations, wherein the main template fields of the mapping relations are as follows: target table, group, source table extraction mode, loading strategy, mapping description, connection table, etc.;

s7, the internal and external data are communicated, the external data are divided into enterprise clients and individual clients from the client perspective during access, and the key fields are the necessary transmission or necessary return fields during interface access. The business client contains a unified social credit code, registration number, business name, organization code (which may be truncated from the unified social credit code), and the individual client contains a name, a certificate number. The principal number is added in the external data service table uniformly for uniquely distinguishing the service main body, the enterprise unifies the social credit code/registration number/enterprise name/organization code and other relations, the individual passes through the certificate number (such as encryption of the certificate number, then the personal is associated with the internal data through data collision after encryption), the personal is associated with the internal data enterprise client to obtain the internal data unique identification code, the relation between the external principal number and the internal data unique identification code is maintained in the relation table, and the internal and external data communication is realized.

Summarizing layer: the related theme design of the summarization layer is oriented to business application and prefers business classification definition, and the classification defined at present is as follows: technical details of enterprise basic information, enterprise risk information, enterprise annual report, enterprise rating information, enterprise upstream and downstream, enterprise index summarization and the like include:

s1, an enterprise external data unified view, wherein the external data access enterprise related data comprise industrial and commercial class, tax class, judicial/punishment class, credit and comprehensive class data, and the information class is used as a single theme and is not included in the enterprise external data unified view range. Determining industrial and commercial data as main data, confirming the sequence of updating data according to the timeliness and the data integrity of an interface, processing based on basic layer data, performing redundant description on code values in a summary layer in the basic layer, realizing unique latest records of clients in the summary layer, reserving a data reservation strategy, reserving data of the last month and the latest 7 days, defining a unique enterprise identification strategy, prioritizing enterprise unified social credit codes, registering numbers, organizing organization codes and finally enterprise names;

s2, integrating views of personal external data, wherein the external data access personal related data comprise wind control type, personal basic information type, credit investigation type and blacklist type data, the personal identification information is unique and is composed of three elements, name, certificate type and certificate number, the sequence of updating personal basic information is confirmed according to timeliness and data integrity of an interface, processing is carried out based on basic layer data, score data are combined, and effective latest scores, contact person information, associated enterprise information and the like are provided.

It will be appreciated that in the specific embodiments of the present application, related data such as user information is referred to, and when the above embodiments of the present application are applied to specific products or technologies, user permissions or consents need to be obtained, and the collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.

It should be noted that, for simplicity of description, the foregoing method embodiments are all expressed as a series of action combinations, but it should be understood by those skilled in the art that the present application is not limited by the order of actions described, as some steps may be performed in other order or simultaneously in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required in the present application.

According to another aspect of the embodiments of the present application, there is also provided a data management apparatus for implementing the above data management method. As shown in fig. 7, the apparatus includes:

a first obtaining unit 702, configured to obtain external data provided by a target data source, where the external data is data to be stored in a target database;

A first processing unit 704, configured to perform classification processing on external data to obtain a first processing result, and perform type marking on the external data by using the first processing result, where the type marking is used to indicate a data type to which each data in the external data belongs;

a second processing unit 706, configured to perform attribution processing on the external data to obtain a second processing result, and perform attribution marking on each data in the external data by using the second processing result, where the attribution marking is used to indicate an attribution subject of each data in the external data;

a first sorting unit 708, configured to aggregate and sort the external data in combination with the type tag and the home tag, and store the sorted external data in the target database.

Alternatively, in this embodiment, the data management apparatus may be applied, but not limited to, in the scenario of a large data platform in the financial industry. The majority of financial institutions currently employ system acquisition program development to perform external data access operations in the form of API access or client database deployment. However, as financial institutions become intricate in systems and constantly updated, small institutions may have a system count on the order of tens, while larger institutions (e.g., commercial banks) may include hundreds of systems. In this case, both transaction-based and data-based systems are more or less dependent on data provided by external data providers and comprise hundreds of systems for larger institutions. Many organizations currently do not always have unified management of external data usage. Resulting in the problems of data redundancy and difficult arrangement.

It should be noted that, external data provided by a target data source is obtained, wherein the external data is data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type mark and the attribution mark to perform summarization and arrangement on the external data, and storing the arranged external data into a target database. By the technical means, the data arrangement efficiency is improved, and the beneficial effects of wide data applicability are achieved.

According to the embodiment provided by the application, external data provided by a target data source is obtained, wherein the external data is data to be stored in a target database; classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data; performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data; and integrating the type mark and the attribution mark to perform summarization and arrangement on the external data, and storing the arranged external data into a target database. The data is subdivided, summarized and sorted through data classification and theme attribution, and the technical means of displaying unified views of external data is further achieved, so that the purposes of enabling the external data to fall to the ground, multiplexing uniformly and participating in data calculation and business logic calculation are achieved, the value of the external data is achieved, the technical effects of unified storage and unified sharing of the external data are achieved, and the technical problem that the data management efficiency is low is solved.

Specific embodiments may refer to the examples shown in the above data management apparatus, and in this example, details are not described herein.

As an alternative, the first finishing unit includes: the first processing module is used for summarizing the external data by combining the type mark and the attribution mark to obtain a third processing result, and carrying out subdivision marking on each data in the external data by utilizing the third processing result, wherein the subdivision mark is a subdivision label which is determined by each data in the external data according to the data type and the attribution subject; the first sorting module is used for sorting the external data by utilizing the subdivision marks to obtain the sorted external data, and storing the sorted external data into the target database.

Specific embodiments may refer to the examples shown in the above data management method, and this example is not described herein.

As an alternative, the first sorting module, after sorting the external data using the subdivision label to obtain sorted external data, and storing the sorted external data in the target database, includes: and the first display module is used for displaying a plurality of external data unified views according to the subdivision labels, wherein each external data unified view in the plurality of external data unified views is used for displaying the external data under the same subdivision label.

As an alternative, the first processing unit 704 includes: the first acquisition module is used for acquiring the data service type of the target data source, wherein the data service type is the type of the data service which the target data source is allowed to provide; and the second processing module is used for carrying out classification processing on the external data in combination with the data service type to obtain a first processing result, and carrying out type marking on the external data by utilizing the first processing result.

As an alternative, the second processing module includes at least one of: the first marking sub-module is used for marking the external data belonging to the wind control class in the external data by utilizing the first processing result; the second marking sub-module is used for marking the industry and commerce type of the external data belonging to the industry and commerce type in the external data by utilizing the first processing result; the third marking sub-module is used for marking the tax type of the external data belonging to the tax type in the external data by utilizing the first processing result; a fourth marking sub-module for marking the external data belonging to the industry and the commerce in the external data by using the first processing result; a fifth marking sub-module for marking the financial account number type of the external data belonging to the financial account number type by using the first processing result; a sixth marking sub-module for marking the personal basic information class of the external data belonging to the personal basic information class by using the first processing result; a seventh marking sub-module for marking the judicial and punishment class of the external data belonging to the judicial and punishment class by using the first processing result; a ninth marking sub-module for marking the external data belonging to the credit category in the external data by using the first processing result; a tenth marking sub-module for marking the information class of the external data belonging to the information class by using the first processing result; an eleventh marking sub-module for marking the external data belonging to the comprehensive class in the external data by using the first processing result; and a twelfth marking sub-module for marking the external data belonging to the blacklist class in the external data by using the first processing result.

As an alternative, the second processing unit includes: the second acquisition module is used for acquiring target fields corresponding to all data in the external data; and the third processing module is used for determining attribution subjects corresponding to each data in the external data based on the target field and attributing and marking the external data of different attribution subjects.

As an optional solution, the second obtaining module, before obtaining the target field corresponding to each data in the external data, includes: the third acquisition module is used for acquiring original fields corresponding to each data in the external data; the fourth processing module is configured to perform screening processing on the original resource to obtain a target field, where the screening processing includes at least one of: screening out technical fields of a business process, screening out redundant fields, screening out fields with null values higher than a first threshold value, screening out fields with accuracy smaller than or equal to a second threshold value, screening out fields without clear business meanings, and screening out fields processed through basic fields.

According to a further aspect of the embodiments of the present application, there is also provided an electronic device for implementing the above-mentioned data management method, as shown in fig. 8, the electronic device comprising a memory 802 and a processor 804, the memory 802 having stored therein a computer program, the processor 804 being arranged to perform the steps of any of the method embodiments described above by means of the computer program.

Alternatively, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of the computer network.

Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:

s1, obtaining external data provided by a target data source, wherein the external data are data to be stored in a target database;

s2, classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data;

s3, carrying out attribution processing on the external data to obtain a second processing result, and carrying out attribution marking on each data in the external data by utilizing the second processing result, wherein the attribution marking is used for indicating attribution subject of each data in the external data;

And S4, integrating the type marks and the attribution marks, sorting the external data, and storing the sorted external data into a target database.

Alternatively, it will be understood by those skilled in the art that the structure shown in fig. 8 is only schematic, and the electronic device may also be a terminal device such as a smart phone (e.g. an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a mobile internet device (Mobile Internet Devices, MID), a PAD, etc. Fig. 8 is not limited to the structure of the electronic device described above. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 8, or have a different configuration than shown in FIG. 8.

The memory 802 may be used to store software programs and modules, such as program instructions/modules corresponding to the data management methods and apparatuses in the embodiments of the present application, and the processor 804 executes the software programs and modules stored in the memory 802, thereby performing various functional applications and data processing, that is, implementing the data management methods described above. Memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 802 may further include memory remotely located relative to processor 804, which may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 802 may be used to store, but is not limited to, a candidate set of probe points, a blocking degree, and a target set of probe points. As an example, as shown in fig. 8, the memory 802 may include, but is not limited to, the first acquiring unit 702, the second acquiring unit 704, the screening unit 706, and the third acquiring unit 708 in the data management apparatus. In addition, other module units in the above data management apparatus may be included, but are not limited to, and are not described in detail in this example.

Optionally, the transmission device 806 is used to receive or transmit data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission means 806 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices and routers via a network cable to communicate with the internet or a local area network. In one example, the transmission device 806 is a Radio Frequency (RF) module for communicating wirelessly with the internet.

In addition, the electronic device further includes: a display 808, configured to display information such as the first candidate probe point set, the blocking degree, and the target probe point set; and a connection bus 810 for connecting the respective module parts in the above-described electronic device.

In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the plurality of nodes through a network communication. Among them, the nodes may form a Peer-To-Peer (P2P) network, and any type of computing device, such as a server, a terminal, etc., may become a node in the blockchain system by joining the Peer-To-Peer network.

According to one aspect of the present application, a computer program product is provided, comprising a computer program/instructions containing program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. When executed by a central processing unit, performs the various functions provided by the embodiments of the present application.

The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.

It should be noted that the computer system of the electronic device is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.

The computer system includes a central processing unit (Central Processing Unit, CPU) which can execute various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) or a program loaded from a storage section into a random access Memory (Random Access Memory, RAM). In the random access memory, various programs and data required for the system operation are also stored. The CPU, the ROM and the RAM are connected to each other by bus. An Input/Output interface (i.e., I/O interface) is also connected to the bus.

The following components are connected to the input/output interface: an input section including a keyboard, a mouse, etc.; an output section including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and the like, and a speaker, and the like; a storage section including a hard disk or the like; and a communication section including a network interface card such as a local area network card, a modem, and the like. The communication section performs communication processing via a network such as the internet. The drive is also connected to the input/output interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and the like are mounted on the drive as needed so that a computer program read therefrom is mounted into the storage section as needed.

In particular, according to embodiments of the present application, the processes described in the various method flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such embodiments, the computer program may be downloaded and installed from a network via a communication portion, and/or installed from a removable medium. The computer program, when executed by a central processing unit, performs the various functions defined in the system of the present application.

According to one aspect of the present application, there is provided a computer-readable storage medium, from which a processor of a computer device reads the computer instructions, the processor executing the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

Alternatively, in the present embodiment, the above-described computer-readable storage medium may be configured to store a computer program for executing the steps of:

Alternatively, in this embodiment, it will be understood by those skilled in the art that all or part of the steps in the methods of the above embodiments may be performed by a program for instructing a terminal device to execute the steps, where the program may be stored in a computer readable storage medium, and the storage medium may include: flash disk, read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), magnetic or optical disk, and the like.

The integrated units in the above embodiments may be stored in the above-described computer-readable storage medium if implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to cause one or more computer devices (which may be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods of the various embodiments of the present application.

In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and are merely a logical functional division, and there may be other manners of dividing the apparatus in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application and are intended to be comprehended within the scope of the present application.

Claims

1. A method of data management, comprising:

obtaining external data provided by a target data source, wherein the external data is data to be stored in a target database;

classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data;

performing attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data;

And summarizing and sorting the external data by combining the type mark and the attribution mark, and storing the sorted external data into the target database.

2. The method of claim 1, wherein the summarizing the external data in combination with the type tag and the home tag and storing the consolidated external data in the target database comprises:

summarizing the external data by combining the type mark and the attribution mark to obtain a third processing result, and carrying out subdivision marking on each data in the external data by utilizing the third processing result, wherein the subdivision label is a subdivision label of each data in the external data, which is determined together according to the data type and the attribution subject;

and sorting the external data by using the subdivision marks to obtain the sorted external data, and storing the sorted external data into the target database.

3. The method according to claim 2, wherein after the sorting of the external data using the segment labels, the sorting of the external data is obtained, and the sorting of the external data is stored in the target database, comprising:

And displaying a plurality of external data unified views according to the subdivision labels, wherein each external data unified view in the plurality of external data unified views is used for displaying external data under the same subdivision label.

4. The method of claim 1, wherein classifying the external data to obtain a first processing result, and marking the external data with the type by using the first processing result, comprises:

acquiring a data service type of the target data source, wherein the data service type is a type of data service which the target data source is allowed to provide;

and classifying the external data according to the data service type to obtain the first processing result, and marking the type of the external data by using the first processing result.

5. The method of claim 4, wherein said marking said external data with said first processing result comprises at least one of:

carrying out wind control type marking on external data belonging to wind control types in the external data by utilizing the first processing result;

Carrying out industrial and commercial marking on the external data belonging to the industrial and commercial category in the external data by utilizing the first processing result;

carrying out tax type marking on external data belonging to tax types in the external data by utilizing the first processing result;

utilizing the first processing result to mark the external data belonging to the financial account types in the external data;

marking the external data belonging to the personal basic information class in the external data by using the first processing result;

marking the judicial and punishment class of the external data belonging to the judicial and punishment class by using the first processing result;

carrying out credit sign type marking on the external data belonging to the credit sign type in the external data by utilizing the first processing result;

marking the information class of the external data belonging to the information class in the external data by utilizing the first processing result;

carrying out comprehensive class marking on external data belonging to comprehensive classes in the external data by utilizing the first processing result;

And marking the external data belonging to the blacklist class in the external data by using the first processing result.

6. The method according to claim 1, wherein the performing the attribution processing on the external data to obtain a second processing result, and performing attribution marking on each data in the external data by using the second processing result, includes:

acquiring target fields corresponding to each data in the external data;

and determining attribution subjects corresponding to each data in the external data based on the target field, and carrying out attribution marking on the external data of different attribution subjects.

7. The method of claim 6, comprising, prior to said obtaining the destination field for each of the external data:

acquiring original fields corresponding to each data in the external data;

screening the original resources to obtain the target field, wherein the screening comprises at least one of the following steps: screening out technical fields of a business process, screening out redundant fields, screening out fields with null values higher than a first threshold value, screening out fields with accuracy smaller than or equal to a second threshold value, screening out fields without clear business meanings, and screening out fields processed through basic fields.

8. A data management apparatus, comprising:

the first acquisition unit is used for acquiring external data provided by a target data source, wherein the external data are data to be stored in a target database;

the first processing unit is used for classifying the external data to obtain a first processing result, and marking the type of the external data by using the first processing result, wherein the type mark is used for indicating the data type of each data in the external data;

the second processing unit is used for carrying out attribution processing on the external data to obtain a second processing result, and carrying out attribution marking on each data in the external data by utilizing the second processing result, wherein the attribution marking is used for indicating attribution subjects of each data in the external data;

the first sorting unit is used for summarizing and sorting the external data by combining the type mark and the attribution mark, and storing the sorted external data into the target database.

9. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program is executable by a terminal device or a computer to perform the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method according to any of the claims 1 to 7 by means of the computer program.