CN108959381B - Data management method and device and electronic equipment - Google Patents
Data management method and device and electronic equipment Download PDFInfo
- Publication number
- CN108959381B CN108959381B CN201810535427.XA CN201810535427A CN108959381B CN 108959381 B CN108959381 B CN 108959381B CN 201810535427 A CN201810535427 A CN 201810535427A CN 108959381 B CN108959381 B CN 108959381B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- field
- data table
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present specification provides a data management scheme: receiving a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data; determining corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one; determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata; and processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
Description
Technical Field
The embodiment of the specification relates to the technical field of internet, in particular to a data management method and device and electronic equipment.
Background
A Database (Database) is a repository built on computer storage devices that organizes, stores, and manages data by data type. Because different data fields exist in the data of different data types, different data tables need to be designed in the database according to the different data types, and thus the data of the same data type can be stored in the corresponding data table.
However, since the data tables with different data types are involved, a separate set of processing schemes needs to be designed for each data type when processing data, and the processing scheme corresponding to each data type can be generally referred to as a data processing model.
There is a need to provide a generic data management scheme.
Disclosure of Invention
The embodiment of the specification provides a data management method and device and an electronic device:
according to a first aspect of embodiments herein, there is provided a method of managing data, the method comprising:
receiving a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
determining corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata;
and processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
According to a second aspect of embodiments herein, there is provided an apparatus for managing data, the apparatus comprising:
a receiving unit that receives a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
the first determining unit is used for determining corresponding target metadata and a target data table according to the metadata identification; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
the second determining unit is used for determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata;
and the processing unit is used for processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
According to a third aspect of embodiments herein, there is provided an electronic apparatus including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured as a management method of any one of the above data.
The embodiment of the specification provides a data management scheme, and a mapping relation between a data field of target data to be processed and a data table field in a target data table is established by using metadata. In one aspect, the same set of data processing models can be employed for any data type. On the other hand, when the data table is established, the data fields specifically existing for the data of the data type do not need to be concerned; the data sheet design has universality, and the same data sheet field, namely the model field, can be adopted when different data sheet designs are carried out.
Drawings
FIG. 1 is a schematic diagram of a prior art data management scheme;
FIG. 2 is a schematic diagram of a data management scheme provided by an embodiment of the present description;
FIG. 3 is a flow chart of a method for managing data provided by an embodiment of the present disclosure;
fig. 4 is a hardware configuration diagram of a data management apparatus according to an embodiment of the present specification;
fig. 5 is a block diagram of a data management apparatus according to an embodiment of the present disclosure.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.
The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
As described above, since different data fields exist in data of different data types, different data tables need to be designed in the database according to different data types, so that data of the same data type can be stored in the corresponding data table.
For example, taking two data types, namely student data and teacher data, as an example, two different data tables are correspondingly designed in the traditional data table design concept;
assuming that the student data has data fields of student _ ID (student status number), student _ name (student name), student _ class (student class), student _ sex (student sex), student _ age (student age), it can be designed as a student data table as shown in table 1 below.
TABLE 1
| student_ID | student_name | student_class | student_sex | student_age |
Assuming that the teacher data has data fields of teacher _ ID (teacher number), teacher _ name (teacher name), teacher _ class (teacher class), teacher _ six (teacher gender), teacher _ age (teacher age), it can be designed as a teacher data table as shown in table 2 below.
TABLE 2
| teacher_ID | teacher_name | teacher_class | teacher_sex | teacher_age |
However, since the data tables with different data types are involved, a separate set of processing schemes needs to be designed for each data type when processing data, and the processing scheme corresponding to each data type can be generally referred to as a data processing model.
Fig. 1 is a schematic diagram of an existing data management scheme, and fig. 1 is a three-level schema structure of a database, which is divided into a view layer, a model layer and a data persistence layer. Wherein, the view layer can be understood as the layer where the data processing initiator is located; the model layer may be understood as the layer where the data processing model is located; the data persistence layer can be understood as a layer where data is stored; it can be seen from the illustration of fig. 1 that a separate data processing model needs to be designed for each data type and a separate management interface needs to be designed for each data type. For example, an account data processing model is designed corresponding to the account data, a business data processing model is designed corresponding to the business data, a transaction data processing model is designed corresponding to the transaction data, a product data processing model is designed corresponding to the product data, and a global data processing model is designed corresponding to the global data.
In fact, a data management scheme may involve more data types than those shown in FIG. 1; according to the design thought, each new data type needs to be designed, and as the related data types are more and more, the data processing models needing to be maintained are more and more, so that the data management system is overstaffed and the stability of the system is influenced; for example, data processing efficiency may be deteriorated, response time may be lengthened, and the like.
In order to solve the above problem, the present description as shown in fig. 2 proposes a general data management scheme; the same set of data processing models can be employed for any data type. Compared with the method in FIG. 1, the method is also divided in a three-level mode structure of the database; the difference is that the model layer is designed into a universal metadata model; the data tables corresponding to different data types are unified into a universal data table with a unified format in the data persistence layer, and the data persistence layer can also store configured metadata. Specifically, the metadata is used for establishing a mapping relation between a data field of target data to be processed and a data table field in a target data table. The metadata is data for describing data.
The configuration process of the metadata is described in detail below:
in one embodiment, the content configured in the metadata may include the following table 3:
TABLE 3
| model_id | Metadata identification |
| model_field | Data field |
| model_field_type | Type of data field |
| model_field_cols | Model field |
Table 3 may be a correspondence established between a data field and a model field.
Taking the above student data as an example, in the conventional student data table design, the data fields are the same as the data table fields, for example, 5 data fields of student _ ID (student status number), student _ name (student name), student _ class (student class), student _ sex (student sex), student _ age (student age) can be used; the conventional data sheet is shown in table 1 above;
in this specification, by configuring metadata, a corresponding relationship between each data field and a model field can be established.
For example, the following steps are carried out:
1. the correspondence between the student status number student _ ID and the model field is shown in table 4 below:
TABLE 4
| model_id | alm_student |
| model_field | student_ID |
| model_field_type | int |
| model_field_cols | k1 |
Table 4 may represent that a correspondence between the data field student _ ID and the model field k1 is established;
wherein the metadata identification (i.e., model _ id) is alm _ student; and alm _ student can also represent data table name, that is, target metadata (configuration content of all model _ id alm _ student) and target data table (data table with data table name alm _ student) can be determined according to metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 1; and k1 may also represent a data table field in the target data table alm _ student; namely, the data table fields of the target data table correspond to the model fields one by one;
the data field student _ ID is in corresponding relation with the model field k1, and the model field k1 is in one-to-one correspondence with the data table field of the target data table; therefore, there is a correspondence relationship between the data field student _ ID and the data table field k 1.
2. The correspondence between the student name student _ name and the model field is shown in table 5 below:
TABLE 5
Table 5 may represent that a correspondence between the data field student _ name and the model field k2 is established;
wherein the metadata identification (i.e., model _ id) is alm _ student; and alm _ student can also represent data table name, that is, target metadata (configuration content of all model _ id alm _ student) and target data table (data table with data table name alm _ student) can be determined according to metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 2; and k2 may also represent a data table field in the target data table alm _ student; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field student _ name has a corresponding relationship with the model field k2, and the model field k2 has a one-to-one correspondence with the data table fields of the target data table; therefore, there is a correspondence relationship between the data field student _ name and the data table field k 2.
3. The correspondence between student class student _ class and model fields is shown in Table 6 below:
TABLE 6
| model_id | alm_student |
| model_field | student_class |
| model_field_type | string |
| model_field_cols | k3 |
Table 6 may represent that a correspondence between the data field student _ class and the model field k3 is established;
wherein the metadata identification (i.e., model _ id) is alm _ student; and alm _ student can also represent data table name, that is, target metadata (configuration content of all model _ id alm _ student) and target data table (data table with data table name alm _ student) can be determined according to metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 3; and k3 may also represent a data table field in the target data table alm _ student; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field student _ class has a corresponding relationship with the model field k3, and the model field k3 has a one-to-one correspondence with the data table fields of the target data table; therefore, there is a correspondence between the data field student _ class and the data table field k 3.
4. The correspondence between the student gender student _ sex and the model field is shown in table 7 below:
TABLE 7
| model_id | alm_student |
| model_field | student_sex |
| model_field_type | string |
| model_field_cols | k4 |
Table 7 may represent that a correspondence relationship between the data field student _ sex and the model field k4 is established;
wherein the metadata identification (i.e., model _ id) is alm _ student; and alm _ student can also represent data table name, that is, target metadata (configuration content of all model _ id alm _ student) and target data table (data table with data table name alm _ student) can be determined according to metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 4; and k4 may also represent a data table field in the target data table alm _ student; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field student _ seq has a corresponding relationship with the model field k4, and the model field k4 has a one-to-one correspondence with the data table fields of the target data table; therefore, there is a correspondence relationship between the data field student _ sex and the data table field k 4.
5. The correspondence between the student age student _ age and the model field is shown in table 8 below:
TABLE 8
| model_id | alm_student |
| model_field | student_age |
| model_field_type | string |
| model_field_cols | k5 |
Table 8 may represent that a correspondence between the data field student _ age and the model field k5 is established;
wherein the metadata identification (i.e., model _ id) is alm _ student; and alm _ student can also represent data table name, that is, target metadata (configuration content of all model _ id alm _ student) and target data table (data table with data table name alm _ student) can be determined according to metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 5; and k5 may also represent a data table field in the target data table alm _ student; namely, the data table fields of the target data table correspond to the model fields one by one;
the data field student _ age and the model field k5 have a corresponding relationship, and the model field k5 and the data table field of the target data table have a one-to-one correspondence; therefore, there is a correspondence relationship between the data field student _ age and the data table field k 5.
In this specification, in the case of configuring the metadata in the above tables 4 to 8, it is also necessary to design a data table for storing student data, which may be identified as the metadata as shown above, namely named alm _ student, and the specific data table is shown in the following table 9:
TABLE 9
| k1 | k2 | k3 | k4 | k5 |
By comparing the data table 1 of the student data designed in the conventional manner with the data table 9 of the student data designed in the manner of this specification, it can be seen that the data table fields in the data table designed in the manner of this specification do not need to be the same as the data fields, but may be the same as the simulation fields defined in the metadata.
In an embodiment, the simulation fields defined in different metadata may be the same.
For example, taking the aforementioned teacher data as an example, suppose that the teacher data has data fields of teacher _ ID (teacher number), teacher _ name (teacher name), teacher _ class (teacher class), teacher _ six (teacher gender), teacher _ age (teacher age); the conventional data table is shown in table 2 above;
in this specification, by configuring metadata, a corresponding relationship between each data field and a model field can be established.
1. The correspondence between the teacher number teacher _ ID and the model field is shown in table 10 below:
watch 10
| model_id | alm_teacher |
| model_field | teacher_ID |
| model_field_type | int |
| model_field_cols | k1 |
Table 10 may represent that a correspondence between the data field teacher _ ID and the model field k1 is established;
wherein the metadata identification (i.e., model _ id) is alm _ teacher; and alm _ teacher can also represent the data table name, that is, the target metadata (all model _ id alm _ teacher configuration content) and the target data table (data table name alm _ teacher) can be determined according to the metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 1; and k1 may also represent a data table field in the target data table alm _ teacher; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field teacher _ ID has a corresponding relationship with the model field k1, and the model field k1 has a one-to-one correspondence with the data table fields of the target data table; thus, there is a correspondence between the data field teacher _ ID and the data table field k 1.
2. The correspondence between the teacher name teacher _ name and the model field is shown in table 11 below:
TABLE 11
| model_id | alm_teacher |
| model_field | teacher_name |
| model_field_type | string |
| model_field_cols | k2 |
Table 11 may represent that a correspondence between the data field teacher _ name and the model field k2 is established;
wherein the metadata identification (i.e., model _ id) is alm _ teacher; and alm _ teacher can also represent the data table name, that is, the target metadata (all model _ id alm _ teacher configuration content) and the target data table (data table name alm _ teacher) can be determined according to the metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 2; and k2 may also represent a data table field in the target data table alm _ teacher; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field teacher _ name has a corresponding relationship with the model field k2, and the model field k2 has a one-to-one correspondence with the data table fields of the target data table; thus, there is a correspondence between the data field teacher _ name and the data table field k 2.
3. The correspondence between the teacher number teacher _ class and the model field is shown in table 12 below:
TABLE 12
| model_id | alm_teacher |
| model_field | teacher_class |
| model_field_type | string |
| model_field_cols | k3 |
Table 12 may represent that a correspondence between the data field teacher _ class and the model field k3 is established;
wherein the metadata identification (i.e., model _ id) is alm _ teacher; and alm _ teacher can also represent the data table name, that is, the target metadata (all model _ id alm _ teacher configuration content) and the target data table (data table name alm _ teacher) can be determined according to the metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 3; and k3 may also represent a data table field in the target data table alm _ teacher; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field teacher _ class has a corresponding relationship with the model field k3, and the model field k3 has a one-to-one correspondence with the data table fields of the target data table; thus, there is a correspondence between the data field teacher _ class and the data table field k 3.
4. The correspondence between the teacher number teacher _ sex and the model field is shown in table 13 below:
watch 13
| model_id | alm_teacher |
| model_field | teacher_sex |
| model_field_type | string |
| model_field_cols | k4 |
Table 13 may represent that a correspondence between the data field teacher _ sex and the model field k4 is established;
wherein the metadata identification (i.e., model _ id) is alm _ teacher; and alm _ teacher can also represent the data table name, that is, the target metadata (all model _ id alm _ teacher configuration content) and the target data table (data table name alm _ teacher) can be determined according to the metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 4; and k4 may also represent a data table field in the target data table alm _ teacher; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field teacher _ sex has a corresponding relationship with the model field k4, and the model field k4 has a one-to-one correspondence with the data table fields of the target data table; thus, there is a correspondence between the data field teacher _ sex and the data table field k 4.
5. The correspondence between the teacher number teacher _ age and the model field is shown in table 14 below:
TABLE 14
| model_id | alm_teacher |
| model_field | teacher_age |
| model_field_type | string |
| model_field_cols | K5 |
Table 14 may represent that a correspondence between the data field teacher _ age and the model field k5 is established;
wherein the metadata identification (i.e., model _ id) is alm _ teacher; and alm _ teacher can also represent the data table name, that is, the target metadata (all model _ id alm _ teacher configuration content) and the target data table (data table name alm _ teacher) can be determined according to the metadata identification;
wherein the model field (i.e., model _ field _ cols) is k 5; and k5 may also represent a data table field in the target data table alm _ teacher; namely, the data table fields of the target data table correspond to the model fields one by one;
because the data field teacher _ age has a corresponding relationship with the model field k5, and the model field k5 has a one-to-one correspondence with the data table fields of the target data table; thus, there is a correspondence between the data field teacher _ age and the data table field k 5.
In this specification, in the case of configuring the metadata of the above tables 9 to 14, it is also necessary to design a data table for storing teacher data, which may be identified as the metadata as shown above, namely, named alm _ teacher, and the specific data table is shown in the following table 15:
watch 15
| k1 | k2 | k3 | k4 | k5 |
Comparing a data table 9 of student data and a data table 15 of teacher data which are designed in the mode of the specification; it can be seen that the data table fields in the data table designed in the manner of this specification may all be the same, and the corresponding data field may be determined by querying the metadata using the metadata identifier (i.e., the data table name) + the data table field.
For example, the same data table field k 1; for alm _ student and k1, corresponding metadata, i.e., tables 4 to 8, can be queried through the metadata identifier alm _ student; since table 4 exists in the model field k1, the data field corresponding to the model field k1 can be determined to be student _ ID, i.e. student's number, according to the contents of table 4.
For alm _ teacher and k1, corresponding metadata, i.e., table 10 to table 14, can be queried through the metadata identifier alm _ teacher; since table 10 exists for the model field k1, the data field corresponding to the model field k1 can be determined to be teacher _ ID, i.e., teacher number, according to the contents of table 10.
In summary, the present specification provides a general data management scheme, and for data of different data types, the same set of data processing model, that is, a processing model based on metadata, may be adopted; the mapping relation between the data fields and the data table fields is established through the metadata, and the data type of the data does not need to be concerned during data operation.
On the other hand, when the data table is established, the data fields specifically existing for the data of the data type do not need to be concerned; the data sheet design has universality, and the same data sheet field, namely the model field, can be adopted when different data sheet designs are carried out. The data table fields in each data table may be the same (the model fields are also similar), and may include, for example, 20 fields k1, k2, k3, …, k 20. In some embodiments, these fields may be divided into index fields and normal fields; wherein, the index field can be denoted as k, and the common field can be denoted as c; e.g., 8 index fields and 12 normal fields, k1, k2, k3, …, k 8; c1, c2, c3, … and c 12.
An embodiment of a data management method according to the present disclosure may be described below with reference to an example shown in fig. 3, where the method may be applied to a server for managing data (hereinafter, referred to as a server), and the method may include the following steps:
step 110: receiving a data processing request; the data processing request carries a processing field and a metadata identifier, the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data.
In an embodiment, the data may include flowability data. Liquidity is an indicator of how easily an asset can be exchanged into an exchange medium. Currency is a liquidity asset, as it can be an exchange medium in the economy. In money-related transactions, such as financial transactions, there is often a need to manage flow data.
Step 120: determining corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relationship between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one.
In an embodiment, the step 120 determines, according to the metadata identifier, corresponding target metadata and a target data table, and specifically includes:
inquiring target metadata corresponding to the metadata identification;
inquiring a target data table corresponding to the metadata identification in the target metadata
Step 130: and determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata.
Step 140: and processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
According to the data management scheme provided by the embodiment, the mapping relation between the data field of the target data to be processed and the data table field in the target data table is established by using the metadata. In one aspect, the same set of data processing models can be employed for any data type. On the other hand, when the data table is established, the data fields specifically existing for the data of the data type do not need to be concerned; the data sheet design has universality, and the same data sheet field, namely the model field, can be adopted when different data sheet designs are carried out.
In practical application, the processing types of the data processing request can be generally divided into data storage, data query, data deletion, data modification and the like; each process type is described separately below:
in one embodiment, where the processing type is data storage,
the step 140 may specifically include:
and storing the target data into a column of the target data table field of the target data table.
For example, taking the student data table shown in table 9 as an example, it is assumed that the processing fields and the target data carried by the data processing request received by the server are as follows:
student_ID：010；
student _ name: zhang III;
student _ class: four shifts in three years;
student _ sex: male;
student_age：18；
and the carried metadata is identified as alm _ student;
firstly, according to the metadata identifier alm _ student, it can be determined that the corresponding target metadata is table 4 to table 8, and the target data table is table 9;
then, according to the target metadata table 4, it can be determined that the target model field corresponding to the processing field student _ ID is k1, and the target data table field k 1;
according to the target metadata table 5, the target model field corresponding to the processing field student _ name can be determined to be k2, and the target data table field k 2;
according to the target metadata table 6, the target model field corresponding to the processing field student _ class can be determined to be k3, and the target data table field k 3;
according to the target metadata table 7, it can be determined that the target model field corresponding to the processing field student _ sex is k4, and the target data table field k 4;
according to the target metadata table 8, the target model field corresponding to the processing field student _ age can be determined to be k5, and the target data table field k 5;
finally, since the processing type is data storage, the target data may be stored in the column where the target data table field in the target data table 9 is located, as in table 16 below:
TABLE 16
| k1 | k2 | k3 | k4 | k5 |
| 010 | Zhang San | Three years and four shifts | For male | 18 |
Thus, a new piece of student data is inserted into the student data table shown in the original table 9.
By the embodiment, based on the metadata, a unified metadata processing model can be adopted for data storage of different data types.
In an embodiment, in order to deal with storage of mass data, a database-dividing and table-dividing mode can be adopted to ensure storage efficiency during mass data storage. As previously mentioned, there are several index fields in the metadata, and the sub-repository sub-table is based on these index fields. Assuming that there are 8 index fields, the calculation formula of the sub-database and sub-table is as follows:
dbIndex＝hash(k1,k2,…,k8)％10
tableIndex＝hash(k1,k2,…,k8)％100
by means of hashing, data is dispersed in 10 libraries of 100 tables, and therefore a larger amount of data storage can be supported on the premise that storage efficiency is not reduced.
In an embodiment, when the processing type is data query, the data processing request further carries a query rule;
the step 140 may specifically include:
and inquiring the data which accords with the inquiry rule in the column where the target data table field of the target data table is located.
The query rule may refer to determining a range of data to be queried; such as the student data for school roll number 001-.
By the embodiment, based on the metadata, a unified metadata processing model can be adopted for different data types to perform data query.
In an embodiment, the method further comprises:
determining a data field corresponding to the inquired data according to the target metadata;
and assembling the inquired data and the corresponding data field.
In this embodiment, because the field of the queried data in the data table is a model field, rather than an actual data field, the model field of the queried data needs to be converted into a data field according to the correspondence between the data field and the model field recorded in the metadata. The assembling may be understood as translating a data field into a meaning of the data field that may be understood by a user.
For example, suppose a piece of student data is queried as follows:
k1：010；
k 2: zhang III;
k 3: four shifts in three years;
k 4: male;
k5：18；
first, the model field needs to be converted into the data field, and the inverse process of the foregoing data storage embodiment can be referred to obtain the student data of the data field as follows:
student_ID：010；
student _ name: zhang III;
student _ class: four shifts in three years;
student _ sex: male;
student_age：18；
however, assembled as student data that the user can understand:
the school roll number: 010; the name of the student: zhang III; the student class: four shifts in three years; the gender of the students: male; the age of the student: 18.
in one embodiment, when the processing type is data deletion,
the step 140 may specifically include:
and deleting the target data from the column of the target data table field of the target data table.
For example, taking the student data table shown in table 16 as an example, it is assumed that the processing fields and the target data carried by the data processing request received by the server are as follows:
student_ID：010；
and the carried metadata is identified as alm _ student;
firstly, according to the metadata identifier alm _ student, it can be determined that the corresponding target metadata is table 4 to table 8, and the target data table is table 16;
then, since there is a target metadata table of the process field student _ ID as table 4, it can be determined that the target model field corresponding to the process field student _ ID is k1, and the target data table field k 1;
finally, since the processing type is data deletion, the row in which the data 010 is located can be deleted in the column in which the target data table field k1 is located in the target data table 16, thereby changing back to table 9.
By the embodiment, based on the metadata, the data can be deleted by adopting a uniform metadata processing model for different data types.
In one embodiment, when the processing type is data modification, the target data comprises pre-modification data and post-modification data;
the step 140 may specifically include:
and modifying the data before modification in the column where the target data table field of the target data table is located into the modified data.
By the embodiment, based on the metadata, the data modification can be carried out on different data types by adopting a unified metadata processing model.
Corresponding to the embodiment of the management method of the data, the specification also provides an embodiment of a management device of the data. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer business program instructions in the nonvolatile memory into the memory for operation through the processor of the device in which the device is located. In terms of hardware, as shown in fig. 4, the hardware structure diagram of the device where the management apparatus for data in this specification is located is shown, except for the processor, the network interface, the memory, and the nonvolatile memory shown in fig. 4, the device where the apparatus is located in the embodiment may also include other hardware according to the actual management function of the data, which is not described again.
Referring to fig. 5, a block diagram of an apparatus for managing data according to an embodiment of the present disclosure is shown, where the apparatus corresponds to the embodiment shown in fig. 3, and the apparatus includes:
a receiving unit 210 that receives a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
a first determining unit 220, which determines corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
a second determining unit 230, configured to determine, according to the target metadata, a target model field corresponding to the processing field and a target data table field corresponding to the target model field;
the processing unit 240 processes the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
In an alternative embodiment:
the data includes flowability data.
In an alternative embodiment: different metadata has the same model field.
In an alternative embodiment:
the first determining unit 220 specifically includes:
the first inquiry subunit inquires the target metadata corresponding to the metadata identifier;
and the second query subunit queries a target data table corresponding to the metadata identifier in the target metadata.
In an alternative embodiment:
when the processing type is data storage, the processing unit 240 specifically includes:
and storing the target data into a column of the target data table field of the target data table.
In an alternative embodiment:
when the processing type is data query, the data processing request also carries query rules;
the processing unit 240 specifically includes:
and inquiring the data which accords with the inquiry rule in the column where the target data table field of the target data table is located.
In an alternative embodiment:
the device further comprises:
the first determining subunit determines a data field corresponding to the queried data according to the target metadata;
and the assembly subunit is used for assembling the inquired data and the corresponding data field.
In an alternative embodiment:
when the processing type is data deletion, the processing unit 240 specifically includes:
and deleting the target data from the column of the target data table field of the target data table.
In an alternative embodiment:
when the processing type is data modification, the target data comprises data before modification and data after modification;
the processing unit 240 specifically includes:
and modifying the data before modification in the column where the target data table field of the target data table is located into the modified data.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement it without inventive effort.
Fig. 5 above describes the internal functional modules and the structural schematic of the data management device, and the actual execution subject may be an electronic device, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to:
receiving a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
determining corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata;
and processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
Optionally, the data comprises flowability data.
Optionally, different metadata has the same model field.
Optionally, the determining the corresponding target metadata and the target data table according to the metadata identifier specifically includes:
inquiring target metadata corresponding to the metadata identification;
and inquiring a target data table corresponding to the metadata identification in the target metadata.
Optionally, when the processing type is data storage,
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and storing the target data into a column of the target data table field of the target data table.
Optionally, when the processing type is data query, the data processing request further carries a query rule;
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and inquiring the data which accords with the inquiry rule in the column where the target data table field of the target data table is located.
Optionally, the method further includes:
determining a data field corresponding to the inquired data according to the target metadata;
and assembling the inquired data and the corresponding data field.
Optionally, when the processing type is data deletion,
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and deleting the target data from the column of the target data table field of the target data table.
Optionally, when the processing type is data modification, the target data includes data before modification and data after modification;
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and modifying the data before modification in the column where the target data table field of the target data table is located into the modified data.
In the above embodiments of the electronic device, it should be understood that the Processor may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, and the aforementioned memory may be a read-only memory (ROM), a Random Access Memory (RAM), a flash memory, a hard disk, or a solid state disk. The steps of a method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiment of the electronic device, since it is substantially similar to the embodiment of the method, the description is simple, and for the relevant points, reference may be made to part of the description of the embodiment of the method.
Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This specification is intended to cover any variations, uses, or adaptations of the specification following, in general, the principles of the specification and including such departures from the present disclosure as come within known or customary practice within the art to which the specification pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the specification being indicated by the following claims.
It will be understood that the present description is not limited to the precise arrangements described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.
Claims (11)
1. A method of managing data, the method comprising:
receiving a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
determining corresponding target metadata and a target data table according to the metadata identifier; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata;
and processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
2. The method of claim 1, the data comprising flowability data.
3. The method of claim 1, different metadata having the same model field.
4. The method according to claim 1, wherein determining the corresponding target metadata and the target data table according to the metadata identifier specifically includes:
inquiring target metadata corresponding to the metadata identification;
and inquiring a target data table corresponding to the metadata identification in the target metadata.
5. The method of claim 1, wherein, when the processing type is data storage,
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and storing the target data into a column of the target data table field of the target data table.
6. The method of claim 1, wherein when the processing type is data query, the data processing request further carries query rules;
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and inquiring the data which accords with the inquiry rule in the column where the target data table field of the target data table is located.
7. The method of claim 6, further comprising:
determining a data field corresponding to the inquired data according to the target metadata;
and assembling the inquired data and the corresponding data field.
8. The method of claim 1, wherein, when the type of processing is data deletion,
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and deleting the target data from the column of the target data table field of the target data table.
9. The method of claim 1, when the processing type is data modification, the target data comprises pre-modification data and post-modification data;
the processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request specifically includes:
and modifying the data before modification in the column where the target data table field of the target data table is located into the modified data.
10. An apparatus for managing data, the apparatus comprising:
a receiving unit that receives a data processing request; the data processing request carries a processing field and a metadata identifier, wherein the processing field is a data field corresponding to target data to be processed, and the metadata identifier corresponds to the data type of the target data;
the first determining unit is used for determining corresponding target metadata and a target data table according to the metadata identification; the target metadata is preconfigured with a corresponding relation between data fields and model fields, the target data table is used for storing data of the data types, and the data table fields of the target data table correspond to the model fields one by one;
the second determining unit is used for determining a target model field corresponding to the processing field and a target data table field corresponding to the target model field according to the target metadata;
and the processing unit is used for processing the target data in the column where the target data table field of the target data table is located according to the processing type of the data processing request.
11. An electronic device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured as the method of any of the preceding claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810535427.XA CN108959381B (en) | 2018-05-29 | 2018-05-29 | Data management method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810535427.XA CN108959381B (en) | 2018-05-29 | 2018-05-29 | Data management method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108959381A CN108959381A (en) | 2018-12-07 |
| CN108959381B true CN108959381B (en) | 2021-05-28 |
Family
ID=64492514
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810535427.XA Active CN108959381B (en) | 2018-05-29 | 2018-05-29 | Data management method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108959381B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112486981A (en) * | 2019-09-12 | 2021-03-12 | 阿里巴巴集团控股有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN113568696A (en) * | 2021-07-29 | 2021-10-29 | 成都卓码数智数字科技有限公司 | Method for defining service data model through interface operation in digital transformation |
| CN113590593B (en) * | 2021-08-04 | 2024-09-24 | 浙江大华技术股份有限公司 | Data table information generation method and device, storage medium and electronic device |
| CN114356933A (en) * | 2022-01-04 | 2022-04-15 | 执中数据科技(苏州)有限责任公司 | Enterprise data management method and device based on metadata |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106547786A (en) * | 2015-09-22 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of date storage method and device |
| CN106708891A (en) * | 2015-11-17 | 2017-05-24 | 中兴通讯股份有限公司 | Network management data synchronizing method and device |
| CN107391739A (en) * | 2017-08-07 | 2017-11-24 | 北京奇艺世纪科技有限公司 | A kind of query statement generation method, device and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9898493B2 (en) * | 2014-10-17 | 2018-02-20 | Sap Se | Runtime generation of a mapping table for uploading data into structured data marts |
-
2018
- 2018-05-29 CN CN201810535427.XA patent/CN108959381B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106547786A (en) * | 2015-09-22 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of date storage method and device |
| CN106708891A (en) * | 2015-11-17 | 2017-05-24 | 中兴通讯股份有限公司 | Network management data synchronizing method and device |
| CN107391739A (en) * | 2017-08-07 | 2017-11-24 | 北京奇艺世纪科技有限公司 | A kind of query statement generation method, device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| Real-Time Data Stream Management System for Large Volume of RFID Events;So Young Choi et al.;《Institute of Electrical and Electronics Engineers》;20080909;全文 * |
| 基于元数据驱动的通用数据迁移工具;任庆东 等;《东北石油大学学报》;20110228(第1期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108959381A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108959381B (en) | Data management method and device and electronic equipment | |
| US10713654B2 (en) | Enterprise blockchains and transactional systems | |
| CN106202548B (en) | Date storage method, lookup method and device | |
| WO2019024496A1 (en) | Enterprise recommendation method and application server | |
| CN111352902A (en) | Log processing method and device, terminal equipment and storage medium | |
| US20180144061A1 (en) | Edge store designs for graph databases | |
| CN111125229A (en) | Data blood margin generation method and device and electronic equipment | |
| US10445370B2 (en) | Compound indexes for graph databases | |
| WO2019100853A1 (en) | Page query method, device, and electronic apparatus | |
| CN109656986A (en) | A kind of householder method that business datum summarizes, device and electronic equipment | |
| CN112597168A (en) | Processing method, device and platform of multi-source customer data and storage medium | |
| CN107735781B (en) | Method and device for storing query result and computing equipment | |
| CN111506613A (en) | Method, system, device and equipment for querying incidence relation of data record | |
| CN112328575A (en) | Data asset blood margin generation method and device and electronic equipment | |
| CN112527900A (en) | Method, device, equipment and medium for database multi-copy reading consistency | |
| US20180144060A1 (en) | Processing deleted edges in graph databases | |
| CN116483829A (en) | Data query method, device, computer equipment and storage medium | |
| CN112069164B (en) | Data query method, device, electronic equipment and computer readable storage medium | |
| CN112632115A (en) | BI-based data query method and system | |
| CN111414364A (en) | User information generation method and device and electronic equipment | |
| CN110597849A (en) | Data query method and device | |
| WO2020139282A1 (en) | A data comparison system | |
| CN110674363B (en) | Similarity matching method and device between interface services and electronic equipment | |
| CN113642309B (en) | Data comparison method and related device | |
| JP2020035332A (en) | Type estimation method, information processing apparatus and type estimation program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200922 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200922 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |