CN113779016A

CN113779016A - Data management method and device

Info

Publication number: CN113779016A
Application number: CN202010641101.2A
Authority: CN
Inventors: 李秀海
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2020-07-06
Filing date: 2020-07-06
Publication date: 2021-12-10

Abstract

The invention discloses a method and a device for data management, and relates to the technical field of computers. One embodiment of the method comprises: when writing service data into a service system, acquiring service dependent data corresponding to the service data; and starting a data writing transaction, storing the service dependent data into a data storage table of the service system, and then closing the data writing transaction. The service dependent data and the service data are written into the same database by adopting data writing transaction, so that the atomicity of the writing operation of the service dependent data and the service data can be ensured, and the abnormal service flow in a service system caused by the inconsistency of the service dependent data and the service data is avoided; the data model to which the service dependent data in the service system belongs and the data storage object are associated together through the attribute mapping table, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized, the development cost of the system is reduced, and the componentization is realized.

Description

Data management method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for data management.

Background

The business system needs a large number of tables to store the business dependent data such as system degraded data, key log information and the like, and the business data related to the business process. At present, service dependent data and service data of a service system are generally stored respectively, and atomicity of write-in operation of the service dependent data and the service data cannot be guaranteed, so that an abnormal service flow occurs in the service system.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method and an apparatus for data management, where service dependent data and service data are written into a same database, so that atomicity of write operations of the service dependent data and the service data can be ensured, and occurrence of an abnormal situation in a service flow in a service system due to inconsistency between the service dependent data and the service data is avoided; the data model to which the service dependent data in the service system belongs and the data storage object are associated together through the attribute mapping table of the service system, and no matter how many data models exist in the service system, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized, the development cost of the system is reduced, and the componentization is realized.

To achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a data management method including:

when writing service data into a service system, acquiring service dependent data corresponding to the service data;

starting a data writing transaction, placing an operation of writing service data and an operation of writing service dependent data in the transaction, and storing the service dependent data in a data storage table of the service system;

closing the data write transaction.

Optionally, storing the service dependent data in a data storage table of the service system includes:

acquiring an attribute record set of a data model to which service dependent data belongs from an attribute mapping table of a service system; the set of attribute records includes an attribute record for each model field in the data model;

constructing a data storage object instance according to the attribute record set and the service dependent data; the data structure of the data storage object instance is consistent with the data structure of a data storage table of the service system;

and writing the data storage object instance into the data storage table.

Optionally, the attribute record comprises: (ii) a model attribute;

acquiring an attribute record set of a data model to which service dependent data belongs from an attribute mapping table of a service system, wherein the attribute record set comprises: and screening the attribute values of the model attributes from the attribute mapping table as the attribute records of the data model to obtain the attribute record set.

Optionally, the attribute record comprises: mapping the attributes; the mapping attribute represents the mapping relation between a model field of a data model and a storage field in the data storage table;

constructing a data storage object instance according to the attribute record set and the service dependency data, including:

and circularly traversing each attribute record in the attribute record set: obtaining a field value of a model field corresponding to the attribute record from the service dependent data, and assigning the field value to a storage field corresponding to the attribute record according to the mapping attribute of the attribute record to obtain a configuration data object;

and taking a set of configuration data objects corresponding to all attribute records in the attribute record set as the data storage object instance.

Optionally, the attribute record further comprises: storing the attribute; the storage attribute represents whether the field value of the model field corresponding to the attribute record needs to be written into the data storage table or not;

circularly traversing each attribute record in the attribute record set, comprising:

before the field value of the model field corresponding to the attribute record is obtained from the business dependent data, the field value is confirmed to be written into the data storage table according to the storage attribute, and the field value exists in the business dependent data.

Optionally, the attribute record further comprises: a first identification attribute and an index attribute; the data storage table comprises an index field; the index attribute represents whether the first identification attribute participates in composing the index field;

constructing a data storage object instance according to the attribute record set and the service dependency data, and further comprising: acquiring attribute values of all first identification attributes which participate in the index field in the attribute record set according to the index attribute of each attribute record; and generating the field value of the index field according to the attribute values of all the first identification attributes.

Optionally, the method of the embodiment of the present invention further includes:

acquiring an attribute record set of a target data model from the attribute mapping table;

generating an index field instance according to the attribute values of all first identification attributes participating in forming an index field in the attribute record set of the target data model, and constructing a query object according to the index field instance and the model identification of the target data model; the data structure of the query object is consistent with the data structure of the data storage table;

acquiring a data storage record corresponding to the query object from the data storage table;

and constructing a search data instance according to the data storage record and the attribute record set of the target data model and returning.

Optionally, the attribute record further comprises: a second identification attribute and a search attribute; the index attribute represents whether a second identification attribute can be searched;

before generating an index field instance according to the attribute values of all the first identification attributes in the attribute record set of the target data model, the method further includes: confirming that the attribute values of all first identification attributes participating in the index field in the attribute record set of the target data model are not null; and

if the attribute value of any one first identification attribute participating in the index field in the attribute record set of the target data model is null, acquiring the attribute values of all second identification attributes and the attribute values corresponding to the mapping attributes, which can be searched in the attribute record set of the target data model, according to the search attribute of each attribute record of the target data model, and constructing a query object according to the model identification of the target data model, the attribute values of all second identification attributes and the attribute values corresponding to the mapping attributes.

Optionally, the model information of the data model is stored in a preset model table.

According to a second aspect of the embodiments of the present invention, there is provided an apparatus for data management, including:

the data acquisition module is used for acquiring service dependent data corresponding to the service data when the service data is written into the service system;

and the data writing module is used for starting a data writing transaction, placing the operation of writing in the service data and the operation of writing in the service dependent data in the transaction, storing the service dependent data in a data storage table of the service system, and then closing the data writing transaction.

Optionally, the data writing module includes:

the attribute acquisition module is used for acquiring an attribute record set of a data model to which the service dependent data belongs from an attribute mapping table of the service system; the set of attribute records includes an attribute record for each model field in the data model;

the data conversion module is used for constructing a data storage object instance according to the attribute record set and the service dependent data; the data structure of the data storage object instance is consistent with the data structure of a data storage table of the service system;

and the data storage module writes the data storage object instance into the data storage table.

Optionally, the attribute record comprises: (ii) a model attribute;

the attribute obtaining module obtains an attribute record set of a data model to which the service dependent data belongs from an attribute mapping table of the service system, and the attribute record set comprises: and screening the attribute values of the model attributes from the attribute mapping table as the attribute records of the data model to obtain the attribute record set.

the data conversion module constructs a data storage object instance according to the attribute record set and the service dependent data, and the method comprises the following steps:

the data conversion module circularly traverses each attribute record in the attribute record set, and the method comprises the following steps: before the field value of the model field corresponding to the attribute record is obtained from the business dependent data, the field value is confirmed to be written into the data storage table according to the storage attribute, and the field value exists in the business dependent data.

the data conversion module is further configured to: acquiring attribute values of all first identification attributes which participate in the index field in the attribute record set according to the index attribute of each attribute record; and generating the field value of the index field according to the attribute values of all the first identification attributes.

Optionally, the attribute obtaining module is further configured to: acquiring an attribute record set of a target data model from the attribute mapping table;

the data conversion module is further configured to: generating an index field instance according to the attribute values of all first identification attributes participating in forming an index field in the attribute record set of the target data model, and constructing a query object according to the index field instance and the model identification of the target data model; the data structure of the query object is consistent with the data structure of the data storage table;

the data storage module is also used for acquiring a data storage record corresponding to the query object from the data storage table;

the data conversion module is further configured to: and constructing a search data instance according to the data storage record and the attribute record set of the target data model and returning.

the data conversion module is further configured to: before generating an index field instance according to the attribute values of all first identification attributes participating in the index field in the attribute record set of the target data model, confirming that the attribute values of all first identification attributes participating in the index field in the attribute record set of the target data model are not null; and

According to a third aspect of embodiments of the present invention, there is provided an electronic device for data management, including:

one or more processors;

a storage device for storing one or more programs,

when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method provided by the first aspect of the embodiments of the present invention.

According to a fourth aspect of embodiments of the present invention, there is provided a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the method provided by the first aspect of embodiments of the present invention.

One embodiment of the above invention has the following advantages or benefits: by writing the service dependent data and the service data into the same database, the atomicity of the writing operation of the service dependent data and the service data can be ensured, and the occurrence of abnormal service flow in a service system caused by inconsistency of the service dependent data and the service data is avoided; the data model to which the service dependent data in the service system belongs and the data storage object are associated together through the attribute mapping table of the service system, and no matter how many data models exist in the service system, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized, the development cost of the system is reduced, and the componentization is realized.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of the main flow of a method of data management of an embodiment of the present invention;

FIG. 2 is a schematic diagram of the main flow of storing service dependent data in an alternative embodiment of the present invention;

FIG. 3 is a schematic diagram of the main modules of an apparatus for data management of an embodiment of the present invention;

FIG. 4 is a schematic diagram of the main components of a data write module in an alternative embodiment of the invention;

FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

According to an aspect of an embodiment of the present invention, a method of data management is provided.

Fig. 1 is a schematic diagram of a main flow of a data management method according to an embodiment of the present invention, and as shown in fig. 1, the data management method includes:

step S101, when writing service data into a service system, acquiring service dependent data corresponding to the service data;

step S102, starting a data writing transaction, placing an operation of writing in service data and an operation of writing in service dependent data in the transaction, and storing the service dependent data in a data storage table of the service system;

and step S103, closing the data writing transaction.

The business data refers to data related to business processes of the business system, such as order status (ordering success, ordering failure, etc.), logistics progress, and the like. The business dependent data refers to data which is dependent on business processes of the business system, such as system degradation data, key log information related to order transaction links and the like. For example, in a link of calling a proxy interface to produce an air ticket, since the amount of the air ticket is generally large, when the proxy interface explicitly returns error information (the error information is the service dependent data mentioned in the embodiment of the present invention) that can be refunded, the air ticket system stores the error information returned by the proxy interface into a database, and then updates the order state to a refunded state to refund the air ticket. Here, saving error information and changing order states can guarantee atomicity of two operations in one transaction operation. By writing the service dependent data and the service data into the same database (namely the database of the service system) by adopting the same data writing transaction, the atomicity of the writing operation of the service dependent data and the service data can be ensured, and the condition that the service flow in the service system is abnormal due to the inconsistency of the service dependent data and the service data is avoided.

Fig. 2 is a schematic diagram of a main flow of storing business dependency data in an alternative embodiment of the present invention, and as shown in fig. 2, the main flow of storing business dependency data includes: step S201, step S202, and step S203. In the embodiment of the invention, an attribute mapping table and a data storage table are preset in the service system, and service dependent data of the service system are stored in a database of the service data system after data structure conversion is carried out according to the attribute mapping table. And storing the data into a database of the service system, and ensuring the atomicity of the write-in operation of the service dependent data and the service data through the characteristics of the database. The database may use a relational database, such as MySQL (a relational database management system).

In step S201, an attribute record set of a data model to which the service dependent data belongs is obtained from an attribute mapping table of the service system; the set of attribute records includes an attribute record for each model field in the data model.

The data models, i.e. data types, have the same data type for the service dependent data belonging to the same data model. The division method of the data model or the data type can be selectively set according to the actual situation. For example, the business dependent data is divided into a system degraded data model, a business critical log information data model, and the like according to its function. As another example, business dependent data is divided into data models that depend on different programs based on its source.

In the actual application process, a model table may be preset in the business system, and model information of the data model, such as model code, model name, description information, etc., may be stored in the model table. The names and the number of the fields in the model table can be selectively set according to actual conditions. Table 1 shows an example of a model table data structure, and the model table structure shown in table 1 includes three fields of a primary key (ID), a model Name (Name), and description information (Desc). In table 1, Long represents the Long integer data, and String represents the String data.

TABLE 1 model Table Structure

Name of field	Type (B)	Remarks for note
			ID	Long	Main key
Name	String	Model name
			Desc	String	Description information

Taking key error log information (KeyErrorLog) as an example of a data model, table 2 shows an example of the data model.

TABLE 2 data model

ID	Name	Desc
			1	KeyErrorLog	Key error log information

And the unified model table is adopted to store the relevant information of each data model in the service system, so that each data model and the relevant information thereof in the service system can be conveniently and visually checked. A service system usually includes a plurality of subsystems, and for example, in the e-commerce field, a service system of an internet e-commerce company may have hundreds of subsystems supporting, such as a charging system, an air ticket selling system, and the like. In a complete business system, each table is created, and corresponding query statements, entity class structures are defined, reading and writing methods are written, and the like. Except that the names of the table fields and the number of the fields are different among the subsystems, other logics are basically the same. The same function is developed once in each subsystem, so that repeated construction is realized, the cost is wasted, and the componentization principle is not met. The model table in the embodiment has universality, can be suitable for each subsystem in a service system, can avoid repeated construction, reduces development cost and realizes componentization.

The attribute mapping table is used to define the attributes of the various fields in the data model. Each row in the attribute mapping table corresponds to a field of the data model, and each row in the attribute mapping table comprises a plurality of fields, each field representing an attribute of the corresponding model field. A data model typically has multiple fields, each field having an attribute record in an attribute mapping table. The attribute records define attributes of the corresponding field, such as which data model the field belongs to, whether it can be searched, whether it participates in constructing index fields when writing and querying data, and so on. Optionally, the attribute record comprises: and (6) model attributes. Acquiring an attribute record set of a data model to which service dependent data belongs from an attribute mapping table of a service system, wherein the attribute record set comprises: and screening the attribute value of the model attribute from the attribute mapping table as the attribute record of the data model to obtain an attribute record set. By setting the model attribute field in the attribute mapping table, the attribute record set of the data model can be determined conveniently and rapidly.

Table 3 shows an example of a data structure of the attribute Mapping table, and the Model table structure shown in table 3 includes fields of an identification attribute (such as a primary key (ID), an attribute name (Attr _ name), a constraint attribute (Uuid)), a Model attribute (Model _ name), an index attribute (Is _ index), a search attribute (Is _ search), a Mapping attribute (Mapping _ column), and a storage attribute (Is _ request). In table 3, Long represents Long integer data, String represents String data, and Int represents integer data.

Table 3 attribute mapping table structure

Taking key error log information (KeyErrorLog) as an example of a data model, table 4 shows an attribute record set corresponding to the data model.

Table 4 collection of attribute records

The attribute mapping table can associate the data model to which the service dependent data in the service system belongs and the data storage object together, and can convert the data model into the data storage object with a data structure through the attribute mapping table regardless of the data structure of the data model, so that the data storage object is stored in one data storage table. A service system usually includes a plurality of subsystems, and for example, in the e-commerce field, a service system of an internet e-commerce company may have hundreds of subsystems supporting, such as a charging system, an air ticket selling system, and the like. In a complete business system, each time a data storage table is created, corresponding query statements, entity class structures are defined, reading and writing methods are written, and the like. Except that the names of the table fields and the number of the fields are different among the subsystems, other logics are basically the same. The same function is developed once in each subsystem, so that repeated construction is realized, the cost is wasted, and the componentization principle is not met. By adopting the uniform attribute mapping table, on one hand, the service dependent data and the service data can be written into the same database, the atomicity of the writing operation of the service dependent data and the service data is ensured, and the condition that the service flow in the service system is abnormal due to the inconsistency of the service dependent data and the service data is avoided. And on the other hand, the attribute mapping table has universality, can be suitable for each subsystem in a service system, can avoid repeated construction, reduces development cost and realizes componentization.

In step S102, a data storage object instance is constructed according to the attribute record set and the service dependent data; and the data structure of the data storage object instance is consistent with the data structure of the data storage table of the service system. In step S103, the data storage object instance is written into the data storage table.

A data storage table is preset in the service system, has universality and can be suitable for each subsystem in the service system. Table 5 shows an example of a data storage table data structure, and the data storage table structure shown in table 5 includes fields such as a primary key (ID), a unique constraint (Uuid), an attribute of the Model (Model _ name), and a dynamic Field (e.g., Field _1, Field _2, … …, Field _50 in table 5). The name and the number of the fields in the data storage table and the name and the number of the dynamic fields can be selectively set according to actual conditions. A unique constraint (Uuid) field in the data storage table is used to perform anti-re-constraint. In table 5, Long represents the Long integer data, and String represents the String data. Each row in the data storage table corresponds to a field of the data model (for convenience of description, a field in the data model is referred to as a model field, and a field in the data storage table is referred to as a storage field), and each row in the data storage table includes a plurality of storage fields. Each row in the attribute mapping table (each row representing an attribute record) corresponds to a storage field in the data storage table.

TABLE 5 data storage Table Structure

Name of field	Type (B)	Remarks for note
			ID	Long	Main key
Model_name	String	Belonged model
			Uuid	String	Unique constraint
Field_1	String	Dynamic field
			Field_2	String	Dynamic field
Field_3	String	Dynamic field
			……	……	……
Field_50	String	Dynamic field

And constructing a data storage object instance according to the attribute record set and the service dependent data, namely assigning the field value of the related field in the service dependent data to the model field corresponding to each attribute record in the attribute record set. And the attribute record set is a data storage object, the field value of the relevant field in the service dependent data is assigned to the field corresponding to each attribute record in the attribute record set, and the assigned data storage object is the data storage object instance. Because the data structure of the data storage object is consistent with the data structure of the data storage table, the data structure of the data storage object instance is also consistent with the data structure of the data storage table. The data model to which the service dependent data in the service system belongs and the data storage object are associated together through the attribute mapping table of the service system, and no matter how many data models exist in the service system, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized, the development cost of the system is reduced, and the componentization is realized.

Optionally, the attribute record comprises: and Mapping attributes, wherein the Mapping attributes represent the Mapping relationship between the model fields of the data model and the storage fields in the data storage table, and refer to Mapping _ column fields in table 3. Constructing a data storage object instance according to the attribute record set and the service dependent data, wherein the method comprises the following steps: and circularly traversing each attribute record in the attribute record set: obtaining the field value of the model field corresponding to the attribute record from the service dependent data, and assigning the field value to the storage field corresponding to the attribute record according to the mapping attribute of the attribute record to obtain a configuration data object; and taking a set of configuration data objects corresponding to all attribute records in the attribute record set as a data storage object instance.

Illustratively, the Mapping attribute Mapping _ column of the attribute record with ID 1 points to the Field _1 Field in the data storage table, and has a first Mapping attribute value and a second Mapping attribute value, where the first Mapping attribute value represents that the Field value of the model Field corresponding to the attribute record with ID 1 is assigned to the corresponding Field _1 Field in the data storage table, so as to obtain the configuration data object corresponding to the model Field. And processing all the attribute records in the attribute record set according to the mode to obtain a set of configuration data objects as data storage object examples.

The implementation mode of constructing the data storage object instance through assignment can be selectively set according to actual conditions, and only assignment can be realized. Illustratively, assignments may be made by JAVA (one-door object oriented programming language) reflection technology to construct data storage object instances. By setting the mapping attributes, the data storage object instance can be quickly constructed.

Optionally, the attribute record further comprises: and storing the attribute, wherein the storing attribute indicates whether the field value of the model field corresponding to the attribute record needs to be written into the data storage table, and the storing attribute refers to the Is _ require field in table 3. And circularly traversing each attribute record in the attribute record set, wherein the method comprises the following steps: before the field value of the model field corresponding to the attribute record is obtained from the business dependent data, the field value is confirmed to be written into a data storage table according to the storage attribute, and the field value exists in the business dependent data.

Illustratively, if the data needs to be written into the data storage table, checking whether the field in the business dependent data has a value, and if the field has no value, stopping the data storage operation; if so, storing. If the data need not be written into the data storage table, the field in the service dependent data is not checked for a value.

Whether to write to the data storage table may be whether it is a mandatory field in the data storage table. By setting the storage attribute, unnecessary fields in the data storage table can be quickly filtered during data storage, and the data writing efficiency is greatly improved.

Optionally, the attribute record further comprises: a first identification attribute and an index attribute; the data storage table comprises an index field, which is referred to as a Uuid field in table 5; the index attribute indicates whether the first identification attribute participates in composing the index field, see the Is _ index field in table 3. Constructing a data storage object instance according to the attribute record set and the service dependency data, and further comprising: acquiring attribute values of all first identification attributes participating in forming an index field in an attribute record set according to the index attribute of each attribute record; and generating a field value of the index field according to the attribute values of all the first identification attributes.

The first identification attribute is used to uniquely represent one of the attributes in the attribute record, such as the primary key (ID), the attribute name (Attr _ name), the constraint attribute (Uuid), and the like in table 3. Exemplarily, the first identifier attribute Is an attribute name (Attr _ name), and when the value of the Is _ index Is 1, the corresponding constraint attribute (Uuid) participates in forming the index field; and when the value of the Is _ index Is 0, the corresponding constraint attribute (Uuid) does not participate in forming the index field. When a data storage object instance Is constructed according to the attribute record set and the service dependent data, acquiring attribute values of constraint attributes (Uuid) of all attribute records with the value of the Is _ index of 1 in the attribute record set according to the value of the Is _ index of each attribute record; and generating a field value of the index field according to the attribute values of the constraint attributes (Uuid) of all the attribute records.

The mode of generating the field value of the index field may be selectively set according to the actual situation, for example, sequentially concatenating the values according to the sequence, and then concatenating the values according to the sequence of the initials in the value of the constraint attribute (Uuid). The field values corresponding to the index fields in the data storage table are generated according to the first identification attributes and the field values of the index attributes of all the attribute records in the attribute record set, so that query objects can be spliced and generated in the same mode during data query, the query objects serve as indexes to perform query, and the data stored in the data storage table can be queried conveniently.

When the data management method is used for inquiring data, the data structure of a target data model to be inquired is converted into an inquiry object example consistent with the data structure of a data storage table through an attribute mapping table, a data storage record is inquired from the data storage table according to the inquiry object example, and then the data structure of the data storage record is converted into a form consistent with the data structure of the target data model through the attribute mapping table, so that the required data is quickly inquired from a database for storing business dependency data and business data.

In some embodiments, when a data query is made: acquiring an attribute record set of a target data model from an attribute mapping table; generating an index field instance according to the attribute values of all first identification attributes participating in the index field in the attribute record set of the target data model, and constructing a query object according to the index field instance and the model identification of the target data model; the data structure of the query object is consistent with the data structure of the data storage table; acquiring a data storage record corresponding to the query object from a data storage table; and constructing a search data instance according to the data storage record and the attribute record set of the target data model and returning.

When generating an index field instance according to the attribute values of all first identification attributes participating in the index field in the attribute record set of the target data model, acquiring the attribute record set of the target data model from an attribute mapping table, acquiring the attribute values of all first identification attributes participating in the index field in the attribute record set according to the index attribute of each attribute record in the attribute record set, and generating a field value of the index field, namely the index field instance, according to the attribute values of all the first identification attributes. And when a query object is constructed according to the index field instance and the model identification of the target data model, querying the data storage table according to the index field instance and the model identification of the target data model to obtain a corresponding data storage record. The data structure of the data storage record is consistent with the data structure of the data storage table. Thus, before returning the query results, the data structure of the data storage record is converted to a form consistent with the data structure of the target data model, i.e., a search data instance is built from the data storage record and the set of attribute records of the target data model.

Exemplarily, an attribute record set of Model _ name ═ keyerror log Is queried from an attribute mapping table, whether constraint attribute (Uuid) fields of attribute records in the attribute record set whose Is _ Index value Is 1 have values Is checked, and if yes, all constraint attribute (Uuid) values are spliced together according to the sequence from front to back of the first letter in the constraint attribute (Uuid) values, so as to obtain an Index field instance. And inquiring a data storage record from the data storage table according to the spliced index field instance and the Model _ name value, constructing a return object instance by a JAVA reflection technology, and returning the return object instance to the user.

In other embodiments, the attribute record further comprises: a second identification attribute and a search attribute, the Index attribute indicating whether the second identification attribute can be searched, see the Is _ Index field in table 3. Before generating an index field instance according to the attribute values of all the first identification attributes in the attribute record set of the target data model, the method further includes: and confirming that the attribute values of all the first identification attributes in the attribute record set of the target data model, which form the index field, are not null. If the attribute value of any one first identification attribute participating in the index field in the attribute record set of the target data model is null, acquiring the attribute values of all second identification attributes and the attribute values corresponding to the mapping attributes, which can be searched in the attribute record set of the target data model, according to the search attribute of each attribute record of the target data model, and constructing a query object according to the model identification of the target data model, the attribute values of all second identification attributes and the attribute values corresponding to the mapping attributes.

The second identification attribute is used to uniquely represent one of the attributes in the attribute record, such as the primary key (ID), the attribute name (Attr _ name), the constraint attribute (Uuid), and the like in table 3. Exemplarily, the second identification attribute Is an attribute name (Attr _ name), an attribute record set of a Model _ name ═ KeyErrorLog Is queried from the attribute mapping table, and whether all fields with an Is _ Index value of 1 in the attribute record set have values or not Is checked. If the field with the value of Is _ Index not being 1 exists, judging that the field with the value of Is _ Search Is 1, and extracting the value of attr _ name and the value corresponding to Mapping _ column by the JAVA reflection technology to obtain an Index field example. And inquiring a data storage record from the data storage table according to the spliced index field instance and the Model _ name value, constructing a return object instance by a JAVA reflection technology, and returning the return object instance to the user. According to the embodiment, when the attribute value of part of the first identification attribute of the target data model does not exist, the query object can be constructed according to the second identification attribute and the search attribute, and the data query mode is expanded.

The following is an exemplary description of the method of the embodiments of the present invention, taking tables 2 and 4 as examples. In this embodiment, the data write operation process is as follows:

1. firstly, opening a transaction, and putting a business operation SQL statement and an operation SQL statement of key data into one transaction;

2. inquiring an attribute record set of a data Model to which the persistent data belong from an attribute mapping table according to a Model _ name parameter in the persistent data (namely the service dependent data to be stored);

3. matching each attribute record in the cyclic attribute record set with each model field of the data object to be persisted, checking the is _ index attribute of each model field, and judging whether the model field participates in combining the unique constraint. If the unique constraint is combined, the Uuid attributes of all the model fields are spliced according to the sequence of the initial letters from front to back to generate the Uuid fields in the data storage table;

4. checking the is _ require attribute of each model field, judging whether the model field is a mandatory field, and then checking whether the model field has a value in the data object to be persisted; if the field is the mandatory field, checking whether the model field in the data object to be persisted has a value, and if the model field does not have the value, stopping the persistence action; and if so, persisting the data. If not, not checking whether the field has a value in the data object to be persisted;

5. copying the value of the field of the data object to be persisted to the mapping _ name field corresponding to the Attr _ name according to the matching relationship between the Attr _ name field and the field name of the data object to be persisted by a JAVA reflection technology;

6. constructing a data storage object example according to mapping _ field;

7. the data store object instance is persisted (i.e., saved in the data store table) and the transaction is closed.

In this embodiment, the data write operation process is as follows:

1. searching all attribute records of the error log object according to the condition of the Model _ name ═ KeyErrorLog in the attribute mapping table, see table 4;

2. and checking whether the Uuid fields with the Is _ Index of 1 in all the attribute records have values, if so, sorting the Uuid fields according to character strings, and then splicing the Uuid fields together. And inquiring a data storage record from the data storage table according to the spliced data and the Model _ name, constructing a return object by a JAVA reflection technology, and returning the return object to the user.

3. If the Uuid field with the value of Is _ Index of 1 in the partial attribute record has no value in step 2, searching the attribute record with the value of Is _ Search of 1, and extracting the value of Attr _ name and the value of corresponding Mapping _ column by the JAVA reflection technology. And inquiring the corresponding record from the data storage table according to the value of the Attr _ name, the value of the Mapping _ column and the value of the Model _ name. According to the Java reflection technology, a JAVA object instance is constructed according to the inquired actual storage record and is returned to the user.

The embodiment of the invention ensures the atomicity of the writing of the key data and the writing of the service data by storing the key data into the same relational database with the service system. The self-defined data model and the data storage object in the service system are associated together through the attribute mapping table, no matter how many data models are self-defined in the service system, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized.

According to a second aspect of the embodiments of the present invention, there is provided an apparatus for implementing the above method.

Fig. 3 is a schematic diagram of main modules of an apparatus for data management according to an embodiment of the present invention, and as shown in fig. 3, the apparatus 300 for data management includes:

the data acquisition module 301 is configured to acquire service dependent data corresponding to service data when the service data is written into a service system;

the data writing module 302 is configured to start a data writing transaction, place an operation of writing in service data and an operation of writing in service dependent data in the transaction, store the service dependent data in a data storage table of the service system, and then close the data writing transaction.

Optionally, as shown in fig. 4, the data writing module 302 includes:

the attribute obtaining module 3021 obtains an attribute record set of a data model to which the service dependent data belongs from an attribute mapping table of the service system; the set of attribute records includes an attribute record for each model field in the data model;

the data conversion module 3022, which constructs a data storage object instance according to the attribute record set and the service dependent data; the data structure of the data storage object instance is consistent with the data structure of a data storage table of the service system;

and the data storage module 3023 writes the data storage object instance into the data storage table.

Optionally, the attribute record comprises: (ii) a model attribute;

one or more processors;

a storage device for storing one or more programs,

Fig. 5 illustrates an exemplary system architecture 500 of a data management apparatus or method to which embodiments of the invention may be applied.

As shown in fig. 5, the system architecture 500 may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The

terminal devices

501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the

terminal devices

501, 502, 503. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.

It should be noted that the method for data management provided by the embodiment of the present invention is generally executed by the server 305, and accordingly, the apparatus for data management is generally disposed in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprising: the data acquisition module is used for acquiring service dependent data corresponding to the service data when the service data is written into the service system; and the data writing module is used for starting a data writing transaction, placing the operation of writing in the service data and the operation of writing in the service dependent data in the transaction, storing the service dependent data in a data storage table of the service system, and then closing the data writing transaction. The names of these modules do not form a limitation to the modules themselves in some cases, for example, the data acquisition module may also be described as "storing the service-dependent data in a data storage table of the service system".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: when writing service data into a service system, acquiring service dependent data corresponding to the service data; and starting a data writing transaction, placing an operation of writing service data and an operation of writing service dependent data in the transaction, storing the service dependent data in a data storage table of the service system, and then closing the data writing transaction.

According to the technical scheme of the embodiment of the invention, the service dependent data and the service data are written into the same database, so that the atomicity of the writing operation of the service dependent data and the service data can be ensured, and the condition that the service flow in a service system is abnormal due to the inconsistency of the service dependent data and the service data is avoided; the data model to which the service dependent data in the service system belongs and the data storage object are associated together through the attribute mapping table of the service system, and no matter how many data models exist in the service system, only one storage object is needed, and only one data storage table is needed, so that the transparency of the service system is realized, the development cost of the system is reduced, and the componentization is realized.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of data management, comprising:

closing the data write transaction.

2. The method of claim 1, wherein storing the service dependency data in a data storage table of the service system comprises:

and writing the data storage object instance into the data storage table.

3. The method of claim 2, wherein the attribute record comprises: (ii) a model attribute;

4. The method of claim 2, wherein the attribute record comprises: mapping the attributes; the mapping attribute represents the mapping relation between a model field of a data model and a storage field in the data storage table;

5. The method of claim 4, wherein the attribute record further comprises: storing the attribute; the storage attribute represents whether the field value of the model field corresponding to the attribute record needs to be written into the data storage table or not;

6. The method of claim 4, wherein the attribute record further comprises: a first identification attribute and an index attribute; the data storage table comprises an index field; the index attribute represents whether the first identification attribute participates in composing the index field;

7. The method of claim 6, further comprising:

8. The method of claim 7, wherein the attribute record further comprises: a second identification attribute and a search attribute; the index attribute represents whether a second identification attribute can be searched;

9. The method of claim 2, wherein the model information of the data model is stored in a preset model table.

10. An apparatus for data management, comprising:

11. An electronic device for data management, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.

12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-9.