WO2020192663A1

WO2020192663A1 - Data management method and related device

Info

Publication number: WO2020192663A1
Application number: PCT/CN2020/080952
Authority: WO
Inventors: 田文罡
Original assignee: 华为技术有限公司
Priority date: 2019-03-26
Filing date: 2020-03-24
Publication date: 2020-10-01
Also published as: CN111753141A

Abstract

Disclosed in the embodiments of the present application are a data management method and a related device, used to implement uniformity of data stored for a same data object across a plurality of types of storage systems. Data corresponding to a structured attribute of the data object is stored in a first storage system, for example, a relational database. Data corresponding to an unstructured attribute of the data object is stored in another type of storage system, for example, a KV system or a file system. An association relationship between the structured data and the unstructured data of the data object is recorded by means of relationship data stored in the database. When the data object is operated on, the record in the database is accessed first, and a key value and a path for the unstructured attribute is acquired from the record in the database, and then the data corresponding to the unstructured attribute is accessed by means of an interface of the storage system of the other type. Thus, data uniformity between systems of a plurality of storage types can be implemented by means of database transaction uniformity and a specified data access sequence.

Description

A data management method and related equipment

Technical field

This application relates to the field of computer technology, in particular to a data management method and related equipment.

Background technique

In computer systems, data is generally divided into structured data and unstructured data. Among them, structured data is data that is logically expressed and realized by a two-dimensional table structure and follows data format and length specifications, such as sales information, property information Wait. Unstructured data is irregular or incomplete data structure, and there is no predefined data, such as documents, pictures, audio, and video.

For data storage, structured data is generally stored in a relational database, large unstructured data is generally stored in a file storage system, and small unstructured data is generally stored in a key value (KV) system. In practical applications, a data object may contain both structured data and unstructured data. For example, the data object is the information of a picture. The attribute information of the picture, such as the name, size, shooting time, and latitude and longitude information of the shooting location, is structured data. The picture itself is unstructured data, based on The thumbnail generated by this picture is unstructured data. Then, the name, size, shooting time, and latitude and longitude information of the picture will be stored in the relational database, the picture itself will be stored in the file storage system, and the thumbnail generated based on the picture will be stored in the KV storage system. It can be seen that since a data object may contain structured data and unstructured data at the same time, a data object may be stored across multiple data systems.

In the prior art, when a data object is stored across multiple data systems, users can generally operate on the data in multiple data systems separately, and there may be a problem of inconsistency of the same data object in each data system. . For example, if a user deletes a file of a picture from the file storage system, the user can still obtain the attribute information of the picture through the database, but because the file of the picture has been deleted, the picture cannot be displayed normally. How to maintain data consistency when data is stored across multiple data systems is a problem to be solved by those skilled in the art.

Summary of the invention

The embodiments of the present application provide a data management method and related equipment, which are used to achieve data consistency when data objects are stored across multiple data systems.

In the first aspect, an embodiment of the present application provides a data management method. The method includes: generating a record of a data object in a relational data table, the data object having multiple attributes, and the multiple attributes include structured attributes and Unstructured attributes, the record indicates the association relationship between the structured attributes and unstructured attributes of the data object, the relational data table is stored in a first storage system; the unstructured attributes of the data object The corresponding data is stored in the second storage system; an operation instruction is received, the operation instruction is used to perform an operation on the data object; in response to the operation instruction, the data object is determined from the first storage system The record; obtaining data corresponding to at least one of the attributes of the data object from at least one of the first storage system and the second storage system according to the record; based on The data corresponding to the at least one attribute performs the operation on the data object. Since the data corresponding to the multiple attributes of the data object stored across multiple data systems are obtained through the record, the data object can be kept in data consistency when stored across multiple data systems.

With reference to the first aspect, in a possible implementation manner, the generating a record of a data object in a relational data table includes: receiving an insert instruction or an update instruction, the insert instruction is used to insert the data object, the The update instruction is used to update the data object; the insert instruction and the update instruction both include the object type of the data object, and the data corresponding to the structured attribute and the data corresponding to the unstructured attribute of the data object; Determine the relational data table corresponding to the data object according to the object type; generate the relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute A record of the data object; submit the transaction corresponding to the insert instruction or update instruction; wherein the transaction corresponding to the insert instruction or update instruction is stored in the second storage system after the data corresponding to the unstructured attribute of the data object is stored submit.

With reference to the first aspect, in a possible implementation manner, the received instruction is the insert instruction, the unstructured attributes of the data object include key-value KV attributes, and the second storage system is a KV storage system; According to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute, generating the record of the data object in the relational data table corresponding to the data object includes: according to the first version identifier and the The first key value in the data corresponding to the KV attribute generates the second key value; according to the data corresponding to the structured attribute in the record and the data corresponding to the KV attribute in the record, it is generated in the relational data table corresponding to the data object The record of the data object; wherein the data corresponding to the KV attribute in the record includes the second key value, and the data corresponding to the structured attribute in the record includes data corresponding to the structured attribute of the data object .

With reference to the first aspect, in a possible implementation manner, the received instruction is the insert instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system; The data corresponding to the structured attribute and the data corresponding to the unstructured attribute, generating a record of the data object in a relational data table corresponding to the data object includes: according to the first version identifier and the The first path in the data corresponding to the file attribute generates the second path; according to the data corresponding to the structured attribute in the record and the data corresponding to the file attribute in the record, the data object is generated in the relational data table corresponding to the data object. A record of a data object; wherein the data corresponding to the file attribute in the record includes the second path, and the data corresponding to the structured attribute in the record includes data corresponding to the structured attribute of the data object.

With reference to the first aspect, in a possible implementation manner, the received instruction is the update instruction, the unstructured attributes of the data object include KV attributes, and the second storage system is a KV storage system; The data corresponding to the structured attribute and the data corresponding to the unstructured attribute, generating a record of the data object in the relational data table corresponding to the data object, includes: according to the second version identifier and the KV attribute The first key value in the corresponding data generates the third key value; according to the data corresponding to the structured attribute in the record and the data corresponding to the KV attribute in the record, the data object is generated in the relational data table corresponding to the data object. A record of a data object; wherein the data corresponding to the KV attribute in the record includes the third key value, and the data corresponding to the structured attribute in the record includes data corresponding to the structured attribute of the data object.

With reference to the first aspect, in a possible implementation manner, the received instruction is the update instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system; The data corresponding to the structured attribute and the data corresponding to the unstructured attribute, generating a record of the data object in a relational data table corresponding to the data object includes: according to the second version identifier and the The first path in the data corresponding to the file attribute generates the third path; according to the data corresponding to the structured attribute in the record and the data corresponding to the file attribute in the record, the data object is generated in the relational data table corresponding to the data object. A record of a data object; wherein the data corresponding to the file attribute in the record includes the third path, and the data corresponding to the structured attribute in the record includes data corresponding to the structured attribute of the data object.

With reference to the first aspect, in a possible implementation manner, the data corresponding to the unstructured attribute of the data object stored in the second storage system includes the identifier and content corresponding to the unstructured attribute; The data corresponding to the unstructured attribute in the record stored in the first storage system includes an identifier corresponding to the unstructured attribute.

With reference to the first aspect, in a possible implementation manner, the operation instruction includes a query instruction, and the query instruction includes a query condition; in response to the operation instruction, the determination is made from the first storage system. The record of the data object includes: in response to the operation instruction, selecting a record of the data object that meets the query condition from the first storage system; and selecting from the first storage system according to the record Obtaining data corresponding to at least one attribute of the plurality of attributes of the data object from at least one of the storage systems in the second storage system includes: according to the record from the first storage system and the Acquiring data corresponding to the multiple attributes of the data object in the second storage system; the performing the operation on the data object based on the data corresponding to the at least one attribute includes: according to the data object The data corresponding to each of the multiple attributes and the sequence of the multiple attributes in the record establishes the data object; and the data object is returned as a query result.

With reference to the first aspect, in a possible implementation manner, the unstructured attribute of the record includes a KV attribute, the second storage system is a KV storage system, and the record is obtained from the first storage system according to the record. And acquiring the data corresponding to the multiple attributes of the data object in the second storage system includes: reading the KV data corresponding to the key value from the second storage system according to the key value, and removing the The version identifier in the key value; the key value is the data corresponding to the KV attribute in the record, and the version identifier includes a first version identifier and a second version identifier; wherein, the data corresponding to the KV attribute of the data object It includes the key value after removing the version identifier and the KV data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.

With reference to the first aspect, in a possible implementation manner, the unstructured attributes of the record include file attributes, the second storage system is a file storage system, and the record is obtained from the first storage system according to the record. And acquiring the data corresponding to the multiple attributes of the data object in the second storage system includes: reading the file data corresponding to the path from the second storage system according to the path, and removing the data from the path The version identifier; the path is the data corresponding to the file attribute in the record, the version identifier includes a first version identifier and a second version identifier; wherein the data corresponding to the file attribute of the data object includes the removal version identifier The following path and the file data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.

With reference to the first aspect, in a possible implementation manner, the operation instruction further includes a deletion instruction, and the deletion instruction includes the object type of the data object, and the data and non-compatibility corresponding to the structured attribute of the data object. Data corresponding to a structured attribute; wherein, in response to the operation instruction, determining the record of the data object from the first storage system includes: determining the data object corresponding to the data object according to the object type A relational data table; the record of the data object is determined from the relational data table, and the data corresponding to the structured attribute in the record is the same as the data corresponding to the structured attribute of the data object; The record acquiring data corresponding to at least one attribute of the multiple attributes of the data object from at least one of the first storage system and the second storage system includes: said recording from Acquiring data corresponding to the multiple attributes of the data object in the first storage system; and performing the operation on the data object based on the data corresponding to the at least one attribute includes: Delete data corresponding to the multiple attributes of the data object in a storage system; submit a transaction corresponding to the delete instruction.

With reference to the first aspect, in a possible implementation manner, the unstructured attributes include KV attributes, the second storage system is a KV storage system, and the method further includes: when a verification instruction is received or when When it is detected that the verification condition is satisfied, the key value in the second storage system is traversed, and the key value is the data corresponding to the KV attribute in the second storage system; in the process of traversing the key value, if If the same key value as the fourth key value cannot be found in the relational data table stored in the first storage system, delete the fourth key value and the fourth key value correspondence in the second storage system The fourth key value is one of multiple key values in the second storage system.

With reference to the first aspect, in a possible implementation manner, the unstructured attributes include file attributes, the second storage system is a file storage system, and the method further includes: when a verification instruction is received or when When it is detected that the verification condition is met, the path in the second storage system is traversed, and the path is the data corresponding to the file attribute in the second storage system; in the process of traversing the path, if the path is If the same path as the fourth path cannot be found in the relational data table stored in the first storage system, the fourth path and the file data corresponding to the fourth path are deleted in the second storage system. The four-path is one of the multiple paths in the second storage system.

With reference to the first aspect, in a possible implementation manner, before determining the relational data table corresponding to the data object according to the object type, the method further includes: receiving a definition instruction for the object type to which the data object belongs , The definition instruction includes definition information of the object type, and the definition information is used to define the structure of the relational data table of the object type; according to the definition instruction, the first storage system generates the The relational data table of the object type.

With reference to the first aspect, in a possible implementation manner, the determining the relational data table corresponding to the data object according to the object type includes: determining the object to which the data object belongs according to the insert instruction or update instruction Type; the relational data table corresponding to the data object is determined according to the object type.

In a second aspect, an embodiment of the present application provides a data management device, which includes a unit for executing the method described in the first aspect or various possible implementations of the first aspect.

The above-mentioned data management device may be an electronic device, or a device for implementing data management in an electronic device (for example, an operating system, a database management system), or a server, such as a database server, an application server, etc.

The units included in the aforementioned data management device may be hardware circuits, software, or hardware circuits combined with software.

In the third aspect, the embodiments of the present application provide another data management device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store program instructions, and the processor is used to call the The program instructions execute the method described in the first aspect or any possible implementation of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer storage medium stores program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned first aspect or the first aspect. Any possible implementation of the method described.

In a fifth aspect, an embodiment of the present application provides a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

In the embodiment of the present application, the data management device may generate a record of the data object in the relational data table. The record indicates the association relationship between the structured data and the unstructured data of the data object. The relational data table is stored in the first storage. In the system, the unstructured data of the data object is stored in the second storage system. When the data needs to be manipulated, the record of the data object can be obtained from the first storage system, and the multiple attributes of the data object can be obtained from the first storage system and/or the second storage system according to the record Data corresponding to at least one attribute; and then based on the data corresponding to the at least one attribute, performing the operation on the data object. Since the data corresponding to the multiple attributes of the data object stored across multiple data systems are all obtained through the records in the relational data table, the data object can maintain data consistency when stored across multiple data systems.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art.

FIG. 1A is a schematic diagram of a data management device provided by an embodiment of the present application;

FIG. 1B is a schematic diagram of yet another data management device provided by an embodiment of the present application;

FIG. 1C is a schematic diagram of another data management device provided by an embodiment of the present application;

2 is a schematic diagram of the architecture of a data management system provided by an embodiment of the present application;

FIG. 3 is a flowchart of a data management method provided by an embodiment of the present application;

4 is a schematic diagram of another data management device provided by an embodiment of the present application;

Fig. 5 is a schematic diagram of yet another data management device provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are described in more detail below.

The data management method provided in the embodiments of the present application can be applied to a data management device, and the data management device includes a first storage system and a second storage system. The first storage system and the second storage system are different types of storage systems. Among them, the first storage system can be used to store the structured attributes of the data object and corresponding structured data, and the second storage system can be used to store the unstructured attributes of the data object and the corresponding unstructured data. Structured attributes are attributes used to describe or define characteristics of structured data, and unstructured attributes are attributes used to describe or define characteristics of unstructured data. Structured data, also known as row data, is data logically expressed and realized by a two-dimensional table structure, which is mainly stored and managed by relational databases; unstructured data is irregular or incomplete data structure, and has no predefined Data model, data that is not convenient to use two-dimensional logical tables of the database, such as documents, texts, pictures, reports, images, audio/video information, etc. In an embodiment, the first storage system may be a database, such as a relational database. Optionally, there may be multiple unstructured attributes of the data object, and the second storage system may be multiple. For example, the unstructured attributes of the data object may include key-value (KV) Attribute and file attribute, the data management device may include a KV storage system (such as a KV database) and a file storage system (referred to as "file system"). Specifically, the data management device may include terminal devices such as mobile phones, tablet computers, personal digital assistants (personal digital assistants, PDAs), mobile internet devices (mobile internet devices, MIDs), etc., and may also include database servers, application servers, etc. The storage and processing function equipment is not limited in the embodiment of the present invention.

The data management device can receive an operation instruction for a data object input by a user through an application program it runs, and execute the operation instruction on the data object. For example, the application can be an album that stores images or videos, and can receive user-input operation instructions on images or videos; or the application can be a text-creation software that can receive user-input operations on text Instructions; or the application program may be instant messaging software, which can receive operating instructions input by the user for office documents, text, pictures, images, audio/video, and other data in the software.

Refer to FIG. 1A, which is a schematic diagram of a data management device provided by an embodiment of the present application. The data management device includes an application module, an operation module, an interface module, and a storage system module. These modules will be further introduced below.

Wherein, the application module may include one or more application programs, and these application programs may receive operation instructions for the data object input by the user. For example, the application can include photo albums, mailboxes, document processing software, and so on.

The operation module is a data object management component that provides an interface to the application module. Through the operation module, the application module can implement operations such as defining, inserting, modifying, deleting, querying, and verifying data objects. Specifically, the operation module may execute the operation indicated by the instruction according to the instruction received from the application module. The received instructions are different, and the operations performed by the operation module are different. The following examples illustrate several different instructions.

If the instruction is a definition instruction, the operation module can determine the definition information of the object type of the data object according to the definition instruction, and then store the definition information in the storage system module through the interface module. Wherein, the definition information of the object type of the data object includes the structured attribute and the unstructured attribute of the data object.

If the instruction is an insert instruction, the operation module can determine the information of the data object that needs to be stored according to the insert instruction, and then store the information of the data object that needs to be inserted into the storage system module through the interface module. Among them, the operation module saves different types of data to different storage systems. Specifically, structured data is saved to the database, file data is saved to the file storage system, and key-value data is saved to the KV storage system.

If the instruction is an update instruction, the operation module can determine the information of the data object that needs to be updated according to the update instruction, and then store the information of the data object that needs to be updated in the storage system module through the interface module. Among them, the operation module saves different types of data to different storage systems. Specifically, structured data is saved to the database, file data is saved to the file storage system, and key-value data is saved to the KV storage system.

If the instruction is a query instruction, the operation module can determine the query condition according to the query instruction, and then select data objects that meet the query condition from the storage system module through the interface module according to the query condition and feed it back to the application module.

If the instruction is a delete instruction, the operation module can determine the data object to be deleted according to the delete instruction, and then delete the data object to be deleted from the storage system module through the interface module.

If the command is a check command, the operating module can check the data stored in the storage system module through the interface module according to the check command to clear invalid data.

The interface module provides an interface for accessing the storage system module, and the operation module can access the data in the storage system module through the interface module. In one embodiment, as shown in FIG. 1B, the interface module includes a first storage system interface submodule and a second storage system interface submodule; the storage system module includes a first storage system submodule and a second storage system submodule. The operating module can access the data in the first storage system through the first storage system interface sub-module, and the operating module can access the data in the second storage system through the second storage system interface sub-module.

In one embodiment, the first storage system is a database system, such as a relational database, the first storage system interface submodule is a database system interface; the second storage system is a storage system for storing non-relational data, such as KV The storage system and/or the file storage system. Correspondingly, the second storage system interface submodule includes a KV system interface submodule and/or a file storage system interface submodule. The data management device stores the data corresponding to the structured attributes of the same data object in the first storage system, and stores the data corresponding to the unstructured data of the data object in the second storage system. Further, the data management device generates a relational data table in the first storage system to establish an association relationship between the structured attribute and the unstructured attribute of the data object. The relational data table contains the record of the data object, the record includes the name of the structured attribute of the data object, the name of the unstructured attribute, the data content (value) corresponding to the structured attribute, and the corresponding unstructured attribute The identification of the data (the data content corresponding to the unstructured attribute is stored in the second storage system). When the operation module receives the operation instruction for the data object from the application module, it first determines the record of the data object from the relational data table stored in the first storage system, and then can obtain the data from the relational data table according to the record The data corresponding to the structured attribute of the object, and the identifier of the data corresponding to the unstructured attribute can be determined based on the record, and then the data corresponding to the unstructured attribute of the data object is obtained from the second storage system according to the identifier (data content ). Finally, the data management device performs corresponding operations on the data object based on the acquired data corresponding to the structured attribute of the data object and/or the data corresponding to the unstructured attribute. The data management device generates the relational data table of the data object, and the detailed process of storing the data and operating the data can refer to the related embodiment in FIG. 3.

In an embodiment, the first storage system sub-module may be a database that supports multi-version concurrency control (MVCC), such as a lightweight database (SQLite). Among them, MVCC can maintain multiple snapshot copies for each record in the database, and maintain the visibility of the copies through a start timestamp (begin timestamp) and an end timestamp (end timestamp). The second storage system sub-module is a storage system that supports persistence, such as KV system, flash friendly file system (F2FS), fourth-generation extended file system (EXT4) and so on. Among them, the meaning of persistence is a mechanism for converting data between persistent state and transient state. In layman's terms, transient data (such as data in memory) is persisted into persistent data, and the persistent data can be stored for a long time. When the data management device accesses a data object, it first accesses the record of the data object in the database, and then operates the data stored in the second storage system according to the record content, so that the second storage system can be realized by means of the concurrency control of the database ( Such as file system, KV system) concurrent control access. For the insertion and modification operations of data objects, the data management device must be in the file system and the KV system operation is completed before submitting the database transaction; for the deletion operation, the data management device must first manipulate the data in the database, and then submit the transaction. Operate the file system, the data in the KV system.

Figure 1C shows a more specific implementation of the data management device. According to FIG. 1C, the operation module may include a data definition sub-module, a data insertion, update, deletion, and query sub-module, and a data verification sub-module. Among them, the data definition sub-module is used to determine the definition information of the data object according to the definition instruction. The data insert, update, delete, and query sub-module, and the data check sub-module can perform operations such as insert, update, delete, and query on the actually stored data object according to the insert instruction, update instruction, delete instruction, and query instruction. The data verification sub-module can verify data in different storage systems to clear invalid data and ensure data consistency in multiple storage systems. It should be noted that the operation modules can also be divided into other ways as needed. For example, the data insertion, update, deletion, and query submodules can be divided into data insertion submodules, data update submodules, data deletion submodules, and data The query sub-module is not specifically limited in the embodiment of this application.

Optionally, the operation module may further include a first storage system operation sub-module and a second storage system operation sub-module. Wherein, the first storage system operation submodule is used to perform operations on data in the first storage system, and the second storage system operation submodule is used to perform operations on data in the second storage system.

Optionally, since the storage system module may include multiple storage systems, the operation module may also include a system adaptation sub-module that can process the data object so that the data object can adapt to multiple storage systems. Storage system, or adapt the data objects of operation feedback to the application environment of different applications. For example, it may include a first storage system adaptation sub-module and a second storage system adaptation sub-module. For example, the first storage system adaptation submodule may include a database adaptation submodule, which can interface with different databases, facilitate database switching, encapsulate database operation interfaces, and provide database-like interfaces for upper-level services, including Open the database (open), perform database operations to create (create), insert (insert), update (update), delete (delete), query (query) operations, and perform begin, commit, etc. Transaction operation. In yet another possible implementation solution, the second storage system adapter submodule may include a KV storage system adapter submodule, which can be connected to different KV storage systems, can facilitate KV storage switching, and package the KV operation interface, Provide KV-like interfaces for upper-layer services, including input (put), output (get), delete (delete) and other operations. In yet another possible implementation solution, the second storage system adaptation submodule may also include a file storage system adaptation submodule, which can interface with different file storage systems, facilitate file storage system switching, and interface with file storage systems. Encapsulation provides an interface similar to a file storage system for upper-level services, including operations such as opening files (open), reading files (read), writing files (write), and closing files (close). Refer to FIG. 1C, which is a schematic diagram of another data management device provided by an embodiment of the present application.

In another possible implementation manner, the data management method of the embodiment of the present application can also be applied to a data management system. Refer to FIG. 2, which is a schematic diagram of the architecture of a data management system provided by an embodiment of the present application. Including client and data management equipment. The following two devices are further introduced.

The client is a device that provides local services to customers. Except for some applications that only run locally, the operation of the client generally needs to cooperate with the server. More commonly used clients include web browsers used on the World Wide Web, email clients for receiving and sending emails, photo album clients for storing images or videos, text clients for creating text, and client software for instant messaging. In the embodiment of the present application, the client may receive an operation instruction for the data object input by the user. The operation instruction may include an insert instruction, an update instruction, a definition instruction, a query instruction, a delete instruction, a verification instruction, and so on. For example, the client can be an album client that stores images or videos, and can receive operation instructions for images or videos entered by the user; the client can be a text client that creates text, and can receive the text input by the user. The operating instructions; the client can be instant messaging client software, which can receive operating instructions entered by the user for office documents, text, pictures, images, and audio/video data in the software.

A data management device is a device that provides data storage and processing services for clients, and can implement data management. For example, the management can include definition, storage, update, deletion, verification, and so on. The client and the data management device are two independent devices, and the client and the server communicate through a network or a data line. The data management device can receive an operation instruction from the client, and then execute the operation instruction on the data object. The structure of the data management device can refer to the structure described in Figures 1A to 1C, and only replace the "application module" illustrated in Figures 1A to 1C with a "receiving module". The interface module is used to receive from the client Operation instructions for data objects. In addition, the functions of the remaining modules except for the application module in the modules illustrated in FIGS. 1A to 1C can refer to the above description, which will not be repeated here.

Refer to FIG. 3, which is a flowchart of a data management method provided by an embodiment of the present application. The data management device described below may be the data management device shown in any one of FIG. 1A to FIG. 1C and FIG. 2; the method includes but is not limited to the following steps.

S301: Generate a record of the data object in the relational data table.

The data object has multiple attributes, and the multiple attributes include structured attributes and unstructured attributes. Structured attributes are attributes used to describe or define characteristics of structured data, and unstructured attributes are attributes used to describe or define characteristics of unstructured data. Structured data is data logically expressed and realized by a two-dimensional table structure, which is mainly stored and managed by relational databases; unstructured data is irregular or incomplete data structure, and there is no predefined data model, which is inconvenient to use Data represented by two-dimensional logical tables of the database, such as documents, texts, pictures, reports, images, audio/video information, etc. The record generated in the relational data table includes the structured attribute of the data object and the data corresponding to the structured attribute, and the association relationship between the structured attribute and the unstructured attribute. The relational data table is stored in the first storage system in.

S302: Store data corresponding to the unstructured attribute of the data object in a second storage system.

The record of a data object in the relational data table can contain structured attribute fields and unstructured attribute fields. The value of the structured attribute field is the data corresponding to the structured attribute, and the value of the unstructured field is the data corresponding to the unstructured attribute. The identifier of the, such as key value, path, etc. Further, the unstructured data corresponding to the unstructured attribute is stored in the second storage system. In other words, through the relational data table in the first storage system, the structured attributes and unstructured attributes of the data object can be associated. The records in the relational data table include structured attributes and unstructured attributes, as well as structure The data corresponding to each of the chemical attribute and the unstructured attribute. It should be noted that in the records of the relational data table, the data corresponding to the structured attribute is the data itself, that is, the data content or data value, not the data corresponding to the structured attribute, not the original data content, but an identification of the data , The real data content is stored in the second storage system. The following examples will explain in detail.

S303. Receive an operation instruction, where the operation instruction is used to perform an operation on the data object.

Specifically, the operation instruction may be a query statement (query), a check statement (check), a delete statement (delete statement) described in a database definition language (data definition language, DDL), data manipulation language (data manipulation language, DML), etc. ) Or a function call statement, etc. The operation instruction indicates the object type of the data object involved in the operation. Optionally, the operation instruction may also include data required to perform an operation on the data object, such as data corresponding to the structured attribute and data corresponding to the unstructured attribute of the data object.

S304. In response to the operation instruction, determine the record of the data object from the first storage system.

S305. Obtain data corresponding to at least one attribute of the plurality of attributes of the data object from at least one of the first storage system and the second storage system according to the record.

S306: Perform the operation on the data object based on the data corresponding to the at least one attribute.

In one embodiment, before the data management device generates the record of the data object, the data object may be defined based on the data definition instruction, that is, the name and type of each attribute of the data object are defined, for example, the defined data object One or more structured attributes of and one or more unstructured attributes. In a specific implementation manner, the definition process of the data object includes: the data management device receives a definition instruction for the object type to which the data object belongs, and the definition instruction includes definition information of the object type, and The definition information is used to define the structure of the relational data table of the object type; in the first storage system according to the definition instruction, the relational data table of the object type is generated to associate the structured attributes and non-conformities of the data object. Structured attributes.

For example, the definition information may be definition information of an object type of "picture", and the unstructured attributes of the data object may include files and key values. The definition information of the “picture” type can be “picture(name STRING, size INT, path FILE, latitude DOUBLE, longitude DOUBLE, time_taken STRING, thumbnail KV)”, and the object type “picture” generated according to the definition information The relational data table can refer to Table 1.

Table I

属性名称Attribute name	namename	sizesize	pathpath	latitudelatitude	longitudelongitude	time_takentime_taken	thumbnailthumbnail
属性类型Attribute type	STRINGSTRING	INTINT	FILEFILE	DOUBLEDOUBLE	DOUBLEDOUBLE	STRINGSTRING	KVKV

Among them, the structural attributes in the definition information of the "picture" type are "name", "size", "latitude", "longitude" and "time_taken". The structured attributes are "path" and "thumbnail", where "path" is the file attribute, and "thumbnail" is the KV attribute. The definition command containing the definition information can be "create(picture(nameSTRING,sizeINT,pathFILE,latitudeDOUBLE,longitudeDOUBLE,time_takenSTRING,thumbnailKV))". Among them, "picture" is the object type to which the data object belongs.

In step S301, there may be two situations in which the data management device generates a record of the data object in the relational data table. The first case is to generate a record of the data object based on the received insert instruction, and the second case is to generate a record of the data object based on the received update instruction. Wherein, the insert instruction is used to insert the data object, and the update instruction is used to update the data object. Both the insert instruction and the update instruction for the data object indicate the object type of the data object and the data corresponding to the attribute of the data object. The following will give a specific introduction to these two situations.

For the first case, after the data management device receives the insert instruction for inserting the data object, the process of generating a record of the data object in the relational data table includes: according to the data corresponding to the structured attribute and the For the data corresponding to the unstructured attribute, a record of the data object is generated in the relational data table corresponding to the data object; and, after the data corresponding to the unstructured attribute of the data object is stored in the second storage system Submit the transaction corresponding to the insert instruction or update instruction.

Wherein, the data management device may determine the object type to which the data object belongs according to the insertion instruction; and determine the relational data table corresponding to the data object according to the object type.

For example, the insert instruction for the data object is "insert(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key, t_snoopy.jpg)))". The data management device may determine the object type to which the data object belongs is "picture" according to the insert instruction, and then determine the relational data table corresponding to the data object according to the object type "picture" as the "picture" in the first storage system. "This kind of object type relational data table. The relational data table can refer to Table 1 above.

Specifically, the data management device may obtain the data corresponding to the structured attribute and the data corresponding to the unstructured attribute of the data object from the insert instruction. For example, the insert instruction is "insert(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg) ))". The data management device can obtain the structured attributes of the data object "name", "size", "latitude", "longitude", and "time_taken" from the update instruction. )" corresponding data "Snoopy", 2M, 39.92, 116.46, "2018-10-12", and fill these data into the corresponding structured attribute fields in the relational data table. At the same time, the data management device can obtain the data (data/Snoopy.jpg, Snoopy.jpg), (t_snoopy_key, t_snoopy.jpg) corresponding to the unstructured attributes "path" and "thumbnail" respectively, And fill these data into the corresponding unstructured attribute fields in the relational data table. It can be seen that the data corresponding to the unstructured attributes of the data object includes the identification and content of the data. For example, the data (data/Snoopy.jpg, Snoopy.jpg) corresponding to the unstructured attribute "path" includes the identification of the unstructured data, namely the path: data/Snoopy.jpg, and the data content, namely Snoopy. jpg file; the data (t_snoopy_key, t_snoopy.jpg) corresponding to the unstructured attribute "thumbnail" includes the thumbnail identifier t_snoopy_key and the content of the thumbnail t_snoopy.jpg.

The following describes the detailed process of the data management device generating the record of the data object in the relational data table corresponding to the data object during the process of inserting the data object.

In an embodiment, the unstructured attribute of the data object includes a KV attribute, and the second storage system is a KV storage system. According to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute, the data management device generates a record of the data object in the relational data table corresponding to the data object. The method includes: according to the first version Identify the first key value in the data corresponding to the KV attribute to generate the second key value; according to the data corresponding to the structured attribute in the record, and the data corresponding to the KV attribute in the record, in the relational data corresponding to the data object A record of the data object is generated in a table; wherein the data corresponding to the KV attribute in the record includes the second key value, and the data corresponding to the structured attribute in the record includes the structured attribute of the data object The corresponding data.

The version identifier is used to indicate the version of the record. During the update process of the data object, the version identifier of the record can distinguish whether the record is a record of the data object before update or a record of the data object after update. Optionally, the meaning represented by the first version identifier is the version identifier of the record that has been stored in the database before the data object is updated. The meaning represented by the second version identifier is the version identifier of the record stored in the database after the data object is updated. In a possible situation, there are only two version identifiers in the data management device, for example, version1 and version2. If the first version identifier is version1, the data management device determines that the second version identifier is version2; if the first version identifier is version2, the data management device determines that the second version identifier is version1. In yet another possible situation, there may be multiple version identifiers in the data management device, such as version 1, version 2, and version 3, and so on. If the first version identification is version1, the data management device determines that the second version identification is version2 or other version identifications except version1; if the first version identification is version2, the data management device determines that the second version identification is version3 Or other version identifiers except version2.

Optionally, the method of generating the second key value according to the first version identifier and the first key value in the data corresponding to the KV attribute may include: adding the first key value to the first key value in the data corresponding to the KV attribute Version identification to generate the second key value. It should be noted that there may also be other ways of generating the second key value based on the first version identifier and the first key value in the data corresponding to the KV attribute, which is not limited here.

In an embodiment, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system. According to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute, the data management device generates a record of the data object in a relational data table corresponding to the data object. The method includes: A version identification and the first path in the data corresponding to the file attribute generate a second path; according to the data corresponding to the structured attribute in the record and the data corresponding to the file attribute in the record, the relationship between the data object corresponds to A record of the data object is generated in a data table; wherein the data corresponding to the file attribute in the record includes the second path, and the data corresponding to the structured attribute in the record includes the structured attribute of the data object The corresponding data.

Optionally, the method of generating the second path according to the first version identifier and the first path in the data corresponding to the file attribute may include: adding the first version identifier to the first path in the data corresponding to the file attribute To generate the second path. Wherein, if the second path does not exist before executing the insert instruction, the data management device creates the second path in the database. It should be noted that there may also be other ways of generating the second path based on the first version identifier and the first path in the data corresponding to the file attribute, which is not limited here.

The following uses the data object picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg)) in the relational data table Take the generation of the record as an example to introduce the record generation process of generating a data object in a relational data table. The unstructured attributes included in the data object are file attributes (data/Snoopy.jpg, Snoopy.jpg) and KV attributes ( t_snoopy_key,t_snoopy.jpg). The insert instruction for the data object is "insert(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg )))", the data object contains 7 attributes, and the first version is identified as version1. Wherein, the first attribute of the data object is a structured attribute, and the data corresponding to the structured attribute in the record is the data "Snoopy" corresponding to the structured attribute. In addition, other structured attributes other than the first structured attribute among the seven attributes of the data object can also refer to this method, which will not be repeated here. The third attribute of the data object is the file attribute, then the first path "data/Snoopy.jpg" in the data "(data/Snoopy.jpg,Snoopy.jpg)" corresponding to the first version identifier version1 and the file attribute is used. "The second path "data/version1/Snoopy.jpg" is generated, and the data corresponding to the file attribute in the record is the second path "data/version1/Snoopy.jpg". The seventh attribute of the data object is the KV attribute, and the second key value is generated according to the first key value "t_snoopy_key" in the data "(t_snoopy_key,t_snoopy.jpg)" corresponding to the first version identifier version1 and the KV attribute "T_snoopy_key_version1", the data corresponding to the KV attribute in the record is the second key value "t_snoopy_key_version1". Then, according to the data corresponding to the structured attribute in the record, the data corresponding to the KV attribute in the record, and the data corresponding to the file attribute in the record, a record of the data object is generated in the relational data table corresponding to the data object . The record can refer to Table 2 below.

Table II

The data management device will store the data "(t_snoopy_key_version1,t_snoopy.jpg)" corresponding to the KV attribute of the data object in the KV storage system, and the data corresponding to the file attribute of the data object "(data/version1/Snoopy.jpg, Snoopy.jpg)” is stored in the file storage system. The data corresponding to the KV attributes stored in the KV storage system can be referred to Table 3:

Table Three

属性名称Attribute name	键值Key value	缩略图Thumbnail
数据data	t_snoopy_key_version1t_snoopy_key_version1	t_snoopy.jpgt_snoopy.jpg

The data corresponding to the file attributes stored in the file storage system can be referred to Table 4:

Table Four

属性名称Attribute name	路径path	图片image
数据data	data/version1/Snoopy.jpgdata/version1/Snoopy.jpg	Snoopy.jpgSnoopy.jpg

After the data management device stores the data corresponding to the unstructured attribute of the data object in the second storage system, the data management device will submit the database transaction corresponding to the insert instruction. It should be noted that the operation of submitting database transactions by the data management device needs to be executed after both the first storage system and the second storage system save the corresponding data. This operation mode can ensure that the data of the data object is in each storage system. All were successfully stored.

In step S301, for the second case, after the data management device receives the update instruction for updating the data object, the process of generating the relational data table of the data object includes: in the relational data table corresponding to the data object A record of the data object is generated in the data object; the transaction corresponding to the insert instruction or the update instruction is submitted; wherein the transaction corresponding to the insert instruction or the update instruction is stored in the second data corresponding to the unstructured attribute of the data object Submit later in the storage system.

The data management device may determine the object type to which the data object belongs according to the update instruction; determine the relational data table corresponding to the data object according to the object type.

For example, the update instruction for the data object is "update(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key, t_snoopy.jpg)))". The data management device may determine the object type to which the data object belongs is "picture" according to the update instruction, and then determine the relational data table corresponding to the data object according to the object type "picture" as the "picture" in the first storage system "This kind of object type relational data table. For the relational data table, please refer to Table 1 above.

The following describes how the data management device generates the record of the data object in the relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute during the update process. method.

In an embodiment, the unstructured attribute of the data object includes a KV attribute, and the second storage system is a KV storage system. According to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute, the data management device generates a record of the data object in a relational data table corresponding to the data object. The method includes: according to a second version Identify the first key value in the data corresponding to the KV attribute to generate the third key value; according to the data corresponding to the structured attribute in the record, and the data corresponding to the KV attribute in the record, the relationship between the data object A record of the data object is generated in a data table; wherein the data corresponding to the KV attribute in the record includes the third key value, and the data corresponding to the structured attribute in the record includes the structured data object The data corresponding to the attribute. Among them, the meaning of the second version can refer to the content introduced above. In addition, the method of generating the third key value according to the second version identifier and the first key value in the data corresponding to the KV attribute can refer to the above introduction, according to The manner in which the first key value in the data corresponding to the first version identifier and the KV attribute generates the second key value is not repeated here.

In an embodiment, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system. According to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute, the data management device generates a record of the data object in a relational data table corresponding to the data object. The method includes: The second version identifier and the first path in the data corresponding to the file attribute generate a third path; according to the data corresponding to the structured attribute in the record and the data corresponding to the file attribute in the record, the relationship between the data object corresponds to A record of the data object is generated in a data table; wherein the data corresponding to the file attribute in the record includes the third path, and the data corresponding to the structured attribute in the record includes the structured attribute of the data object The corresponding data. Wherein, the method of generating the third path according to the second version identifier and the first path in the data corresponding to the file attribute can refer to the above introduction, according to the first version identifier and the first path in the data corresponding to the file attribute The way the path generates the second path will not be repeated here.

The following uses the data object picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg)) in the relational data table Take the generation of records as an example to introduce the process of generating records of data objects in relational data tables. The unstructured attributes included in the data objects are file attributes (data/Snoopy.jpg, Snoopy.jpg) and KV attributes ( t_snoopy_key,t_snoopy.jpg). The update command is "update(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg)))", The data object contains 7 attributes, and the second version is identified as version2. Wherein, the first attribute of the data object is a structured attribute, and the data corresponding to the structured attribute in the record includes the data "Snoopy" corresponding to the structured attribute of the data object. In addition, other structured attributes other than the first structured attribute among the seven attributes in the data object can also refer to this method, which will not be repeated here. The third attribute of the data object is a file attribute, and the first path "data/Snoopy.jpg" in the data "(data/Snoopy.jpg,Snoopy.jpg)" corresponding to the second version identifier version2 and the file attribute is used. jpg" generates the third path "data/version2/Snoopy.jpg", and the data corresponding to the file attribute in the record is the third path "data/version2/Snoopy.jpg". The seventh attribute of the data object is the KV attribute, and the third key value is generated according to the first key value "t_snoopy_key" in the data "(t_snoopy_key,t_snoopy.jpg)" corresponding to the second version identifier version2 and the KV attribute "T_snoopy_key_version2", the data corresponding to the KV attribute in the record is the third key value "t_snoopy_key_version2". Then, according to the data corresponding to the structured attribute in the record, the data corresponding to the KV attribute in the record, and the data corresponding to the file attribute in the record, a record of the data object is generated in the relational data table corresponding to the data object . The record can refer to Table 5 below.

Table 5

Before the data object is updated, the record of the earlier version of the data object is already stored in the relational data table, so after the data management device receives the update instruction and generates the record of the updated version of the data object in the relational database, the The relational data table stored in the first storage system contains both the old and new version records, as shown in Table 6:

Table 6

Correspondingly, the data corresponding to the KV attributes stored in the KV storage system can be referred to Table 7:

Table Seven

属性名称Attribute name	键值Key value	缩略图Thumbnail
数据data	t_snoopy_key_version1t_snoopy_key_version1	t_snoopy.jpgt_snoopy.jpg
数据data	t_snoopy_key_version2t_snoopy_key_version2	t_snoopy.jpgt_snoopy.jpg

The data corresponding to the file attributes stored in the file storage system can be referred to Table 8:

Table 8

属性名称Attribute name	路径path	图片image
数据data	data/version1/Snoopy.jpgdata/version1/Snoopy.jpg	Snoopy.jpgSnoopy.jpg
数据data	data/version2/Snoopy.jpgdata/version2/Snoopy.jpg	Snoopy.jpgSnoopy.jpg

After the data management device stores the data corresponding to the unstructured attribute of the data object in the second storage system, the data management device will submit the database transaction corresponding to the update instruction. It should be noted that the operation of submitting database transactions by the data management device needs to be executed after the first storage system and the second storage system both save the corresponding data. This operation mode can ensure that the data object is successful in each storage system storage.

In an embodiment, the operation command received in step S303 is a query command, and the query command includes a query condition. In step S304, the process of determining the record of the data object from the first storage system includes: selecting a record of the data object satisfying the query condition from the first storage system.

For example, the query command is "query(picture("time_taken≥2018-10-12"))", and the meaning of the query command is to query pictures taken on October 12, 2018 or after the shooting time . Then the data management device traverses the relational data table corresponding to the picture in the first storage system, and selects records whose shooting time is greater than or equal to 2018-10-12. For example, the obtained record can be picture("Snoopy",2M,data/version1/Snoopy.jpg,39.92,116.46,"2018-10-12",t_snoopy_key_version1) and picture("Stitch",1.5M,data /version2/Stitch.jpg,38.23,129.78,"2018-10-17",t_Stitch_key_version2).

In one embodiment, the operation instruction received in step S303 is a delete instruction, and the delete instruction includes the object type of the data object, and the data corresponding to the structured attribute and the data corresponding to the unstructured attribute of the data object. . In step S304, the method for determining the record of the data object from the first storage system includes: determining a relational data table corresponding to the data object according to the object type; and determining the relational data table from the relational data table. For the record of the data object, the data corresponding to the structured attribute in the record is the same as the data corresponding to the structured attribute of the data object.

For example, the records stored in the database include: the record picture("Snoopy", 2M, data/version1/Snoopy.jpg, 39.92, 116.46, "2018-10-12", t_snoopy_key_version1) in the first relational data table , Picture("Stitch",1.5M,data/version2/Stitch.jpg,38.23,129.78,"2018-10-17",t_Stitch_key_version2) and the record video("Show",300M,data in the second relational data table /version1/Show.avi,47.56,119.73,"2018-10-23",t_Show_key_version1). Specifically, the delete instruction is "delete(picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg)) )". The meaning of this delete instruction is to delete the data object "picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg)) ". After receiving the delete instruction, the data management device determines that the relation data table corresponding to the data object is the first relation data table according to the object type picture. After that, the data management device determines the record of the data object from the first relational data table, and the data corresponding to the structured attribute in the record is the same as the data corresponding to the structured attribute of the data object. The determined record is picture("Snoopy", 2M, data/version1/Snoopy.jpg, 39.92, 116.46, "2018-10-12", t_snoopy_key_version1).

The following describes a detailed process in which the data management device obtains data corresponding to the multiple attributes of the data object from the first storage system and the second storage system according to the record in step S305.

In a possible implementation manner, the recorded unstructured attributes include KV attributes, and the second storage system is a KV storage system. The method for the data management device to obtain corresponding data in the multiple attributes of the data object from the first storage system and the second storage system according to the record includes: obtaining data from the second storage system according to a key value The KV data corresponding to the key value is read in the key value, and the version identifier in the key value is removed; the key value is the data corresponding to the KV attribute in the record, and the version identifier includes the first version identifier and the first version identifier. Two version identification; wherein the data corresponding to the KV attribute of the data object is the key value after removing the version identification and the KV data; the data corresponding to the structured attribute of the data object is, the structured data in the record The data corresponding to the attribute.

In another possible implementation manner, the unstructured attributes of the record include file attributes, and the second storage system is a file storage system. The data management device obtains data corresponding to multiple attributes of the data object from the first storage system and the second storage system according to the record, including: reading from the second storage system according to a path Fetch the file data corresponding to the path, and remove the version identifier in the path; the path is the data corresponding to the file attribute in the record, and the version identifier includes a first version identifier and a second version identifier; where The data corresponding to the file attribute of the data object is the path after removing the version identifier and the file data; the data corresponding to the structured attribute of the data object is the data corresponding to the structured attribute in the record.

The following will take the record picture("Snoopy",2M,data/version1/Snoopy.jpg,39.92,116.46,"2018-10-12",t_snoopy_key_version1) as an example to introduce the data management device from the first A process of acquiring data corresponding to multiple attributes of the data object in a storage system and the second storage system.

Among them, the first attribute in the record is a structured attribute, and the data corresponding to the structured attribute of the data object is the data "Snoopy" corresponding to the first attribute. In addition, the 7 attributes in the record are divided by Other structured attributes of the first structured attribute can also refer to this method, which will not be repeated here. The third attribute in the record is a path, then read the file data “Snoopy.jpg” corresponding to the path from the file storage system according to the path “data/version1/Snoopy.jpg”, and remove the Version ID version1, the data corresponding to the file attribute of the data object is the path "data/Snoopy.jpg" after removing the version ID and the file data "Snoopy.jpg", that is, "(data/Snoopy.jpg, Snoopy.jpg" )". The seventh attribute in the record is a key value, then read the KV data “t_snoopy” corresponding to the key value from the KV storage system according to the key value “t_snoopy_key_version1”, and remove the version identifier version1 in the key value, The data corresponding to the KV attribute of the data object is the key value "t_snoopy_key" after removing the version identifier and the KV data "t_snoopy", that is, "(t_snoopy_key, t_snoopy.jpg)".

In the embodiment in which the operation instruction listed above is a delete instruction, the multiple attributes of the data object are obtained from at least one of the first storage system and the second storage system according to the record The data corresponding to the at least one attribute in the data includes: obtaining the data corresponding to the multiple attributes of the data object from the first storage system according to the record. For example, if the record is picture("Snoopy",2M,data/version1/Snoopy.jpg,39.92,116.46,"2018-10-12",t_snoopy_key_version1), then the data management device obtains the data object Data corresponding to multiple data: "Snoopy", 2M, data/version1/Snoopy.jpg, 39.92, 116.46, "2018-10-12", t_snoopy_key_version1.

In one embodiment, in step S306, if the operation instruction is a query instruction, based on the data corresponding to the at least one attribute, the process of performing the operation on the data object includes: generating a query result according to the at least one attribute , And return the query result to the request initiator, such as an application.

Taking the foregoing example as an example, the data management device creates the data object according to the data corresponding to each of the seven attributes in the data object and the sequence of the seven attributes in the record. The data object is "picture("Snoopy",2M,(data/Snoopy.jpg,Snoopy.jpg),39.92,116.46,"2018-10-12",(t_snoopy_key,t_snoopy.jpg))". In a similar way, the data management device can also create a data object picture("Stitch",1.5M,(data/Stitch.jpg,Stitch.jpg),38.23,129.78,"2018-10-17",(t_Stitch_key_version2, t_Stitch.jpg)). After that, the data management device uses these two data objects as query results.

In one embodiment, in step S306, if the operation instruction is a delete instruction, based on the data corresponding to the at least one attribute, the method for performing the operation on the data object is: deleting from the first storage system Data corresponding to the at least one attribute; commit the transaction corresponding to the delete instruction. Taking the foregoing example as an example, the data management device deletes the data corresponding to the multiple data of the data object in the first relational data table of the first storage system: "Snoopy", 2M, data/version1/Snoopy.jpg ,39.92,116.46,"2018-10-12",t_snoopy_key_version1. After the deletion, the data management device submits the transaction corresponding to the deletion instruction.

In a possible implementation manner, if an update instruction for a data object is received, before the data management device submits the database transaction, there are two corresponding records for a data object in the database. Based on the multiversion currency control (MVCC) mechanism of the database, the database only keeps updated records.

This MVCC mechanism will be further introduced below. The MVCC mechanism can maintain multiple snapshot copies for each record in the database, and maintain the visibility of the copies through a start timestamp (begin timestamp) and an end timestamp (end timestamp). Among them, the start timestamp is used to indicate when a record is created, and the end timestamp is used to indicate when a record expires (or is deleted). It should be noted that the timestamp does not store the actual time when a record was created or expired, it stores the system version number when the record occurred. The system version number will continue to grow as the transaction is created, and each transaction will record its own system version number at the beginning of the transaction.

In one embodiment, during the execution of the insert instruction, the start timestamp of the first record corresponding to the data object is the system version number of the current storage transaction, and the end timestamp of the first record is undefined. In the process of executing the update instruction, the start timestamp of the second record corresponding to the updated data object is the system version number of the current update transaction, and the end timestamp of the second record is undefined; wherein, the system version of the update transaction The number is greater than the system version number of the stored transaction. In addition, the end timestamp of the first record will be defined as the system version number of the update transaction. When the data management device executes the update instruction and commits the database transaction, the first record will be deleted, that is, after the data management device commits the database transaction, only the updated record is kept in the database. In addition, if another transaction performs read access to the data object while the update transaction is being performed, the record of the data object read by the other transaction is the first record. This method can make the database update and Reading does not block each other.

For example, in the process of executing the update instruction by the data management device, the records stored in the database can refer to the content of Table 6 above. Among them, there are two corresponding records for a data object in the database, which are the record containing the first version identifier and the record containing the second version identifier. Based on the multi-version concurrency control mechanism of the database, after the data management device submits the database transaction, only the records containing the second version identifier are kept in the database. At this time, the records stored in the database can refer to the contents of Table 3 above. If during the execution of the update instruction, another transaction performs read access to the data object, the record of the data object read by the other transaction is a record containing the first version identifier.

After the update instruction is executed and the database commits the database transaction, there is a record corresponding to the data object in the database. Taking the update process exemplified above as an example, the records stored in the final database can refer to the contents shown in Table 3 . However, for KV storage systems and file storage systems, there will be two types of unstructured attribute data of the data object. For the KV storage system, the unstructured attribute data corresponding to the data object includes "t_snoopy_key_version1, t_snoopy.jpg" and "t_snoopy_key_version2, t_snoopy.jpg"; for the file storage system, the data object corresponds to the unstructured The attribute data are "data/version1/Snoopy.jpg, Snoopy.jpg" and "data/version2/Snoopy.jpg, Snoopy.jpg". Among them, "t_snoopy_key_version1, t_snoopy.jpg" and "data/version1/Snoopy.jpg, Snoopy.jpg" are invalid data. This invalid data can be cleaned up through the verification operation. The following will execute the data management device The method of verification operation is introduced.

In an embodiment, the unstructured attribute includes a KV attribute, and the second storage system is a KV storage system. The method for performing a verification operation includes: when a verification instruction is received or when a verification condition is detected to be satisfied, traversing the key value in the second storage system, and the key value is in the second storage system The data corresponding to the KV attribute of the; in the process of traversing the key value, if the same key value as the fourth key cannot be found in the records stored in the first storage system, then the second storage system Delete the fourth key value and the KV data corresponding to the fourth key value, and the fourth key value is one of multiple key values in the second storage system.

Taking the updated data object mentioned above as an example, there is a fourth key value "t_snoopy_key_version1" in the KV storage system, but the same key value as the fourth key cannot be found in the records stored in the database, then Delete the fourth key value "t_snoopy_key_version1" and the KV data "t_snoopy.jpg" corresponding to the fourth key value from the KV storage system.

In an embodiment, the unstructured attributes include file attributes, and the second storage system is a file storage system. The method for performing a verification operation includes: when a verification instruction is received or when a verification condition is detected to be satisfied, traversing a path in the second storage system, where the path is a file in the second storage system Data corresponding to the attribute; in the process of traversing the path, if the same path as the fourth path cannot be found in the relational data table stored in the first storage system, delete all paths in the second storage system The fourth path and the file data corresponding to the fourth path, and the fourth path is one of multiple paths in the second storage system.

Taking the updated data object mentioned above as an example, there is a fourth path "data/version1/Snoopy.jpg" in the file storage system, but the same path as the fourth path cannot be found in the records stored in the database , Delete the fourth path "data/version1/Snoopy.jpg" and the file data "Snoopy.jpg" corresponding to the fourth path in the file storage system.

Wherein, the verification condition may be a preset verification period at the current moment, or the amount of data stored in the data management device is greater than a preset value, and so on. For example, the check instruction may be "check (picture)", which means to check a data object of the object type "picture". When the data management device receives the verification instruction, the data management device will traverse the data corresponding to the non-structural attributes in the second storage system. In this way, by comparing the records stored in the database with the data corresponding to the unstructured attributes in the second storage system, invalid data in the second storage system can be cleared, and data objects can be stored across multiple data systems. To maintain data consistency.

It should be noted that the method for the data management device to delete the data corresponding to the unstructured attribute corresponding to the first record from the second storage system may refer to the method for executing the verification instruction of the data management device. Since the first record has been deleted, the unstructured data corresponding to the first record in the second storage system can be cleared by executing the check instruction.

The method embodiments of the present application are described above, and the device embodiments implementing the above methods are introduced below.

4, it is a data management device provided by an embodiment of the present application. The device includes a data management device including a generating unit 401, a storage unit 402, a receiving unit 403, a determining unit 404, an obtaining unit 405, and an operating unit 406. The generating unit 401, storage unit 402, receiving unit 403, determining unit 404, obtaining unit 405, and operating unit 406 will be introduced below.

The generating unit 401 is configured to generate a record of a data object in a relational data table, the data object has multiple attributes, and the multiple attributes include structured attributes and unstructured attributes, and the record indicates the The relationship between the structured attribute and the unstructured attribute of the data object, and the relational data table is stored in the first storage system. For the operations performed by the generating unit 401, reference may be made to the related description in step 301 in FIG. 3 above.

The storage unit 402 is configured to store data corresponding to the unstructured attributes of the data object in the second storage system. For operations performed by the storage unit 402, reference may be made to the related description in step 302 of FIG. 3 above.

The receiving unit 403 is configured to receive an operation instruction, and the operation instruction is used to perform an operation on the data object. In an embodiment, the receiving unit 403 may be a circuit or component that can be configured to receive information, such as a data transmission interface, a communication interface, or a receiver. For operations performed by the receiving unit 403, refer to step 303 in FIG. 3 above. Related description.

The determining unit 404 is configured to determine the record of the data object from the first storage system in response to the operation instruction. For the operation performed by the determining unit 404, reference may be made to the related description in step 304 in FIG. 3 above.

The acquiring unit 405 is configured to acquire at least one attribute corresponding to at least one of the multiple attributes of the data object from at least one of the first storage system and the second storage system according to the record The data. For the operations performed by the obtaining unit 405, reference may be made to the related description in step 305 of FIG. 3 above.

The operation unit 406 is configured to perform the operation on the data object based on the data corresponding to the at least one attribute. For the operations performed by the obtaining unit 406, reference may be made to the related description in step 306 in FIG.

In addition, the specific implementation details of each operation in FIG. 4 may also correspond to the corresponding description of the method embodiment shown in FIG. 3. The above-mentioned units can be implemented in hardware, software or a combination of software and hardware. In one embodiment, the generation unit 401, the storage unit 402, the determination unit 404, the acquisition unit 405, and the operation unit 406 may be functional modules implemented by software. The functions of these functional modules are implemented by programs or codes stored in the memory. The management device executes these programs or codes through at least one processor to realize the functions of each functional module. Since the data corresponding to the multiple attributes of the data object stored across multiple data systems are obtained through the records in the relational data table, the data management device can allow the data object to maintain data consistency when stored across multiple data systems .

Refer to FIG. 5, which is another data management device provided by an embodiment of the present application. The data management device includes a processor 501, a memory 502, and a communication interface 503. The processor 501, the memory 502, and the communication interface 503 are connected to each other through a bus 504.

The memory 502 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (erasable programmable read-only memory, EPROM), or Portable read-only memory (compact disc read-only memory, CD-ROM), the memory 502 is used for related instructions and data.

The communication interface 503 may be a circuit or component that can be configured to receive information, such as a data transmission interface, a communication interface, or a receiver.

The processor 501 may be one or more central processing units (CPU). In the case where the processor 501 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

The processor 501 in the data management device performs the following operations by reading and executing the program code stored in the memory 502:

Generate records of data objects in relational data tables. Wherein, the data object has multiple attributes, and the multiple attributes include structured attributes and unstructured attributes, and the record indicates the association relationship between the structured attributes and unstructured attributes of the data object. The relational data table is stored in the first storage system.

The data corresponding to the unstructured attribute of the data object is stored in the second storage system.

An operation instruction is received, and the operation instruction is used to perform an operation on the data object.

In response to the operation instruction, the record of the data object is determined from the first storage system.

Obtain data corresponding to at least one of the multiple attributes of the data object from at least one of the first storage system and the second storage system according to the record.

Perform the operation on the data object based on the data corresponding to the at least one attribute.

For specific details of the operations performed by the processor 501 in FIG. 5, reference may also be made to the corresponding description of the method embodiment shown in FIG. 3. Because the data corresponding to the multiple attributes of the data object stored across multiple data systems are all obtained through the records in the relational data table, the data object can maintain data consistency when stored across multiple data systems.

In another embodiment of the present application, a computer program product is provided. When the computer program product runs on a computer, the method of the embodiment shown in FIG. 3 is implemented.

In another embodiment of the present application, a computer-readable storage medium is provided, the computer-readable storage medium stores a computer program, and the computer program implements the method of the embodiment shown in FIG. 3 when the computer program is executed by a computer.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Anyone familiar with the technical field can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

A data management method, characterized in that the method includes:

A record of a data object is generated in a relational data table. The data object has multiple attributes. The multiple attributes include structured attributes and unstructured attributes. The records contain data corresponding to the structured attributes, and The association relationship between the structured attribute and the unstructured attribute of the data object, and the relational data table is stored in the first storage system;

Storing the data corresponding to the unstructured attributes of the data object in the second storage system;

Receiving an operation instruction, the operation instruction being used to perform an operation on the data object;

In response to the operation instruction, determining the record of the data object from the first storage system;

Acquiring data corresponding to at least one attribute of the plurality of attributes of the data object from at least one of the first storage system and the second storage system according to the record;

Perform the operation on the data object based on the data corresponding to the at least one attribute.
The method according to claim 1, wherein said generating a record of a data object in a relational data table comprises:

Receiving an insert instruction or an update instruction, the insert instruction is used to insert the data object, the update instruction is used to update the data object; both the insert instruction and the update instruction include the object type of the data object, And data corresponding to the structured attribute and data corresponding to the unstructured attribute of the data object;

Determining the relational data table corresponding to the data object according to the object type;

Generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute;

Submit the transaction corresponding to the insert instruction or update instruction;

Wherein, the transaction corresponding to the insert instruction or the update instruction is submitted after the data corresponding to the unstructured attribute of the data object is stored in the second storage system.
The method according to claim 2, wherein the received instruction is the insert instruction, the unstructured attribute of the data object includes a key value KV attribute, and the second storage system is a KV storage system;

The generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute includes:

Generating a second key value according to the first version identifier and the first key value in the data corresponding to the KV attribute;

Generate a record of the data object, wherein the data corresponding to the KV attribute in the record includes the second key value, and the data corresponding to the structured attribute in the record includes the data corresponding to the structured attribute of the data object data.
The method according to claim 2, wherein the received instruction is the insert instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system;

The generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute includes:

Generating a second path according to the first path in the data corresponding to the first version identifier and the file attribute;

A record of the data object is generated in a relational data table corresponding to the data object; wherein the data corresponding to the file attribute in the record includes the second path, and the data corresponding to the structured attribute in the record includes Data corresponding to the structured attributes of the data object.
The method according to claim 2, wherein the received instruction is the update instruction, the unstructured attributes of the data object include KV attributes, and the second storage system is a KV storage system;

The generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute includes:

Generating a third key value according to the second version identifier and the first key value in the data corresponding to the KV attribute;

A record of the data object is generated in the relational data table corresponding to the data object; wherein the data corresponding to the KV attribute in the record includes the third key value, and the data corresponding to the structured attribute in the record Including data corresponding to the structured attributes of the data object.
The method according to claim 2, wherein the received instruction is the update instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system;

The generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute includes:

Generating a third path according to the second version identifier and the first path in the data corresponding to the file attribute;

The record of the data object is generated in the relational data table corresponding to the data object; wherein the data corresponding to the file attribute in the record includes the third path, and the data corresponding to the structured attribute in the record includes Data corresponding to the structured attributes of the data object.
The method according to any one of claims 3 to 6, wherein the data corresponding to the unstructured attribute stored in the second storage system includes the identifier and content of the unstructured attribute; the relationship data The data corresponding to the unstructured attribute stored in the table includes the identifier of the unstructured attribute.
The method according to any one of claims 1 to 7, wherein the operation instruction includes a query instruction, and the query instruction includes a query condition;

The determining the record of the data object from the first storage system includes:

Selecting a record of a data object that meets the query condition from the first storage system;

The obtaining data corresponding to at least one of the multiple attributes of the data object from at least one of the first storage system and the second storage system according to the record includes:

Acquiring data corresponding to the multiple attributes of the data object from the first storage system and the second storage system according to the record;

The performing the operation on the data object based on the data corresponding to the at least one attribute includes:

The query result is returned according to the acquired data corresponding to the multiple attributes.
The method according to claim 8, wherein the unstructured attribute of the record comprises a KV attribute, and the data is obtained from the first storage system and the second storage system according to the record Corresponding data in multiple attributes of the object, including:

The KV data corresponding to the key value is read from the second storage system according to the key value, and the version identifier in the key value is removed; the key value is the data corresponding to the KV attribute in the record, so The version identifier includes a first version identifier and a second version identifier;

The data corresponding to the KV attribute of the data object includes the key value after removing the version identifier and the KV data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.
8. The method according to claim 8, wherein the unstructured attributes of the record include file attributes, and the data is obtained from the first storage system and the second storage system according to the record Corresponding data in multiple attributes of the object, including:

The file data corresponding to the path is read from the second storage system according to the path, and the version identifier in the path is removed; the path is the data corresponding to the file attribute in the record, and the version identifier includes The first version identification and the second version identification;

Wherein, the data corresponding to the file attribute of the data object includes the path after removing the version identifier and the file data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.
The method according to any one of claims 1 to 7, wherein the operation instruction includes a delete instruction, and the delete instruction includes an object type of the data object, and a structured attribute corresponding to the data object. Data and data corresponding to unstructured attributes;

Wherein, said determining said record of said data object from said first storage system includes:

Determining the relational data table corresponding to the data object according to the object type;

Determine the record of the data object from the relational data table, and the data corresponding to the structured attribute in the record is the same as the data corresponding to the structured attribute of the data object;

The obtaining data corresponding to at least one attribute of the multiple attributes of the data object from at least one of the first storage system and the second storage system according to the record includes:

Acquiring data corresponding to the multiple attributes of the data object from the first storage system according to the record;

The performing the operation on the data object based on the data corresponding to the at least one attribute includes:

Deleting data corresponding to the multiple attributes of the data object from the first storage system;

Submit the transaction corresponding to the delete instruction.
The method according to any one of claims 1 to 7, wherein the unstructured attribute includes a KV attribute, the second storage system is a KV storage system, and the method further comprises:

When a verification instruction is received or when it is detected that a verification condition is met, traversing the key value in the second storage system, where the key value is the data corresponding to the KV attribute in the second storage system;

In the process of traversing the key value, if the same key value as the fourth key value cannot be found in the relational data table stored in the first storage system, the first storage system will delete the first value. Four key values and KV data corresponding to the fourth key value, where the fourth key value is one of multiple key values in the second storage system.
The method according to any one of claims 1 to 7, wherein the unstructured attributes include file attributes, the second storage system is a file storage system, and the method further comprises:

When a verification instruction is received or when a verification condition is detected to be satisfied, traverse a path in the second storage system, where the path is the data corresponding to the file attribute in the second storage system;

In the process of traversing the path, if the same path as the fourth path cannot be found in the relational data table stored in the first storage system, delete the fourth path and the fourth path in the second storage system. File data corresponding to the fourth path, where the fourth path is one of multiple paths in the second storage system.
The method according to any one of claims 2 to 6, wherein before determining the relational data table corresponding to the data object according to the object type, the method further comprises:

Receiving a definition instruction for the object type to which the data object belongs, where the definition instruction includes definition information of the object type, and the definition information is used to define the structure of a relational data table of the object type;

According to the definition instruction, a relational data table of the object type is generated in the first storage system.
The method according to any one of claims 2 to 6, wherein the determining the relational data table corresponding to the data object according to the object type comprises:

Determine the object type to which the data object belongs according to the insert instruction or update instruction;

The relational data table corresponding to the data object is determined according to the object type.
A data management device, characterized in that the data management device includes a generating unit, a storage unit, a receiving unit, a determining unit, an acquiring unit, and an operating unit:

The generating unit is configured to generate a record of a data object in a relational data table, the data object has multiple attributes, and the multiple attributes include structured attributes and unstructured attributes, and the record contains the structured attributes. Data corresponding to the attribute, and the association relationship between the structured attribute and the unstructured attribute of the data object, the relationship data table is stored in the first storage system;

The storage unit is configured to store data corresponding to the unstructured attributes of the data object in a second storage system;

The receiving unit is configured to receive an operation instruction, and the operation instruction is used to perform an operation on the data object;

The determining unit is configured to determine the record of the data object from the first storage system in response to the operation instruction;

The acquiring unit is configured to acquire at least one attribute corresponding to at least one of the multiple attributes of the data object from at least one of the first storage system and the second storage system according to the record data;

The operation unit is configured to perform the operation on the data object based on the data corresponding to the at least one attribute.
The data management device according to claim 16, wherein the generating unit is specifically configured to:

Receiving an insert instruction or an update instruction, the insert instruction is used to insert the data object, the update instruction is used to update the data object; both the insert instruction and the update instruction include the object type of the data object, And data corresponding to the structured attribute and data corresponding to the unstructured attribute of the data object;

Determining the relational data table corresponding to the data object according to the object type;

Generating a record of the data object in a relational data table corresponding to the data object according to the data corresponding to the structured attribute and the data corresponding to the unstructured attribute;

Submit the transaction corresponding to the insert instruction or update instruction;

Wherein, the transaction corresponding to the insert instruction or the update instruction is submitted after the data corresponding to the unstructured attribute of the data object is stored in the second storage system.
The data management device according to claim 17, wherein the received instruction is the insert instruction, the unstructured attribute of the data object includes a key value KV attribute, and the second storage system is a KV storage system;

The generating unit is specifically used for:

Generating a second key value according to the first version identifier and the first key value in the data corresponding to the KV attribute;

A record of the data object is generated in a relational data table corresponding to the data object; wherein the data corresponding to the KV attribute in the record includes the second key value, and the data corresponding to the structured attribute in the record Including data corresponding to the structured attributes of the data object.
The data management device according to claim 17, wherein the received instruction is the insert instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system;

The generating unit is specifically used for:

Generating a second path according to the first path in the data corresponding to the first version identifier and the file attribute;

The record of the data object is generated in the relational data table corresponding to the data object; wherein the data corresponding to the file attribute in the record includes the second path, and the data corresponding to the structured attribute in the record includes Data corresponding to the structured attributes of the data object.
The data management device according to claim 17, wherein the received instruction is the update instruction, the unstructured attributes of the data object include KV attributes, and the second storage system is a KV storage system;

The generating unit is specifically used for:

Generating a third key value according to the second version identifier and the first key value in the data corresponding to the KV attribute;

A record of the data object is generated in the relational data table corresponding to the data object; wherein the data corresponding to the KV attribute in the record includes the third key value, and the data corresponding to the structured attribute in the record Including data corresponding to the structured attributes of the data object.
The data management device according to claim 17, wherein the received instruction is the update instruction, the unstructured attributes of the data object include file attributes, and the second storage system is a file storage system;

The generating unit is specifically used for:

Generating a third path according to the second version identifier and the first path in the data corresponding to the file attribute;

The record of the data object is generated in the relational data table corresponding to the data object; wherein the data corresponding to the file attribute in the record includes the third path, and the data corresponding to the structured attribute in the record includes Data corresponding to the structured attributes of the data object.
The data management device according to any one of claims 18 to 21, wherein the data corresponding to the unstructured attribute stored in the second storage system includes the identifier and content of the unstructured attribute; the The data corresponding to the unstructured attribute stored in the relational data table includes the identifier of the unstructured attribute.
The data management device according to any one of claims 16 to 22, wherein the operation instruction includes a query instruction, and the query instruction includes a query condition;

The determining unit is specifically used for:

In response to the operation instruction, select a record of a data object that meets the query condition from the first storage system;

The acquiring unit is specifically used for:

Acquiring data corresponding to the multiple attributes of the data object from the first storage system and the second storage system according to the record;

The operating unit is specifically configured to return query results according to the acquired data corresponding to the multiple attributes.
The data management device according to claim 23, wherein the recorded unstructured attributes include KV attributes, the second storage system is a KV storage system, and the obtaining unit is specifically configured to:

The KV data corresponding to the key value is read from the second storage system according to the key value, and the version identifier in the key value is removed; the key value is the data corresponding to the KV attribute in the record, so The version identifier includes a first version identifier and a second version identifier;

The data corresponding to the KV attribute of the data object includes the key value after removing the version identifier and the KV data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.
The data management device according to claim 23, wherein the recorded unstructured attributes include file attributes, the second storage system is a file storage system, and the acquiring unit is specifically configured to:

The file data corresponding to the path is read from the second storage system according to the path, and the version identifier in the path is removed; the path is the data corresponding to the file attribute in the record, and the version identifier includes The first version identification and the second version identification;

Wherein, the data corresponding to the file attribute of the data object includes the path after removing the version identifier and the file data; the data corresponding to the structured attribute of the data object includes the data corresponding to the structured attribute in the record.
The data management device according to any one of claims 16 to 22, wherein the operation instruction includes a delete instruction, and the delete instruction includes an object type of the data object and a structured attribute of the data object Corresponding data and data corresponding to unstructured attributes;

Wherein, the determining unit is specifically used for:

Determining the relational data table corresponding to the data object according to the object type;

Determine the record of the data object from the relational data table, and the data corresponding to the structured attribute in the record is the same as the data corresponding to the structured attribute of the data object;

The acquiring unit is specifically used for:

Acquiring data corresponding to the multiple attributes of the data object from the first storage system according to the record;

The operating unit is specifically used for:

Deleting data corresponding to the multiple attributes of the data object from the first storage system;

Submit the transaction corresponding to the delete instruction.
The data management device according to any one of claims 16 to 22, wherein the unstructured attribute includes a KV attribute, the second storage system is a KV storage system, and the data management device further includes a check unit:

The verification unit is configured to traverse the key value in the second storage system when the verification instruction is received or when it is detected that the verification condition is satisfied, and the key value is the value in the second storage system. The data corresponding to the KV attribute;

In the process of traversing the key value, if the same key value as the fourth key value cannot be found in the relational data table stored in the first storage system, the first storage system will delete the first value. Four key values and KV data corresponding to the fourth key value, where the fourth key value is one of multiple key values in the second storage system.
The data management device according to any one of claims 16 to 22, wherein the unstructured attributes include file attributes, the second storage system is a file storage system, and the data management device further includes verification unit:

The verification unit is configured to traverse a path in the second storage system when a verification instruction is received or when a verification condition is detected to be satisfied, and the path is a file attribute in the second storage system Corresponding data;

In the process of traversing the path, if the same path as the fourth path cannot be found in the relational data table stored in the first storage system, delete the fourth path and the fourth path in the second storage system. File data corresponding to the fourth path, where the fourth path is one of multiple paths in the second storage system.
The data management device according to any one of claims 17-22, wherein the generating unit is further configured to:

Receiving a definition instruction for the object type to which the data object belongs, where the definition instruction includes definition information of the object type, and the definition information is used to define the structure of a relational data table of the object type;

According to the definition instruction, a relational data table of the object type is generated in the first storage system.
The data management device according to any one of claims 17-22, wherein the generating unit is specifically configured to:

Determine the object type to which the data object belongs according to the insert instruction or update instruction;

The relational data table corresponding to the data object is determined according to the object type.
A data management device, characterized by comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to execute any one of claims 1-15 according to the program instructions method.
A computer-readable storage medium, wherein the computer storage medium stores program instructions that when executed by a computer cause the computer to execute the method according to any one of claims 1-15.