WO2018205981A1 - Metadata management - Google Patents

Abstract

A master node in an HBase architecture detects an operating command for a table; the master node parses the operating command and determines whether the operating command changes the table structure of the table; the change comprises at least one of adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and deleting a column; when determining that the operating command changes the table structure of the table and that the change comprises adding a table, adding a column family, or adding a column, the master node records the table name of the added table, the column family name of the added column family, or the column name of the added column; and when determining that the operating command changes the table structure of the table and that the change comprises deleting a table, deleting a column family, or deleting a column, the master node deletes the records comprising the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column.

Description

Metadata management

Cross-reference to related applications

The present disclosure is based on and claims the priority of the Chinese Patent Application No.

Background technique

The Hadoop Database (HBase) is a distributed, scalable, non-relational (NoSQL, Not Only SQL) database for columnar storage built on the Hadoop infrastructure. Like a traditional relational database, HBase's Table is composed of rows and columns, but HBase can store values at different times, and multiple columns can form a Column Family. Therefore, the data model defined by HBase has a large difference from the data model of the traditional relational database (the two-dimensional model composed of rows and columns), which actually defines a four-dimensional data model.

Meta Data is data about data. In a database system, metadata is data that describes the structure and method of building data in a database. It helps database administrators and database developers to easily find the information they care about. data.

DRAWINGS

1 is a schematic diagram of an architecture of an HBase example of the present disclosure;

2 is a flow chart of an example of a method of the present disclosure;

3 is a hardware architecture diagram of an example of an apparatus of the present disclosure;

4 is a functional block diagram of an example of an apparatus of the present disclosure;

FIG. 5 is a schematic diagram of an architecture of an example of a web application of the present disclosure.

detailed description

The present disclosure is described in detail below with reference to the drawings and specific embodiments.

The HBase table (for simplicity of description, the table referred to as the table in the present disclosure, the tables referred to in the subsequent description all refer to the HBase table) has at least the following characteristics: 1) Each row of the table has a sortable row key and any number of The columns and columns can be dynamically increased as needed. Different rows in the same table can have different columns. 2) The data in each cell of the table can have multiple versions. By default, the version number is automatically assigned. For example, it can be the timestamp when the cell is inserted; 3) the data type is single, that is, the data in the table is a string, there is no type; 4) the size of the table is large, one table can have billions of rows, millions of columns .

HBase can be used better with metadata. Currently HBase can support the management of some metadata, such as the management of metadata indicating the storage location of the table. However, there are some metadata, HBase can not provide a unified management means, such as HBase does not support the management of the table's business description and table structure metadata.

In the present disclosure, the coprocessor function possessed by HBase itself is used to realize automatic collection and storage of metadata corresponding to the table structure of the table, and an external interface is provided to collect service descriptions of tables and columns, which promotes table structure and service description. These two types of metadata range from unmanageable to managed change.

Introduce HBase as follows. The architecture of the HBase is as shown in FIG. 1, and includes: a client 201, a master node 202, a region server (Region Server) 203, and a coordination server (Zookeeper) 204.

The client 201 is an interface for the user to access the HBase.

The Master node 202 is connected to each Region Server 203 and is responsible for managing each Region Server 203, such as a Region for the Region Server 203, and load balancing for the Region Server 203. The master node 202 can correspond to a physical device in an actual application, and the physical device can be an X86 architecture device, for example, an X86 server.

Each Region Server 203 maintains its assigned Regions, handling input/output (I/O) requests to these Regions.

The Zookeeper 204 ensures that there is only one master node 202 in the cluster at any time, stores the location of each Region Server 203, monitors the status of each Region Server 203 in real time, and notifies the master node 202 in real time, and also stores other information required for HBase operation. .

In the present disclosure, a metadata collection function may be deployed at the master node 202, and the master node may execute the flow shown in FIG. 2 upon receiving an operation instruction for the table. The deployment of the metadata collection function may be performed in the following manner: the package for collecting metadata is deployed on the master node 202, as shown in FIG. 1, and then HBase can be invoked at a specific timing by using the observer function of the coprocessor. The package is executed and implemented to realize automatic collection of metadata corresponding to the table structure.

Referring to FIG. 2, the metadata management method provided by the present disclosure may include steps 300, 301, 302, and 303.

In step 300, the master node detects an operation instruction for the table.

Step 301: The master node parses the operation instruction, and determines whether the operation instruction changes a table structure of the table, where the change includes adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and deleting a column. At least one is performed; if yes, step 302 is performed; otherwise, step 303 is performed.

The operation instructions herein may include a Data Manipulation Language (DML) operation instruction, a Data Definition Language (DDL) operation instruction, and the like.

The DDL operation instruction for the HBase may include: creating a new table, enumerating the table information, obtaining the table description, deleting the column family, deleting the table, and the like.

Regarding the DML operation instruction for HBase, it may include: adding a record, viewing a record, viewing the total number of records in the table, deleting a record, and the like.

Based on the above description of the DDL operation instruction and the DML operation instruction, operations that may affect the structure of the table include: creating a new table, deleting a column family, deleting a table, adding a record, and the like. Currently HBase does not support deleting columns, but considering the speed of HBase development, the operation of deleting columns may be implemented in the future.

Based on this, the changes to the table structure of the table may include the following:

1) Added a new table;

2) A new column family has been added to the table;

3) A new column is added under the column family of the table;

4) deleted the table;

5) Delete the existing column family in the table;

6) Delete the existing column under the column family of the table.

The master node can use the keyword identification to determine whether the received operation instruction changes the table structure of the table. For example, if the Master node recognizes a Drop statement (used to delete a table) or a Create statement (used to create a table) in the received operation instruction, it can be considered that the operation instruction changes the table structure; if the Master node is receiving If the Alter statement is recognized in the operation instruction (can be used to delete or add a column family), it can be considered that the operation instruction may change the table structure. It is necessary to further determine whether the Alter statement is used to delete or add a column family. If yes, It is determined that the operation instruction changes the table structure; if the Master node recognizes the Put statement (can be used to add a column) in the received operation instruction, it can be considered that the operation instruction may change the table structure, and the Put statement needs to be further determined. Whether to increase the column, if it is, you can determine that the operation instruction changes the table structure.

Step 302: When determining that the operation instruction changes the table structure of the table, the master node records the table name of the added table, the column family name of the added column family, or the column name of the added column, or deletes the table including the deleted table. The name, the column family name of the deleted column family, or the record of the column name of the deleted column. Specifically, if the change includes adding a table, adding a column family, or adding a column, the Master node records the table name of the added table, the column family name of the added column family, or the column name of the added column. If the change includes deleting the table, deleting the column family, or deleting the column, the Master node deletes the record containing the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column.

The master node may record the table name of the added table, the column family name of the added column family, or the column name of the added column in a pre-established two-dimensional table, and/or delete the table containing the deletion from the two-dimensional table. The table name, the column family name of the deleted column family, or the record of the column name of the deleted column. The two-dimensional table described above can be used to reflect the table structure of the table in real time, and the two-dimensional table can include at least the following fields: table name, column family name, and column name. Table 1 shows an example of the structure of the two-dimensional table. The two-dimensional table can be used to quickly obtain an accurate table structure, without having to traverse all the records of the table. For example, by searching for "T1" in Table 1, you can quickly find that "T1" contains the columns A, B, and C under the table.

Table 1 Two-dimensional representation

Table Name	Column family name	Column name
T1
T2	F1
T1	F1	A
T1	F1	B
T1	F2	C

The two-dimensional table can be built in a relational database, such as Access, Oracle, etc.; or, the two-dimensional table can also be built in a text file, such as Extensible Markup Language (XML), EXCEL, etc. In the text.

Further, the storage form of the record including the name of the table, the column family, or the column is not limited to the two-dimensional table, and may be stored in other manners such as text. Based on the received operation instruction, the master node correspondingly modifies the metadata related to the table structure in the stored record, for example, adding or deleting a table name, a column family name, and a column name, so that the stored record can reflect the table structure of the table in real time. .

After determining that the received operation instruction changes the table structure of the table, in step 302, the master node may record the metadata corresponding to the changed table structure into the two-dimensional table by:

1) If the operation instruction is used to add a table, a record containing the table name of the added table may be added to the two-dimensional table. In practical applications, the Create statement is usually used to add a table. For example, if an operation instruction indicates that the table "T1" is created, a record as shown in the first line in Table 1 can be added.

2) If the operation instruction is used to delete the table, one or more records containing the table name of the table may be deleted in the two-dimensional table. In practical applications, the Drop statement is usually used to delete a table. For example, if an operation instruction instructs deletion of the table "T1", the records shown in the first row, the third row, the fourth row, and the fifth row can be deleted in Table 1 shown above.

3) If the operation instruction is used to add a column family to the specified table, a record can be added to the two-dimensional table, including the column family name of the added column family and the table name of the table in which the column family is located. In practical applications, the Alter statement is usually used to increase the column family. For example, if an operation instruction indicates that the column family "F1" is created in the table "T2", then a record as shown in the second row in Table 1 can be added.

4) If the operation instruction is used to delete the specified column family in the specified table, the record containing the column family name of the specified column family and the table name of the table to which the specified column family belongs may be deleted in the two-dimensional table. In practical applications, the Alter statement is usually used to delete the column family in the table. For example, if an operation instruction instructs deletion of the column family "F1" in the table "T1", the records shown in the third row and the fourth row can be deleted in Table 1 shown above.

5) If the operation instruction is used to add a column under the specified column family of the specified table, you can add a record to the two-dimensional table, the record includes the column name of the added column, the table name of the table in which the column is located, and the column of the column The family name of the family. In practical applications, the Put statement is usually used to increase the column. For example, if an operation instruction indicates that the column "A" is added under the column family "F1" of the table "T1", the record as shown in the third row in Table 1 can be added.

6) If the operation instruction is used to delete the specified column under the specified column family in the specified table, the column name containing the specified column, the column family name of the column family to which the specified column belongs, and the column to which the specified column belongs may be deleted in the two-dimensional table. A record of the table name of the table. For example, if an operation instruction instructs deletion of the column "A" in the column family "F1" of the table "T1", the record shown in the third row can be deleted in Table 1 shown above.

The two-dimensional table provided by the present disclosure may include a service description field of a table, a service description field of a column family, a service description field of a column, a creation time field of a table, in addition to a field for describing a table structure of the table, The creation time field of the column family, the creation time field of the column, and so on. The business description fields of the table, column family, and column are used to annotate the business meaning of the table, column family, and column, respectively. For example, by the example in Table 2 below, it can be known that the column name "A" represents the student name. The creation time fields for tables, column families, and columns are used to create time for annotation tables, column families, and columns, respectively, which can be accurate to milliseconds. HBase can provide business descriptions of external interface collection tables, column families, and columns, and store them in a two-dimensional table; and can determine the creation time of tables, column families, and columns through local system time, and store them in a two-dimensional table.

Table 2 Two-dimensional representation

Table Name	Column family name	Column name	Column business description	Column creation time
T1	F1	A	A indicates the student's name	yyyy-MM-dd-hh-mm-ss-SSS
T1	F1	B	B indicates student gender	yyyy-MM-dd-hh-mm-ss-SSS
T1	F2	C	C indicates the age of the student	yyyy-MM-dd-hh-mm-ss-SSS

In step 303, the master node does not perform any processing when it is determined that the operation instruction does not change the table structure of the table.

For example, the Update statement in the SQL language is used to update the value. Generally, the table structure of the table is not changed. If the operation command received by the Master node includes an Update statement, the metadata collection procedure can be directly ended. The package is called without updating the two-dimensional table.

Optionally, the master node may implement the method shown in FIG. 2 in an asynchronous manner, that is, the master node performs step 300 to step 303 on the one hand, and normally responds to the operation instruction on the one hand, according to the action processing table indicated by the operation instruction. This can not affect the normal operation of HBase.

In the present application, after the metadata is collected and stored, the stored metadata can also be presented and maintained. Taking a two-dimensional table stored in a database (hereinafter referred to as a metabase) as an example, the way to present and maintain HBase metadata through a web application is:

First, build a Web container on the HBase cluster or the server that can connect to the metabase, and then build a Java-developed Web application on the Web container. The architecture of the Web application is shown in Figure 5. It can include the data access layer. , business interface and implementation layer, REST (Representational State Transfer) API (Application Programming Interface) and front-end view layer. The web application can interact with the metabase via JDBC (Java Data Base Connectivity).

For example, the data access layer of the web application can read the stored HBase metadata from the metadata database; the service interface and the implementation layer can be responsible for scheduling the data access layer and the REST API, and assembling the assembled metadata in the data access layer. Passed between the REST API; the front-end view layer gets the assembled metadata by calling the REST API and presenting them in an orderly way on the web page. You can add a service description to a specified table or column in a two-dimensional table through an interface provided by a web application, such as a page, or you can ambiguously query related metadata according to a business description or a column name or a table name.

If the two-dimensional table is stored in a text file, the web application can manage the metadata in the form of an existing management file, which will not be described in detail herein.

It can be seen from the flow shown in FIG. 2 that the present disclosure utilizes the coprocessor function provided by HBase to realize automatic collection and storage of metadata corresponding to the table structure of the table, and can quickly learn a certain kind of metadata based on the stored metadata. The table structure of a table eliminates the need to traverse all the records of the table, realizing the transformation of such metadata from unmanageable to manageable. In addition, the present disclosure also provides an interface that can add a service description to a specified table or column, increases the availability of HBase, and reduces the development difficulty of an HBase-based application.

The following describes how the present disclosure manages the metadata corresponding to the table structure of the table through a two-dimensional table through a specific embodiment.

First, create a relational database as a metabase, and create a two-dimensional table in the metabase for storing the metadata corresponding to the table structure of the table. The two-dimensional table includes fields such as table name, column family name, column name, and column creation time.

In the initial state, when there is no data of any table in HBase, the two-dimensional table is an empty table.

At time t1, the master node receives an operation instruction indicating that the table "T1" is created in HBase, the column family "F1" is created in the table "T1", and the column "A" is created under the column family "F1". "and "B". According to the method described above, since the operation instruction changes the structure of the table, the record related to the operation instruction can be added to the two-dimensional table created in advance, as shown in Table 3 below.

table 3

Table Name	Column family name	Column name	Column creation time
T1	F1	A	T1
T1	F1	B	T1

At time t2, the master node receives an operation instruction indicating that data is added to the "F1" column family members "A" and "B" of the "F1" row key of the table "T1", respectively. According to the method described above, since the operation instruction does not change the table structure of the table, the two-dimensional table is not updated, and the two-dimensional table at this time is still as shown in Table 3.

At time t3, the master node receives an operation instruction indicating that data is added to the "F1" column family members "A", "B", and "C" of the "F1" row key of the table "T1", respectively. By comparing the table structure contained in the operation instruction with the table structure contained in Table 3, it can be seen that the column "C" contained in the operation instruction does not appear in Table 3, that is, the operation instruction substantially changes the table structure, so that the table can be " The column "C" added under the column family "F1" of T1" is synchronized to the two-dimensional table, and the updated two-dimensional table is as shown in Table 4 below.

Table 4

Table Name	Column family name	Column name	Column creation time
T1	F1	A	T1
T1	F1	B	T1
T1	F1	C	T3

At time t4, the master node receives an operation instruction indicating the deletion of the column family "F1" under the table "T1". Since the operation instruction changes the table structure, the three records of the column family name "F1" can be deleted in the two-dimensional table shown in Table 4, that is, the three records in Table 4 are deleted.

The method of the present disclosure has been described above. The apparatus of the present disclosure will now be described, which has the function of implementing the above method. The functions may be implemented by hardware or by corresponding software implemented by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

FIG. 3 is a schematic diagram of a hardware structure of a master node according to the disclosure. The Master node can include a processor 601, a machine readable storage medium 602 that stores machine executable instructions. The processor 601 is in communication with a machine readable storage medium 602 that can perform the metadata management methods described above by reading and executing machine executable instructions in the machine readable storage medium 602.

Referring to FIG. 4, the present disclosure also provides a metadata management apparatus, which may be included in a Master node as shown in FIG. As shown in Figure 4, the device can include the following units:

An instruction detecting unit 500, configured to detect an operation instruction for the table;

The instruction parsing unit 501 is configured to parse an operation instruction for the table, and determine whether the operation instruction changes a table structure of the table; the change includes adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and Delete at least one of the columns; and

The metadata collection unit 502 is configured to: when the instruction parsing unit 501 determines that the operation instruction changes the table structure of the table, record the table name of the added table, the column family name of the added column family, or the added column. Column name or delete the record containing the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column. Specifically, if the change includes adding a table, adding a column family, or adding a column, the metadata collection unit 502 records the table name of the added table, the column family name of the added column family, or the column name of the added column. If the change includes deleting the table, deleting the column family, or deleting the column, the metadata collection unit 502 deletes the record including the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column.

Optionally, the instruction parsing unit 501 determines, when the operation instruction satisfies at least one of the following conditions, that the operation instruction changes a table structure of the table: 1) the operation instruction includes a Drop statement; 2) The operation instruction includes a Create statement; 3) the operation instruction includes an Alter statement, and the Alter statement is used to add a column family and/or delete a column family; 4) the operation instruction includes a Put statement, and the Put statement Used to add columns.

Optionally, the metadata collection unit 502 records the table name of the added table, the column family name of the added column family, or the column name of the added column in a pre-established two-dimensional table, and/or from the two-dimensional A record that contains the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column contains records for the name of the table, column family, or column. The two-dimensional table is located in a relational database or text file, and the two-dimensional table includes the following fields: a table name, a column family name, and a column name.

The metadata collection unit 502 can synchronize the changed table structure metadata into a pre-established two-dimensional table by the following steps:

1) if the operation instruction includes a Drop statement, deleting one or more records from the two-dimensional table, the one or more records including a table name of the table indicated by the Drop statement indicating deletion;

2) if the operation instruction includes a Create statement, adding one or more records to the two-dimensional table, the one or more records including the table name of the table indicated by the Create statement;

3) if the operation instruction includes an Alter statement and the Alter statement is used to delete a column family, deleting one or more records from the two-dimensional table, the one or more records including the Alter statement indicating deletion The column family name of the column family and the table name of the table to which the column family belongs;

4) if the operation instruction includes an Alter statement and the Alter statement is used to add a column family, adding one or more records to the two-dimensional table, the one or more records including the Alter statement indicating an increase The column family name of the column family and the table name of the table to which the column family belongs;

5) if the operation instruction includes a Put statement and the Put statement is used to add a column, adding one or more records to the two-dimensional table, the one or more records including the Put statement indicating an increase The column name of the column, the column family name of the column family to which the column belongs, and the table name of the table to which the column belongs.

Optionally, the two-dimensional table may further include at least one of the following fields: a service description of the table, a service description of the column family, a service description of the column, a creation time of the table, a creation time of the column family, and a creation time of the column. .

Optionally, the metadata management apparatus may further include: an instruction execution unit, configured to process the table according to the action indicated by the operation instruction while the instruction parsing unit 501 parses the operation instruction.

As noted above, the present disclosure can be a method, apparatus, program, and machine readable storage medium. The machine readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The machine-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive lists) of machine-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, with instructions stored thereon A raised structure in the hole card or groove, and any suitable combination of the above. A machine-readable storage medium as used herein is not to be interpreted as a transient signal itself, such as radio waves or other free-propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (eg, light pulses through a fiber optic cable), or through wires The electrical signal transmitted.

The machine-executable instructions described herein can be downloaded from a machine-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device can execute instructions from the network receiver and forward the machine executable instructions for storage in a machine readable storage medium in each computing/processing device.

Machine-executable instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages. Source code or object code written in any combination, including object oriented programming languages such as Smalltalk, C++, etc., as well as conventional procedural programming languages such as the "C" language or similar programming languages. The machine executable instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server. . In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider to access the Internet) connection). In some embodiments, the electronic circuitry can be customized by utilizing state information of machine executable instructions, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA). The machine can execute instructions to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and/or FIG.

The machine executable instructions can be provided to a general purpose computer, a special purpose computer, or a processor of other programmable data processing apparatus to produce a machine such that when executed by a processor of a computer or other programmable data processing apparatus. Means for implementing the functions/acts specified in one or more of the blocks of the flowcharts and/or block diagrams. The machine executable instructions can also be stored in a machine readable storage medium that causes the computer, programmable data processing apparatus, and/or other apparatus to operate in a particular manner, such that the machine readable storage medium storing the instructions includes An article of manufacture that includes instructions for implementing various aspects of the functions/acts recited in one or more of the flowcharts.

The machine executable instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on a computer, other programmable data processing device or other device to produce a computer implemented process. Thus, instructions executed on a computer, other programmable data processing apparatus, or other device implement the functions/acts recited in one or more blocks of the flowcharts and/or block diagram.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a portion of an instruction that includes one or more components for implementing the specified logical functions. Executable instructions. In some alternative implementations, the functions noted in the blocks may also occur in a different order than those illustrated in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or function. Or it can be implemented by a combination of dedicated hardware and computer instructions.

The above description is only the preferred embodiment of the present disclosure, and is not intended to limit the disclosure, and any modifications, equivalents, improvements, etc., which are made within the spirit and principles of the present disclosure, should be included in the present disclosure. Within the scope of protection.

Claims (15)

1. A metadata management method comprising:

   A master node in the HBase architecture detects an operation instruction for a table;

   The master node parses the operation instruction, and determines whether the operation instruction changes a table structure of a table; the changes include adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and deleting a column. At least one

   The master node records the table name of the added table and the column family name of the added column family when determining that the operation instruction changes the table structure of the table, and the change includes adding a table, adding a column family, or adding a column. Or the column name of the added column;

   The master node deletes the table name including the deleted table and the column family of the deleted column family when determining that the operation instruction changes the table structure of the table, and the change includes deleting the table, deleting the column family, or deleting the column. A record of the column name of the named or deleted column.
2. The metadata management method according to claim 1, wherein the operation instruction is determined to change a table structure of the table when the operation instruction satisfies at least one of the following conditions:

   The operation instruction includes a Drop statement;

   The operation instruction includes a Create statement;

   The operation instruction includes an Alter statement, and the Alter statement is used to add a column family and/or delete a column family;

   The operation instruction includes a Put statement, and the Put statement is used to add a column.
3. The metadata management method according to claim 2, wherein said master node records the table name of the added table, the column family name of the added column family, or the column name of the added column in a pre-established two-dimensional table And/or deleting from the two-dimensional table a record containing the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column,

   The two-dimensional table is located in a relational database or text file, and the two-dimensional table includes the following fields: a table name, a column family name, and a column name.
4. The metadata management method according to claim 3, wherein

   If the operation instruction includes a Drop statement, the Master node deletes one or more records from the two-dimensional table, the one or more records including a table name of the table indicated by the Drop statement indicating deletion, and

   If the operation instruction includes a Create statement, the Master node adds one or more records to the two-dimensional table, the one or more records including the table name of the table in which the Create statement indicates an addition.
5. The metadata management method according to claim 3, wherein

   If the operation instruction includes an Alter statement and the Alter statement is used to delete a column family, the Master node deletes one or more records from the two-dimensional table, the one or more records including the Alter statement Indicates the column family name of the column family being deleted and the table name of the table to which the column family belongs, and

   If the operation instruction includes an Alter statement and the Alter statement is used to add a column family, the Master node adds one or more records in the two-dimensional table, the one or more records including the Alter statement Indicates the column family name of the added column family and the table name of the table to which the column family belongs.
6. The metadata management method according to claim 3, wherein

   If the operation instruction includes a Put statement and the Put statement is used to add a column, the Master node adds one or more records in the two-dimensional table, and the one or more records include the Put statement indication The column name of the added column, the column family name of the column family to which the column belongs, and the table name of the table to which the column belongs.
7. The metadata management method according to claim 3, wherein the two-dimensional table further comprises at least one of the following fields: a service description of the table, a service description of the column family, a service description of the column, a creation time of the table, and a column. The creation time of the family and the creation time of the column.
8. A master node in an HBase architecture, including:

   Processor;

   a machine readable storage medium storing machine executable instructions,

   Wherein, by reading and executing the machine executable instructions, the processor is caused to:

   Detecting operational instructions for the table;

   Parsing the operation instruction, and determining whether the operation instruction changes a table structure of the table; the change includes at least one of adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and deleting a column;

   Recording the table name of the added table, the column family name of the added column family, or the added column when it is determined that the operation instruction changes the table structure of the table, and the change includes adding a table, adding a column family, or adding a column Column name;

   Determining the table name of the deleted table, the column family name of the deleted column family, or deleting the table structure when the operation instruction changes the table structure of the table, and the change includes deleting the table, deleting the column family, or deleting the column. A record of the column name of the column.
9. The Master node according to claim 8, wherein the operation instruction is determined to change a table structure of the table when the operation instruction satisfies at least one of the following conditions:

   The operation instruction includes a Drop statement;

   The operation instruction includes a Create statement;

   The operation instruction includes an Alter statement, and the Alter statement is used to add a column family and/or delete a column family;

   The operation instruction includes a Put statement, and the Put statement is used to add a column.
10. The master node according to claim 9, wherein said processor records the table name of the added table, the column family name of the added column family, or the column name of the added column in a pre-established two-dimensional table, and / or delete the record containing the table name of the deleted table, the column family name of the deleted column family, or the column name of the deleted column from the two-dimensional table,

    The two-dimensional table is located in a relational database or text file, and the two-dimensional table includes the following fields: a table name, a column family name, and a column name.
11. The Master node according to claim 10, wherein

    If the operation instruction includes a Drop statement, the processor deletes one or more records from the two-dimensional table, the one or more records including a table name of the table indicated by the Drop statement indicating deletion, and

    If the operation instruction includes a Create statement, the processor adds one or more records to the two-dimensional table, the one or more records including the table name of the table in which the Create statement indicates an increase.
12. The Master node according to claim 10, wherein

    If the operation instruction includes an Alter statement and the Alter statement is for deleting a column family, the processor deletes one or more records from the two-dimensional table, the one or more records including the Alter statement Indicates the column family name of the column family being deleted and the table name of the table to which the column family belongs, and

    If the operation instruction includes an Alter statement and the Alter statement is used to add a column family, the processor adds one or more records to the two-dimensional table, the one or more records including the Alter statement Indicates the column family name of the added column family and the table name of the table to which the column family belongs.
13. The Master node according to claim 10, wherein

    If the operation instruction includes a Put statement and the Put statement is used to add a column, the processor adds one or more records to the two-dimensional table, the one or more records including the Put statement indication The column name of the added column, the column family name of the column family to which the column belongs, and the table name of the table to which the column belongs.
14. The master node according to claim 10, wherein the two-dimensional table further comprises at least one of the following fields: a service description of the table, a service description of the column family, a service description of the column, a creation time of the table, and a column family. Create time and column creation time.
15. A machine readable storage medium comprising instructions that, when executed by a master node in an HBase architecture, cause the master node to perform the following operations:

    Detecting operational instructions for the table;

    Parsing the operation instruction, and determining whether the operation instruction changes a table structure of the table; the change includes at least one of adding a table, deleting a table, adding a column family, deleting a column family, adding a column, and deleting a column;

    Recording the table name of the added table, the column family name of the added column family, or the added column when it is determined that the operation instruction changes the table structure of the table, and the change includes adding a table, adding a column family, or adding a column Column name;

    Determining the table name of the deleted table, the column family name of the deleted column family, or deleting the table structure when the operation instruction changes the table structure of the table, and the change includes deleting the table, deleting the column family, or deleting the column. A record of the column name of the column.

Images