CN110502524B - Phoenix index data asynchronous updating method and device - Google Patents

Phoenix index data asynchronous updating method and device Download PDF

Info

Publication number
CN110502524B
CN110502524B CN201910754718.2A CN201910754718A CN110502524B CN 110502524 B CN110502524 B CN 110502524B CN 201910754718 A CN201910754718 A CN 201910754718A CN 110502524 B CN110502524 B CN 110502524B
Authority
CN
China
Prior art keywords
phoenix
index
index table
hbase
updating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910754718.2A
Other languages
Chinese (zh)
Other versions
CN110502524A (en
Inventor
申胜利
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinan Inspur Data Technology Co Ltd
Original Assignee
Jinan Inspur Data Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinan Inspur Data Technology Co Ltd filed Critical Jinan Inspur Data Technology Co Ltd
Priority to CN201910754718.2A priority Critical patent/CN110502524B/en
Publication of CN110502524A publication Critical patent/CN110502524A/en
Application granted granted Critical
Publication of CN110502524B publication Critical patent/CN110502524B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/273Asynchronous replication or reconciliation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention discloses a Phoenix index data asynchronous updating method and a system, wherein metadata information of an original table is obtained in Hbase, and a pre-written log Hlog of the original table is determined based on the metadata information; determining operation information from the marking time to the current time in the Hlog; and updating the index table of the Hbase, namely updating the index table of the Phoenix, based on the pre-generated json and the operation information. By the data asynchronous updating method and system, data in the Hbase original table are updated to the Hbase index table and the Phoenix index table at a preset time point, and the data corresponding to the data stored in the original table can be indexed by using the Phoenix index table.

Description

Phoenix index data asynchronous updating method and device
Technical Field
The invention relates to the field of databases, in particular to a Phoenix index data asynchronous updating method and device.
Background
The distributed Database (Hbase) is a distributed, column-oriented open source Database, and the Phoenix is a Structured Query Language (SQL) layer built on the Hbase. The data of the original table in the Hbase is the bottom layer data of the Phoenix mapping table, and the data of the index table in the Hbase is the bottom layer data of the Phoenix index table, so that the data in the Hbase can be indexed through Phoenix.
However, after data is newly added to the original table in the Hbase in batch, the data of the original table in the Hbase is not synchronous with the data of the index table, so that the data of the original table is inconsistent with the data of the index table, and when the original table is queried through the hoenix execution plan walk-in, the newly added data in batch of the original table cannot be queried.
Disclosure of Invention
In view of this, embodiments of the present invention provide an asynchronous update method and system for Phoenix index data, in which data in an Hbase original table is updated to an index table at a preset time point, and the purpose of indexing corresponding data stored in the original table can be achieved by using the index table.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
the invention discloses a Phoenix index data asynchronous updating method in the first aspect, which comprises the following steps: acquiring metadata information of an original table in Hbase, and determining a pre-written log Hlog of the original table based on the metadata information;
determining operation information from the marking time of the original table to the current time in the Hlog, wherein the operation information comprises an added put and a deleted delete;
and updating the index table of Phoenix based on json generated by the mapping table and the index table of Phoenix and the operation information.
Preferably, the process of generating json by the mapping table and the index table of Phoenix includes:
acquiring a corresponding relation between a mapping table and an index table in Phoenix metadata table SYSTEM. CATALOG;
setting parameters in the mapping table as filling parameters, and setting parameters in the index table as optional parameters;
generating json based on the mapping table and the index table.
Preferably, after updating the index table of the Hbase, that is, the index table of Phoenix, based on the pre-generated json and the operation information, the method further includes:
and acquiring the current system time, and setting the current system time as the marking time.
Preferably, the method further comprises the following steps:
and determining the update time of the index table of the Hbase next time based on the marking time and a preset time interval.
Preferably, the updating the index table of Hbase, that is, the index table of Phoenix, based on the pre-generated json and the operation information includes:
based on the pre-generated json and the operation information, forming logic for updating the row level of the original table to the row level of the index table in the Hbase;
updating the index table of the Hbase based on the logic, i.e. updating the index table of Phoenix.
The second aspect of the invention discloses a Phoenix index data asynchronous updating system, which comprises:
The screening device is used for acquiring metadata information of an original table in Hbase and determining a pre-written log Hlog of the original table based on the metadata information; determining operation information from the marking time to the current time in the Hlog;
and the execution device is used for updating the index table of the Hbase and the index table of the Phoenix based on the pre-generated json and the operation information.
Preferably, the method further comprises the following steps: a reading device;
the reading device is used for acquiring the corresponding relation between a mapping table and an index table in a Phoenix metadata table SYSTEM. CATALOG; setting parameters in the mapping table as filling-necessary parameters, and setting parameters in the index table as optional parameters; and generating json based on the mapping table and the index table.
Preferably, the executing device is further configured to obtain a current system time, and set the current system time as the mark time.
Preferably, the executing device is further configured to determine an update time of the index table of the Hbase next time based on the marking time and a preset time interval.
Preferably, the executing device is specifically configured to form logic for updating the row level of the original table to the row level of the index table in the Hbase based on a pre-generated json and the operation information; and updating the index table of the Hbase based on the logic, i.e. updating the index table of Phoenix.
According to the content, the invention discloses a Phoenix index data asynchronous updating method and system, wherein metadata information of an original table is obtained in Hbase, and a pre-written log Hlog of the original table is determined based on the metadata information; determining operation information from the marking time to the current time in the Hlog; and updating the index table of the Hbase, namely updating the index table of Phoenix, based on the pre-generated json and the operation information. By the data asynchronous updating method and system disclosed above, the data in the Hbase original table is updated to the index table of the Hbase at a preset time point, namely, the index table of Phoenix, and the data corresponding to the data stored in the original table can be indexed by using the index table of Phoenix.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a Phoenix index data asynchronous update method provided by an embodiment of the present invention;
FIG. 2 is a flowchart of another Phoenix index data asynchronous update method provided in the embodiments of the present invention;
FIG. 3 is a flowchart of another Phoenix index data asynchronous update method provided in the embodiments of the present invention;
FIG. 4 is a flowchart of another Phoenix index data asynchronous updating method provided in the embodiment of the present invention;
FIG. 5 is a flowchart of another Phoenix index data asynchronous updating method provided in the embodiment of the present invention;
FIG. 6 is a schematic structural diagram of a Phoenix index data asynchronous update system according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a Phoenix index data asynchronous update system provided in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiment of the invention provides a Phoenix index data asynchronous updating method, which at least comprises the following steps as shown in figure 1:
step S101: and acquiring metadata information of the original table in Hbase, and determining a pre-written log Hlog of the original table based on the metadata information.
In step S101, the metadata information is data describing data, mainly information describing data attributes, including storage locations of the data. Therefore, the Hlog of the original table can be found by acquiring the metadata information of the original table and then by the storage position of the original table recorded in the metadata information.
It should be noted that Hlog is a pre-written log, operation information in an original table is stored in the pre-written log, and the operation information is put and delete operations.
Step S102: and determining operation information from the marking time to the current time in the Hlog.
In step S102, since Hlog records the put and delete operations of the original table, and records the time of each put and delete, the operation information of the original table from the mark time to the current time can be determined by the mark time and the current time.
It should be noted that the marking time may be a manually set time point, or an end time point of each period of data update, and the marking time may be set according to actual needs.
Step S103: and updating the index table of the Hbase, namely updating the index table of Phoenix, based on the pre-generated json and the operation information.
In step S103, json is a relationship generated based on the mapping table and the index table in Phoenix.
It should be noted that the Phoenix mapping table is obtained according to the Hbase original table, that is, the Hbase original table data is the bottom layer data of the Phoenix mapping table, the Phoenix index table is a table created according to the mapping table, and the Hbase index table data is the bottom layer data of the Phoenix index table. Therefore, the data of the index table in the Hbase is updated to the data stored in the original table through the json relation of the mapping table and the index table in Phoenix, namely, the update of the index table in the Hbase is completed.
For ease of understanding, the json relationships generated by the mapping table and the index table are shown below.
For example:
{"data_table":"RECORD2","index_table_list":[{"index_table":"IDS2","index":"INFO:NUM","INCLUDE":"INFO:PROVINCE"}]}。
it should be noted that, based on the json and the operation information from the time marking to the current time, the data in the index table of Hbase and the index table of Phoenix are updated.
It should be noted that, since the operation information records put and delete of the data in the original table, put and delete of the data in the index table of Hbase and the index table of Phoenix are performed through json relationship, and finally, the data in the index table of Hbase and the index table of Phoenix are consistent with the data in the original table.
When the index table of Hbase is updated, the index table is updated in accordance with the row level of the table.
In the process of executing step S103, as shown in fig. 2, the specific execution process includes the following steps:
step S201: and forming logic for updating the row level of the original table to the row level of the index table in the Hbase based on the pre-generated json and the operation information.
In step S201, json is a set of relationship generated by the indexing table and the mapping table of Phoenix, and logic for updating the row level of the original table to the row level of the indexing table in the Hbase is formed by the json.
Step S202: updating the index table of the Hbase based on the logic, i.e. updating the index table of Phoenix.
In step S202, the data in the original table is updated to the index table in Hbase by the line level by the formed logic.
It should be noted that, since the data of the index table in the Hbase is the underlying data of the index table in the Phoenix, updating the data of the original table to the index table in the Hbase is also equivalent to updating the index table of the Phoenix, and the index table at this time can index the data corresponding to the original table storage through the index table.
According to the embodiment of the application, the metadata information of an original table is obtained from Hbase, and a pre-written log Hlog of the original table is determined based on the metadata information; determining operation information from the marking time to the current time in the Hlog; and updating the index table of the Hbase, namely updating the index table of the Phoenix, based on the pre-generated json and the operation information. By the data asynchronous updating method disclosed above, the data in the Hbase original table is updated to the index table of Hbase at a preset time point, that is, the index table of Phoenix, and the data corresponding to the data stored in the original table can be indexed by using the index table of Phoenix.
Based on the asynchronous data updating method disclosed in the foregoing embodiment, in step S103 shown in fig. 1, the specific process of generating json in advance, as shown in fig. 3, includes the following steps:
step S301: and acquiring the corresponding relation between a mapping table and an index table in Phoenix metadata table SYSTEM.
In step S301, in the Phoenix, the index table is a table created from a mapping table, and thus the index table and the mapping table are table data having a certain correspondence relationship.
Step S302: and setting the parameters in the mapping table as filling parameters, and setting the parameters in the index table as optional parameters.
In step S302, since the index table is a table created according to the mapping table, the parameters of the index table can be obtained from the mapping table even if the parameters of the index table are not filled in the index table and the mapping table, that is, the parameters in the mapping table are the parameters that must be filled in, and the parameters of the index table are the optional parameters.
Step S303: generating json based on the mapping table and the index table.
In step S303, json expresses a set of relationships in the mapping table and the index table, where the row key is formed by the index column, the separator, and the row key of the original table in json. The data in the original table can be updated to the index table of Hbase only through the row key.
The method comprises the steps of obtaining a corresponding relation between a mapping table and an index table in Phoenix metadata table SYSTEM. Setting parameters in the mapping table as filling parameters, and setting parameters in the index table as optional parameters; generating json based on the mapping table and the index table. Through the data asynchronous updating method disclosed above, the relationship between the mapping table and the index table in Phoenix is obtained, and a json relationship is generated.
Referring to fig. 4, another Phoenix index data asynchronous updating method provided by the present application includes the following steps:
step S401: and acquiring metadata information of the original table in Hbase, and determining a pre-written log Hlog of the original table based on the metadata information.
Step S402: and determining operation information from the marking time to the current time in the Hlog.
Step S403: updating the index table of the Hbase and the index table of the Phoenix based on the pre-generated json and the operation information.
It should be noted that the execution principle and the specific execution process of steps S401 to S403 are the same as the execution principle and the specific execution process of steps S101 to S103 shown in fig. 1, and reference may be made to the above corresponding description, which is not repeated herein.
Step S404: and acquiring the current system time, and setting the current system time as the marking time.
In step S404, the current system time is set as the mark time, and when the data of the original table needs to be updated to the index table of the Hbase next time, the put and delete operations of the original table from the mark time to the next current time can be updated to the index table of the Hbase.
It should be noted that, the current time is set as the mark time, so that all the data in the original table can be updated to the index table of Hbase.
According to the method, the metadata information of an original table is obtained from Hbase, and a pre-written log Hlog of the original table is determined based on the metadata information; determining operation information from the marking time to the current time in the Hlog; updating the index table of the Hbase based on the pre-generated json and the operation information, namely updating the index table of the Phoenix; and acquiring the current system time, and setting the current system time as the marking time. Through the data asynchronous updating method disclosed above, the data in the Hbase original table is updated to the Hbase index table, namely the Phoenix index table, at a preset time point, the current system time is set to be the marking time, the data asynchronous updating is realized, and the data corresponding to the data stored in the original table can be indexed by using the Phoenix index table.
Referring to fig. 5, another Phoenix index data asynchronous updating method provided by the present application includes the following steps:
step S501: and acquiring metadata information of the original table in Hbase, and determining a pre-written log Hlog of the original table based on the metadata information.
Step S502: and determining operation information from the marking time to the current time in the Hlog.
Step S503: and updating the index table of the Hbase, namely updating the index table of the Phoenix, based on the pre-generated json and the operation information.
Step S504: and acquiring the current system time, and setting the current system time as the marking time.
It should be noted that the execution principle and the specific execution process of steps S501 to S504 are the same as the execution principle and the specific execution process of steps S401 to S404 shown in fig. 4, and reference may be made to the above corresponding description, which is not repeated herein.
Step S505: and determining the updating time of the index table of the Hbase next time based on the marking time and a preset time interval.
In step S505, the preset time interval may be 1 day or 1 hour, and the preset time interval is determined according to actual conditions. In the present application, the time interval is preferably 1 day, i.e., 24 hours, but the time interval is not limited thereto.
According to the method, the metadata information of an original table is obtained from Hbase, and a pre-written log Hlog of the original table is determined based on the metadata information; determining operation information from the marking time to the current time in the Hlog; updating the index table of the Hbase based on the pre-generated json and the operation information, namely updating the index table of the Phoenix; acquiring current system time, and setting the current system time as marking time; and determining the update time of the index table of the Hbase next time based on the marking time and a preset time interval. According to the data asynchronous updating method disclosed by the invention, the data in the Hbase original table is updated to the Hbase index table, namely the Phoenix index table, at the preset time point, the current system time is set as the marking time, so that the automatic asynchronous updating of the data in the Hbase original table and the data in the Hbase index table are realized, and the data corresponding to the data stored in the original table can be indexed by using the Phoenix index table.
Corresponding to the asynchronous data updating method disclosed in the embodiment of the present invention, the embodiment of the present invention provides an asynchronous data updating system for Phoenix index, as shown in fig. 6, where the asynchronous data updating system includes:
The screening device 601 is used for acquiring metadata information of an original table in Hbase and determining a pre-written log Hlog of the original table based on the metadata information; and determining operation information from the marking time to the current time in the Hlog.
And the executing device 602 is configured to update the index table of the Hbase, that is, update the index table of the Phoenix, based on the pre-generated json and the operation information.
Preferably, as shown in fig. 7, the data asynchronous update system further includes: a reading device 603;
the reading device 603 is used for acquiring the corresponding relation between a mapping table and an index table in a Phoenix metadata table SYSTEM. CATALOG; setting parameters in the mapping table as filling-necessary parameters, and setting parameters in the index table as optional parameters; and generating json based on the mapping table and the index table.
Preferably, the executing device 602 is further configured to obtain a current system time, and set the current system time as the mark time.
Preferably, the executing device 602 is further configured to determine an update time of the index table of the Hbase next time based on the marking time and a preset time interval.
Preferably, the executing device 602 is specifically configured to form logic for updating the row level of the original table to the row level of the index table in the Hbase based on a pre-generated json and the operation information; and updating the index table of the Hbase based on the logic, i.e. updating the index table of Phoenix.
It should be noted that, for a specific execution process and an execution principle of each module and unit of the Phoenix index data asynchronous update system disclosed in the foregoing embodiment of the present invention, reference may be made to corresponding parts of the Phoenix index data asynchronous update method disclosed in the foregoing embodiment of the present invention, which are not described herein again.
According to the method, the metadata information of an original table is obtained in Hbase through a screening device, and a pre-written log Hlog of the original table is determined based on the metadata information; and determining operation information from the marking time to the current time in the Hlog; and the execution device updates the index table of the Hbase, namely the index table of the Phoenix, based on the pre-generated json and the operation information. By the data asynchronous updating system disclosed above, the data in the Hbase original table is updated to the Hbase index table and the Phoenix index table at a preset time point, and the data corresponding to the data stored in the original table can be indexed by using the Phoenix index table.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (8)

1. A Phoenix index data asynchronous updating method is characterized by comprising the following steps:
acquiring metadata information of an original table in Hbase, and determining a pre-written log Hlog of the original table based on a storage position of the original table recorded by the metadata information, wherein the metadata information is used for describing data attributes in the original table, and the data attributes comprise storage positions of data;
determining operation information from the marking time of the original table to the current time in the Hlog, wherein the operation information comprises an added put and a deleted delete;
asynchronously updating the index table of the Hbase based on the json pre-generated by the mapping table and the index table of the Phoenix and the operation information, namely asynchronously updating the index table of the Phoenix; the mapping table of Phoenix is a table obtained according to an original table in the Hbase, and the index table of Phoenix is a table created according to the mapping table;
the process of generating json in advance by the mapping table and the index table of Phoenix comprises the following steps:
acquiring a corresponding relation between a mapping table and an index table in Phoenix metadata table SYSTEM. CATALOG;
setting parameters in the mapping table as filling parameters, and setting parameters in the index table as optional parameters;
Generating json based on the mapping table and the index table.
2. The method of claim 1, wherein the generating json in advance based on the mapping table and the index table of Phoenix and the operation information after updating the index table of Phoenix further comprises:
and acquiring the current system time, and setting the current system time as the marking time.
3. The method of claim 2, further comprising:
and determining the update time of the index table of the Hbase next time based on the marking time and a preset time interval.
4. The method of claim 1, wherein updating the Phoenix index table based on json pre-generated by the Phoenix mapping table and index table and the operation information comprises:
based on the pre-generated json and the operation information, forming logic for updating the row level of the original table to the row level of the index table in the Hbase;
updating an index table of Phoenix based on the logic.
5. A Phoenix index data asynchronous update system, comprising:
the screening device is used for acquiring metadata information of an original table in Hbase and determining a pre-written log Hlog of the original table based on a storage position of the original table recorded by the metadata information, wherein the metadata information is used for describing data attributes in the original table, and the data attributes comprise storage positions of data; determining operation information from the marking time of the original table to the current time in the Hlog, wherein the operation information comprises an added put and a deleted delete;
The execution device is used for asynchronously updating the index table of the Hbase based on json generated in advance by the mapping table and the index table of Phoenix and the operation information, namely asynchronously updating the index table of Phoenix; the mapping table of the Phoenix is a table obtained according to an original table in the Hbase, and the index table of the Phoenix is a table created according to the mapping table;
the reading device is used for acquiring the corresponding relation between a mapping table and an index table in a Phoenix metadata table SYSTEM.CATALOG; setting parameters in the mapping table as filling-necessary parameters, and setting parameters in the index table as optional parameters; and generating json based on the mapping table and the index table.
6. The system of claim 5, wherein the execution device is further configured to obtain a current system time and set the current system time as the marker time.
7. The system of claim 6, wherein the executing means is further configured to determine an update time of the index table of the Hbase next time based on the marking time and a preset time interval.
8. The system of claim 5, wherein the execution device is specifically configured to form logic for updating the row level of the original table to the row level of the index table in the Hbase based on a pre-generated json and the operation information; and updating an index table of Phoenix based on the logic.
CN201910754718.2A 2019-08-15 2019-08-15 Phoenix index data asynchronous updating method and device Active CN110502524B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910754718.2A CN110502524B (en) 2019-08-15 2019-08-15 Phoenix index data asynchronous updating method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910754718.2A CN110502524B (en) 2019-08-15 2019-08-15 Phoenix index data asynchronous updating method and device

Publications (2)

Publication Number Publication Date
CN110502524A CN110502524A (en) 2019-11-26
CN110502524B true CN110502524B (en) 2022-06-10

Family

ID=68586486

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910754718.2A Active CN110502524B (en) 2019-08-15 2019-08-15 Phoenix index data asynchronous updating method and device

Country Status (1)

Country Link
CN (1) CN110502524B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113377786B (en) * 2021-08-16 2021-11-02 北京易鲸捷信息技术有限公司 Method for realizing on-line index creation

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9218383B2 (en) * 2013-03-15 2015-12-22 International Business Machines Corporation Differentiated secondary index maintenance in log structured NoSQL data stores
CN104834688B (en) * 2015-04-20 2019-10-08 北京奇艺世纪科技有限公司 A kind of secondary index method for building up and device
CN105608126A (en) * 2015-12-16 2016-05-25 成都创智云科技股份有限公司 Method and apparatus for establishing secondary indexes for massive databases
CN106503243B (en) * 2016-11-08 2019-08-06 国网山东省电力公司电力科学研究院 Electric power big data querying method based on HBase secondary index
CN107506464A (en) * 2017-08-30 2017-12-22 武汉烽火众智数字技术有限责任公司 A kind of method that HBase secondary indexs are realized based on ES

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
使用Phoenix操作数据库;郭建伟;《网络安全和信息化》;20190531(第05期);第82-84页 *

Also Published As

Publication number Publication date
CN110502524A (en) 2019-11-26

Similar Documents

Publication Publication Date Title
CN106325933A (en) Method and device for synchronizing batch data
CN110209728B (en) Distributed heterogeneous database synchronization method, electronic equipment and storage medium
US10803079B2 (en) Timing-based system-period temporal table in a database system
CN106599218A (en) Tree structure processing method and device
CN102023983B (en) Managing method of statistical space-time database
CN112217656B (en) Method and device for synchronizing configuration information of network equipment in SD-WAN (secure digital-to-Wide area network) system
CN105808653A (en) User label system-based data processing method and device
GB2526807A (en) Updating single regions of a navigational database
CN110502524B (en) Phoenix index data asynchronous updating method and device
CN105260446A (en) Data query system and method
CN102622224A (en) Method for achieving tree-like control piece with search function in Windchill system
CN102446161B (en) Digital content reading control method and device, system and terminal adopting same
CN108763256A (en) A kind of electric power GIS Temporal Model construction method based on Dike-pond modes
CN110442634A (en) The method and device of data storage
CN111858767A (en) Synchronous data processing method, device, equipment and storage medium
CN109492014A (en) A kind of database model packet creation method, device and equipment
CN110196883A (en) A kind of method of data synchronization and device on business on line without influence
CN106372121B (en) Server and data processing method
CN115269548A (en) Method and system for generating data warehouse development model and related equipment
CN112307692B (en) Station interlocking annunciator circuit drawing generation method and device
CN115033936A (en) Material model management method for SolidWorks system
CN112817931B (en) Incremental version file generation method and device
CN109408673B (en) Map management method and management platform
JP2009295050A (en) Product management server, product management method and program
CN111506588A (en) Method and device for extracting key information of electronic document

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant