WO2021238341A1 - Method and device for updating data in database, and electronic device - Google Patents

Method and device for updating data in database, and electronic device Download PDF

Info

Publication number
WO2021238341A1
WO2021238341A1 PCT/CN2021/080734 CN2021080734W WO2021238341A1 WO 2021238341 A1 WO2021238341 A1 WO 2021238341A1 CN 2021080734 W CN2021080734 W CN 2021080734W WO 2021238341 A1 WO2021238341 A1 WO 2021238341A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
target
database
data page
log
Prior art date
Application number
PCT/CN2021/080734
Other languages
French (fr)
Chinese (zh)
Inventor
张健
Original Assignee
北京金山云网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京金山云网络技术有限公司 filed Critical 北京金山云网络技术有限公司
Publication of WO2021238341A1 publication Critical patent/WO2021238341A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2457Query processing with adaptation to user needs
    • G06F16/24573Query processing with adaptation to user needs using data annotations, e.g. user-defined metadata
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275Synchronous replication
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present disclosure relates to the technical field of databases, and in particular to a method, device and electronic equipment for updating data in a database.
  • the physical replication mode is generally adopted between the master and the slave, that is: the data synchronized between the database master database and the database slave database is modified data, that is, the modification operation on the master database involves The data page is transferred to the slave library for data overwriting.
  • the physical replication synchronization method for database column operations involves changes in database table metadata and changes in existing data in the database table. For example, if you add a column to a database table, you need to add a column of records to the data recorded in each row of the table. At the same time, you also need to modify the schema information of the table. The amount of data involved is very large. After the master library is modified, all modified data pages need to be synchronously transmitted to the slave library for mapping writing.
  • the purpose of the embodiments of the present disclosure is to provide a method, device, and electronic device for updating data in a database, so as to improve the efficiency of changing data in the database.
  • an embodiment of the present disclosure provides a method for updating data in a database, including: obtaining data read and write instructions generated when a read and write operation is performed on a target data page in the database; after obtaining the data read and write instructions , Determine whether the target data page meets the data update condition, wherein the data update condition is a condition for updating the data content in the target data page; if the target data page meets the data update condition, then The data content in the target data page is updated.
  • determining whether the target data page satisfies the data update condition includes: obtaining the maximum log sequence number of the target data page in the target data page; determining that the target data page corresponds to the target data page in the target system table The target log sequence number of the target log, wherein the target system table contains the latest metadata information of each data table in the database, and the log sequence number of the operation log corresponding to the latest metadata information; if the maximum log sequence If the number and the target log sequence number are not the same, it is determined that the target data page satisfies the data update condition, and the data content in the target data page is updated.
  • updating the data content in the target data page includes: updating the data in the target data page according to the latest metadata information of the target data table in the target system table.
  • determining the target log sequence number corresponding to the target data page in the target system table includes: determining the table space ID information of the target data page; searching the target system table for the table space ID The log sequence number corresponding to the information, and the found log sequence number is used as the log sequence number.
  • the method further includes: when a metadata modification operation on the target data table is detected, generating an operation log corresponding to the metadata modification operation to obtain a target operation log; The operation log updates the target system table.
  • updating the target system table based on the target operation log includes: determining a log sequence number of the target operation log and metadata information corresponding to the target operation log; and storing the target log sequence The number is updated to the log sequence number of the target operation log; the latest metadata information of the target system table is updated to the metadata information corresponding to the target operation log.
  • the metadata modification operation includes: a deletion operation on a data column in the target data table, and/or a new operation on a data column in the target data table.
  • the method further includes: if the database is the master database, updating the data in the data page corresponding to the at least one database slave database; and the corresponding data in the at least one database slave database After the update of the data in the corresponding data page is completed, the end of the update information is returned to the user.
  • an embodiment of the present disclosure provides an apparatus for updating data in a database, including: an acquiring unit, configured to acquire data read and write instructions generated when a read and write operation is performed on a target data page in the database; and a judging unit, configured to After obtaining the data read and write instruction, it is determined whether the target data page satisfies the data update condition, wherein the data update condition is a condition for updating the data content in the target data page; an update unit, setting To update the data content in the target data page if the target data page meets the data update condition.
  • an embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor.
  • the processor executes the computer program, The method described in any one of the above-mentioned first aspects is implemented.
  • embodiments of the present disclosure provide a computer-readable medium having non-volatile program code executable by a processor, and the program code causes the processor to execute any one of the above-mentioned aspects of the first aspect. method.
  • the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed.
  • the technical solution is the technical problem of slow efficiency when changing the data in the database.
  • Figure 1 is a flowchart of a physical database replication method of related technologies
  • Fig. 2 is a flowchart of a method for updating data in a database according to an embodiment of the present disclosure
  • Fig. 3 is a flowchart of another method for updating data in a database according to an embodiment of the present disclosure
  • Fig. 4 is a schematic diagram of a data update device in a database according to an embodiment of the present disclosure
  • Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.
  • the application client initiates the operation of adding/deleting database table columns to the main database of the database
  • the database master database synchronizes the modification records involved in the operation to the database slave database through physical replication
  • the main database of the database returns the ACK confirmation character of the application APP result.
  • a method for updating data in a database is proposed.
  • This method can improve the database by updating the target data page that meets the update condition when the data read and write instruction is detected.
  • the change efficiency of data in the database and then solve the technical problem of slow efficiency when changing the data in the database through related technical solutions. The method will be described below in conjunction with an embodiment.
  • an embodiment of a method for updating data in a database is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and Although the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than here.
  • Fig. 2 is a flowchart of a method for updating data in a database according to an embodiment of the present disclosure. As shown in Fig. 2, the method includes the following steps:
  • Step S202 Obtain data read and write instructions generated when a read and write operation is performed on the target data page in the database.
  • the database architecture includes a database master database and a database slave database.
  • the database may be a database master database or a database slave database, which is not specifically limited in the embodiment of the present disclosure.
  • the data read and write instruction may be an instruction sent by the user to the database through the client to read and write the data page of the data table in the database.
  • the database database master library and database slave library
  • Step S204 after obtaining the data read and write instruction, determine whether the target data page meets the data update condition, where the data update condition is a condition for updating the data content in the target data page.
  • the database main database modifies the column data in the data table to obtain the updated database main database.
  • an application accesses the main database of the database, you can perform the corresponding read and write operations in the main database after the update.
  • the main database operation time is long, and the physical replication and transmission The modified data page has huge records, and the application APP waits for a long time, and it is prone to timeout.
  • passive access to the database can be set to trigger data changes, that is, after the data read and write instructions are obtained, it is determined whether the target data page in the database meets the data update condition.
  • the data update condition is determined based on the log serial number of the target data page and is set as a condition for judging whether to update the data content in the target data page.
  • step S206 if the target data page meets the data update condition, the data content in the target data page is updated.
  • the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed.
  • the technical solution is the technical problem of slow efficiency when changing the data in the database.
  • step S206 judging whether the target data page meets the data update condition, includes the following steps:
  • Step S2061 Obtain the maximum log sequence number of the target data page in the target data page
  • Step S2062 Determine the target log sequence number corresponding to the target data page in the target system table, where the target system table contains the latest metadata information of each data table in the database, and the log sequence number of the operation log corresponding to the latest metadata information ;
  • step S2063 if the maximum log sequence number and the target log sequence number are not the same, it is determined that the target data page meets the data update condition, and the data content in the target data page is updated.
  • a system table Schema_version_table (abbreviated as SVT), namely the target system table (abbreviated as SVT table) is added in advance.
  • SVT system table Schema_version_table
  • N the table space ID
  • LSN_N the log sequence number corresponding to the MLOG_TRX_DDL log LSN.
  • a redo log MLOG_TRX_DDL when a DDL operation is performed on a data table in the database, a redo log MLOG_TRX_DDL will be generated.
  • the log sequence number of the redo log and the new schema description corresponding to the DDL operation ie , The latest metadata information
  • the table space ID is recorded as a triple in the SVT table, where the table space ID is the table space ID information of each data table in the database.
  • the DDL operation is an operation of deleting or adding a column of a data table in the database.
  • the SVT table records the new schema description (the latest metadata information) of each data table in the database after the most recent DDL operation is performed, and the operation log corresponding to the new schema description (For example, redo log) log sequence number LSN.
  • the maximum log sequence number of the target data page can be obtained in the target data page first, and the maximum log sequence number Indicates the log sequence number of the redo log corresponding to the latest DDL operation on the target data page.
  • the target log sequence number can be determined by the table space ID.
  • the table space ID information of the target data page may be determined.
  • the determined table space ID information of the target data page is N.
  • look up the log sequence number corresponding to the table space ID information in the target system table and use the found log sequence number as the log sequence number. It can be seen from the above description that there are multiple triples in the SVT table, and each triple includes: the log sequence number of the redo log, and the new schema description (ie, the latest metadata information) corresponding to the DDL operation, and Table space ID.
  • the target log sequence number corresponding to the table space ID information of the target data table can be searched in the SVT table, for example, N. After the target log sequence number is found, the maximum log sequence number can be compared with the target log sequence number. If the maximum log sequence number and the target log sequence number are inconsistent, it is determined that the target data page meets the data update condition.
  • the data in the target data page can be updated according to the latest metadata information of the target data table in the target system table.
  • the corresponding triplet can be determined in the SVT table according to the table space ID information N, and then the latest metadata information is determined in the triplet, and the target data page is updated according to the latest metadata information. The data in is updated.
  • the above-mentioned processing method enables the database to perform DDL operations without real-time modification of the data in the database.
  • the data can be distributed and updated through passive access to reduce the instantaneous pressure of database DDL operations and shorten The response time of the database.
  • the database is the database master database
  • the data in the corresponding data page in the at least one database slave database is updated.
  • the database master database and the database slave database each correspond to a target system table SVT table, and the target system table contains the triplet data corresponding to each data table in the database: ⁇ N ,LSN_N,new schema description ⁇ , where N is the tablespace ID, and LSN_N is the MLOG_TRX_DDL log corresponding to the log sequence number LSN. That is, in the embodiment of the present disclosure, after the data content in the target data page in the database master library is updated, the data in the corresponding data page can also be updated according to the target system table of the database slave library. And after the update of the data in the corresponding data page in the at least one database from the database is completed, the end of the update information is returned to the user.
  • the method further includes the following process:
  • Step S1 when the metadata modification operation on the target data table is detected, an operation log corresponding to the metadata modification operation is generated to obtain the target operation log; wherein, in the embodiment of the present disclosure, the metadata modification operation includes: The deletion operation of the data column in the data table, and/or the new operation of the data column in the target data table.
  • Step S2 update the target system table based on the target operation log.
  • step S1 and step S2 can be performed before step S202 to step S206, and can also be performed after step S202 to step S206, which is not specifically limited in the embodiment of the present disclosure.
  • a new system table namely, the target system table (SVT table for short) is added in advance.
  • SVT table the target system table
  • the database when the database detects a metadata modification operation on the target data table (that is, a DDL operation), it generates a redo log (that is, a target operation log) corresponding to the metadata modification operation, and then, The target system table can be updated based on the target operation log.
  • a metadata modification operation on the target data table that is, a DDL operation
  • a redo log that is, a target operation log
  • the above update process can be described as:
  • the latest metadata information of the target system table is updated to the metadata information corresponding to the target operation log.
  • the operation log corresponding to the latest metadata modification operation may be generated. Then, update the target log sequence number in the target system table (that is, the log sequence number of the operation log corresponding to the last metadata modification operation) to the log sequence number of the target operation log (that is, the log sequence number of the latest metadata modification operation). The log sequence number of the operation log).
  • the latest metadata information in the target system table that is, the metadata information corresponding to the last metadata modification operation
  • the databases described in the above embodiments can be a database master database and a database slave database. Both the database master database and the database slave database correspond to a target system table SVT table.
  • the table contains the triplet data corresponding to each data table in the database: ⁇ N, LSN_N, new schema description ⁇ , where N is the table space ID, and LSN_N is the log sequence number LSN corresponding to the MLOG_TRX_DDL log.
  • the update process of the target system table of the database master library is the same as the update process of the target system table of the database slave library, and will not be described here.
  • the database master database updates the target system table according to the above-described process, it sends an update completion message to the database slave database.
  • the database slave database performs the update on the target system table according to the process described above. renew. After both the database master library and the database slave database have performed the update operation, the update complete message is returned to the client.
  • Fig. 3 is a flowchart of another method for updating data in a database according to an embodiment of the present disclosure. As can be seen from Fig. 3, the processing procedure of the method is described as follows:
  • the new schema information (that is, the latest metadata information) of the SVT (Schema_version_table) recording table and its corresponding redo log form a triple ⁇ N, LSN_N, new schema information ⁇ .
  • N is the table space ID
  • LSN_N is the MLOG_TRX_DDL log corresponding to the LSN
  • SVT table is a new target system table (abbreviated as SVT table) in the database;
  • the corresponding SVT table when the database replays the MLOG_TRX_DDL log from the library, the corresponding SVT table will also be updated.
  • Both the main database and the slave database correspond to a target system table SVT table.
  • the embodiment of the present disclosure also provides a device for updating data in a database.
  • the device for updating data in the database is mainly configured to execute the method for updating data in the database provided by the above-mentioned content of the embodiment of the present disclosure.
  • the update device will be introduced in detail.
  • Fig. 4 is a schematic diagram of a data update device in a database according to an embodiment of the present disclosure.
  • the data update device in the database mainly includes an acquisition unit 10, a judgment unit 20 and an update unit 30, wherein:
  • the acquiring unit 10 is configured to acquire data read and write instructions generated when a read and write operation is performed on a target data page in the database;
  • the judging unit 20 is configured to judge whether the target data page meets the data update condition after obtaining the data read and write instruction, where the data update condition is a condition for updating the data content in the target data page;
  • the update unit 30 is configured to update the data content in the target data page if the target data page meets the data update condition.
  • the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed. The technical solution is a slow technical problem when changing the data in the database.
  • the judging unit is configured to: obtain the maximum log sequence number of the target data page in the target data page; determine the target log sequence number corresponding to the target data page in the target system table, wherein the target system table contains the database The latest metadata information of each data table in each data table, and the log sequence number of the operation log corresponding to the latest metadata information; if the maximum log sequence number and the target log sequence number are not the same, it is determined that the target data page meets the data update conditions, and the target The data content in the data page is updated.
  • the judging unit is further configured to update the data in the target data page according to the latest metadata information of the target data table in the target system table.
  • the judging unit is further configured to: determine the table space ID information of the target data page; search for the log sequence number corresponding to the table space ID information in the target system table, and use the found log sequence number as the log sequence No.
  • the device is further configured to: when a metadata modification operation on the target data table is detected, generate an operation log corresponding to the metadata modification operation to obtain the target operation log; and compare the target system table based on the target operation log Update.
  • the device is further configured to: determine the log sequence number of the target operation log and the metadata information corresponding to the target operation log; update the target log sequence number to the log sequence number of the target operation log; The latest metadata information is updated to the metadata information corresponding to the target operation log.
  • the metadata modification operation includes: a deletion operation on a data column in the target data table, and/or a new operation on a data column in the target data table.
  • the device is further configured to: if the database is the database master, update the data in the data page corresponding to the at least one database slave library; the data page corresponding to the at least one database slave library After the update of the data in the data is completed, the end of the update information is returned to the user.
  • an embodiment of the present disclosure also provides an electronic device 100, including: a processor 50, a memory 51, a bus 52 and a communication interface 53, the processor 50, the communication interface 53, and the memory 51 are connected by the bus 52; the processor 50 It is configured to execute an executable module stored in the memory 51, such as a computer program.
  • the memory 51 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
  • RAM Random Access Memory
  • non-volatile memory such as at least one disk memory.
  • the communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
  • the bus 52 may be an ISA bus, a PCI bus, an EISA bus, or the like.
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one bidirectional arrow is used to indicate in FIG. 5, but it does not mean that there is only one bus or one type of bus.
  • the memory 51 is configured to store a program, and the processor 50 executes the program after receiving an execution instruction.
  • the method executed by the flow process definition apparatus disclosed in any of the foregoing embodiments of the present disclosure can be applied to the processor 50, Or implemented by the processor 50.
  • the processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor 50 or instructions in the form of software.
  • the aforementioned processor 50 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP for short), etc. ), Application Specific Integrated Circuit (ASIC for short), Field-Programmable Gate Array (FPGA for short) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the methods, steps, and logical block diagrams disclosed in the embodiments of the present disclosure can be implemented or executed.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the steps of the method disclosed in the embodiments of the present disclosure may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers.
  • the storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51, and completes the steps of the above method in combination with its hardware.
  • the terms “installed”, “connected”, and “connected” should be understood in a broad sense, for example, they may be fixed connections or detachable connections. , Or integrally connected; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • installed e.g., they may be fixed connections or detachable connections. , Or integrally connected; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components.
  • the computer program product for performing a method for updating data in a database includes a computer-readable storage medium storing a non-volatile program code executable by a processor, and instructions included in the program code can be set to execute the foregoing method
  • a computer-readable storage medium storing a non-volatile program code executable by a processor, and instructions included in the program code can be set to execute the foregoing method
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of units is only a logical function division, and there may be other divisions in actual implementation.
  • multiple units or components can be combined or integrated.
  • To another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections between devices or units through some communication interfaces, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor.
  • the technical solution of the present disclosure essentially or the part that contributes to the related technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several
  • the instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods of the various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

Abstract

A method and device for updating data in a database, and an electronic device, related to the technical field of cloud computing, comprising: acquiring a data read/write instruction generated when executing a read/write operation with respect to a target data page in a database (S202); when the data read/write instruction is acquired, determining whether the target data page satisfies a data updating criterion (S204), where the data updating criterion is a criterion for updating the data content in the target data page; and if the target data page satisfies the data updating criterion, then updating the data content in the target data page (S206). Solved is the technical problem of slow efficiency of an existing technical solution when modifying data in a database.

Description

数据库中数据更新方法、装置和电子设备Method, device and electronic equipment for updating data in database
本公开要求于2020年05月28日提交中国专利局、申请号为202010471948.0、发明名称为“数据库中数据更新方法、装置和电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本公开中。This disclosure claims the priority of a Chinese patent application filed with the Chinese Patent Office on May 28, 2020, the application number is 202010471948.0, and the invention title is "Data Update Method, Apparatus and Electronic Equipment in Database", the entire content of which is incorporated by reference In this disclosure.
技术领域Technical field
本公开涉及数据库的技术领域,尤其是涉及一种数据库中数据更新方法、装置和电子设备。The present disclosure relates to the technical field of databases, and in particular to a method, device and electronic equipment for updating data in a database.
背景技术Background technique
在支持计算与存储分离的数据库新架构中,一般主从之间采用物理复制模式,即:数据库主库和数据库从库之间同步的数据为修改的数据,即主库上修改操作涉及到的数据页传递到从库上进行数据覆盖写。In the new database architecture that supports the separation of computing and storage, the physical replication mode is generally adopted between the master and the slave, that is: the data synchronized between the database master database and the database slave database is modified data, that is, the modification operation on the master database involves The data page is transferred to the slave library for data overwriting.
针对数据库列操作的物理复制同步方式中,涉及到数据库表元数据的变更和数据库表已有数据的变更。例如对数据库表添加一列,则对表的每一行记录的数据都需要添加一列的记录,同时还需要修改表的schema信息,涉及修改的数据量非常大。当主库修改完后,需要将所有修改的数据页同步传输到从库进行映射写。The physical replication synchronization method for database column operations involves changes in database table metadata and changes in existing data in the database table. For example, if you add a column to a database table, you need to add a column of records to the data recorded in each row of the table. At the same time, you also need to modify the schema information of the table. The amount of data involved is very large. After the master library is modified, all modified data pages need to be synchronously transmitted to the slave library for mapping writing.
也就是说,在相关技术方案中,当数据库表的数据量特别大时,数据库主库操作时间长,数据更新效率低,物理复制传输的修改数据页记录庞大,应用客户端的等待时间很长,很容易发生超时。That is to say, in related technical solutions, when the amount of data in the database table is particularly large, the main database operation time is long, the data update efficiency is low, the modified data page records of the physical copy transmission are huge, and the waiting time of the application client is very long. Timeouts can easily occur.
发明内容Summary of the invention
有鉴于此,本公开实施例的目的在于提供一种数据库中数据更新方法、装置和电子设备,以提高数据库中数据的变更效率。In view of this, the purpose of the embodiments of the present disclosure is to provide a method, device, and electronic device for updating data in a database, so as to improve the efficiency of changing data in the database.
第一方面,本公开实施例提供了一种数据库中数据更新方法,包括:获取对数据库中的目标数据页执行读写操作时生成的数据读写指令;在获取到所述数据读写指令之后,判断所述目标数据页是否满足数据更新条件,其中,所述数据更新条件为对所述目标数据页中的数据内容进行更新的条件;若所述目标数据页满足所述数据更新条件,则对所述目标数据页中的数据内容进行更新。In a first aspect, an embodiment of the present disclosure provides a method for updating data in a database, including: obtaining data read and write instructions generated when a read and write operation is performed on a target data page in the database; after obtaining the data read and write instructions , Determine whether the target data page meets the data update condition, wherein the data update condition is a condition for updating the data content in the target data page; if the target data page meets the data update condition, then The data content in the target data page is updated.
在一实施方式中,判断所述目标数据页是否满足数据更新条件包括:在所述目标数据页中获取所述目标数据页的最大日志序列号;在目标系统表中确定所述目标数据页对应的目标日志序列号,其中,所述目标系统表中包含所述数据库中各个数据表的最新元数据信息,以及所述最新元数据信息对应的操作日志的日志序列号;若所述最大日志序列号和所述目标日志序列号不相同,则确定所述目标数据页满足所述数据更新条件,并对所述目标数据页中的数据内容进行更新。In one embodiment, determining whether the target data page satisfies the data update condition includes: obtaining the maximum log sequence number of the target data page in the target data page; determining that the target data page corresponds to the target data page in the target system table The target log sequence number of the target log, wherein the target system table contains the latest metadata information of each data table in the database, and the log sequence number of the operation log corresponding to the latest metadata information; if the maximum log sequence If the number and the target log sequence number are not the same, it is determined that the target data page satisfies the data update condition, and the data content in the target data page is updated.
在一实施方式中,对所述目标数据页中的数据内容进行更新包括:按照所述目标系统表中所述目标数据表的最新元数据信息对所述目标数据页中的数据进行更新。In one embodiment, updating the data content in the target data page includes: updating the data in the target data page according to the latest metadata information of the target data table in the target system table.
在一实施方式中,在目标系统表中确定所述目标数据页对应的目标日志序列号包括:确定所述目标数据页的表空间ID信息;在所述目标系统表查找与所述表空间ID信息相对应的日志序列号,并将查找到的日志序列号作为所述日志序列号。In one embodiment, determining the target log sequence number corresponding to the target data page in the target system table includes: determining the table space ID information of the target data page; searching the target system table for the table space ID The log sequence number corresponding to the information, and the found log sequence number is used as the log sequence number.
在一实施方式中,所述方法还包括:在检测到对所述目标数据表的元数据修改操作时,生成所述元数据修改操作所对应的操作日志,得到目标操作日志;基于所述目标操作日志对所述目标系统表进行更新。In one embodiment, the method further includes: when a metadata modification operation on the target data table is detected, generating an operation log corresponding to the metadata modification operation to obtain a target operation log; The operation log updates the target system table.
在一实施方式中,基于所述目标操作日志对所述目标系统表进行更新包括:确定所述目标操作日志的日志序列号和所述目标操作日志对应的元数据信息;将所述目标日志序列号更新为所述目标操作日志的日志序列号;将所述目标系统表的最新元数据信息更新为所述目标操作日志对应的元数据信息。In one embodiment, updating the target system table based on the target operation log includes: determining a log sequence number of the target operation log and metadata information corresponding to the target operation log; and storing the target log sequence The number is updated to the log sequence number of the target operation log; the latest metadata information of the target system table is updated to the metadata information corresponding to the target operation log.
在一实施方式中,所述元数据修改操作包括:对目标数据表中数据列的删除操作,和/或,对目标数据表中数据列的新增操作。In one embodiment, the metadata modification operation includes: a deletion operation on a data column in the target data table, and/or a new operation on a data column in the target data table.
在一实施方式中,所述方法还包括:若所述数据库为数据库主库,则对至少一个数据库从库中相对应的数据页中的数据进行更新;在所述至少一个数据库从库中相对应的数据页中的数据更新结束之后,向用户返回更新结束的信息。In one embodiment, the method further includes: if the database is the master database, updating the data in the data page corresponding to the at least one database slave database; and the corresponding data in the at least one database slave database After the update of the data in the corresponding data page is completed, the end of the update information is returned to the user.
第二方面,本公开实施例提供了一种数据库中数据更新装置,包括: 获取单元,设置为获取对数据库中的目标数据页执行读写操作时生成的数据读写指令;判断单元,设置为在获取到所述数据读写指令之后,判断所述目标数据页是否满足数据更新条件,其中,所述数据更新条件为对所述目标数据页中的数据内容进行更新的条件;更新单元,设置为若所述目标数据页满足所述数据更新条件,则对所述目标数据页中的数据内容进行更新。In a second aspect, an embodiment of the present disclosure provides an apparatus for updating data in a database, including: an acquiring unit, configured to acquire data read and write instructions generated when a read and write operation is performed on a target data page in the database; and a judging unit, configured to After obtaining the data read and write instruction, it is determined whether the target data page satisfies the data update condition, wherein the data update condition is a condition for updating the data content in the target data page; an update unit, setting To update the data content in the target data page if the target data page meets the data update condition.
第三方面,本公开实施例提供了一种电子设备,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现上述第一方面中任一项所述方法。In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor. When the processor executes the computer program, The method described in any one of the above-mentioned first aspects is implemented.
第四方面,本公开实施例提供了一种具有处理器可执行的非易失的程序代码的计算机可读介质,所述程序代码使所述处理器执行上述第一方面中任一项所述方法。In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium having non-volatile program code executable by a processor, and the program code causes the processor to execute any one of the above-mentioned aspects of the first aspect. method.
在本公开实施例中,首先,获取对数据库中的目标数据页执行读写操作时生成的数据读写指令,在获取到该数据读写指令之后,判断目标数据页是否满足数据更新条件,如果满足数据更新条件,则对目标数据页中的数据内容进行更新。通过上述描述可知,在本公开实施例中,通过在检测到数据读写指令的情况下,对满足更新条件的目标数据页进行更新的方式,能够提高数据库中数据的变更效率,进而解决通过相关技术方案对数据库中的数据进行变更时效率较慢的技术问题。In the embodiment of the present disclosure, firstly, the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed. The technical solution is the technical problem of slow efficiency when changing the data in the database.
本公开实施例的其他特征和优点将在随后的说明书中阐述,并且,部分地从说明书中变得显而易见,或者通过实施本公开实施例而了解。本公开实施例的目的和其他优点在说明书、权利要求书以及附图中所特别指出的结构来实现和获得。Other features and advantages of the embodiments of the present disclosure will be described in the following description, and partly become obvious from the description, or understood by implementing the embodiments of the present disclosure. The purpose and other advantages of the embodiments of the present disclosure are realized and obtained by the structures specifically pointed out in the specification, claims and drawings.
为使本公开实施例的上述目的、特征和优点能更明显易懂,下文特举较佳实施例,并配合所附附图,作详细说明如下。In order to make the above-mentioned objectives, features and advantages of the embodiments of the present disclosure more obvious and understandable, preferred embodiments are described in detail below in conjunction with accompanying drawings.
附图说明Description of the drawings
为了更清楚地说明本公开具体实施方式或相关技术中的技术方案,下面将对具体实施方式或相关技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本公开的一些实施方式,对于本领域普 通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the specific embodiments of the present disclosure or related technologies, the following will briefly introduce the drawings that need to be used in the specific embodiments or related technical descriptions. Obviously, the drawings in the following description are For some of the embodiments of the present disclosure, for those of ordinary skill in the art, other drawings may be obtained based on these drawings without creative work.
图1为相关技术的一种数据库物理复制方式的流程图;Figure 1 is a flowchart of a physical database replication method of related technologies;
图2是根据本公开实施例的一种数据库中数据更新方法的流程图;Fig. 2 is a flowchart of a method for updating data in a database according to an embodiment of the present disclosure;
图3是根据本公开实施例的另一种数据库中数据更新方法的流程图;Fig. 3 is a flowchart of another method for updating data in a database according to an embodiment of the present disclosure;
图4是根据本公开实施例的一种数据库中数据更新装置的示意图;Fig. 4 is a schematic diagram of a data update device in a database according to an embodiment of the present disclosure;
图5是根据本公开实施例的一种电子设备的示意图。Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.
具体实施方式Detailed ways
为使本公开实施例的目的、技术方案和优点更加清楚,下面将结合附图对本公开实施例的技术方案进行清楚、完整地描述,显然,所描述的实施例是本公开的一部分实施例,而不是全部的实施例。基于本公开中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本公开保护的范围。In order to make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are a part of the embodiments of the present disclosure. Not all examples. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present disclosure.
在对数据库中的列数据进行更新时,如图1所示,相关技术方案描述如下:When updating column data in the database, as shown in Figure 1, the related technical solutions are described as follows:
(1)应用客户端向数据库主库发起添加/删除数据库表列的操作;(1) The application client initiates the operation of adding/deleting database table columns to the main database of the database;
(2)数据库主库完成数据库表列的操作,并修改表的schame元数据;(2) The main database of the database completes the operation of the database table columns and modifies the schema metadata of the table;
(3)数据库主库通过物理复制将操作所涉及的修改记录同步到数据库从库;(3) The database master database synchronizes the modification records involved in the operation to the database slave database through physical replication;
(4)数据库从库回放主库同步过来的修改页数据;(4) The database replays the modified page data synchronized from the main database from the database;
(5)数据库从库返回主库物理复制结果ACK确认字符;(5) The database returns the ACK confirmation character of the physical copy result of the main database from the database;
(6)数据库主库返回应用APP结果ACK确认字符。(6) The main database of the database returns the ACK confirmation character of the application APP result.
针对相关技术,当数据库表的数据量特别大时,数据库主库操作时间长,物理复制传输的修改数据页记录庞大,应用APP等待时间会很长,很容易发生超时。For related technologies, when the data volume of the database table is particularly large, the main database operation time is long, the modified data page records of the physical copy transmission are huge, the waiting time of the application APP will be very long, and it is prone to timeout.
基于此,在本公开实施例中,提出了一种数据库中数据更新方法,该方法通过在检测到数据读写指令的情况下,对满足更新条件的目标数据页进行更新的方式,能够提高数据库中数据的变更效率,进而解决通过相关技术方案对数据库中的数据进行变更时效率较慢的技术问题。下面将结合 实施例介绍该方法。Based on this, in the embodiment of the present disclosure, a method for updating data in a database is proposed. This method can improve the database by updating the target data page that meets the update condition when the data read and write instruction is detected. The change efficiency of data in the database, and then solve the technical problem of slow efficiency when changing the data in the database through related technical solutions. The method will be described below in conjunction with an embodiment.
实施例一:Example one:
根据本公开实施例,提供了一种数据库中数据更新方法的实施例,需要说明的是,在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行,并且,虽然在流程图中示出了逻辑顺序,但是在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤。According to an embodiment of the present disclosure, an embodiment of a method for updating data in a database is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and Although the logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than here.
图2是根据本公开实施例的一种数据库中数据更新方法的流程图,如图2所示,该方法包括如下步骤:Fig. 2 is a flowchart of a method for updating data in a database according to an embodiment of the present disclosure. As shown in Fig. 2, the method includes the following steps:
步骤S202,获取对数据库中的目标数据页执行读写操作时生成的数据读写指令。Step S202: Obtain data read and write instructions generated when a read and write operation is performed on the target data page in the database.
在数据库架构中,包括数据库主库和数据库从库,在步骤S202中,该数据库可以为数据库主库,还可以为数据库从库,本公开实施例对此不做具体限定。在本公开实施例中,该数据读写指令可以为用户通过客户端向数据库发送的读写数据库中数据表的数据页的指令。其中,在数据库(数据库主库和数据库从库)中包含多个数据表,每个数据表中的数据存在于多个数据页中。The database architecture includes a database master database and a database slave database. In step S202, the database may be a database master database or a database slave database, which is not specifically limited in the embodiment of the present disclosure. In the embodiment of the present disclosure, the data read and write instruction may be an instruction sent by the user to the database through the client to read and write the data page of the data table in the database. Among them, the database (database master library and database slave library) contains multiple data tables, and the data in each data table exists in multiple data pages.
步骤S204,在获取到数据读写指令之后,判断目标数据页是否满足数据更新条件,其中,数据更新条件为对目标数据页中的数据内容进行更新的条件。Step S204, after obtaining the data read and write instruction, determine whether the target data page meets the data update condition, where the data update condition is a condition for updating the data content in the target data page.
通过上述描述可知,在相关技术中,在应用向数据库主库发起添加/删除数据库表列的操作之后,数据库主库修改数据表中的列数据,得到更新之后的数据库主库。当有应用访问该数据库主库时,则可以在更新之后的数据库主库中执行相应的读写操作,但是,当数据库表的数据量特别大时,数据库主库操作时间长,物理复制传输的修改数据页记录庞大,应用APP等待时间很长,很容易发生超时。It can be known from the above description that in the related technology, after the application initiates the operation of adding/deleting database table columns to the database main database, the database main database modifies the column data in the data table to obtain the updated database main database. When an application accesses the main database of the database, you can perform the corresponding read and write operations in the main database after the update. However, when the amount of data in the database table is particularly large, the main database operation time is long, and the physical replication and transmission The modified data page has huge records, and the application APP waits for a long time, and it is prone to timeout.
基于此,在本公开实施例中,可以设置数据库被动访问触发数据变更,也就是说,在获取到数据读写指令之后,判断数据库中的目标数据页是否满足数据更新条件。其中,该数据更新条件为基于目标数据页的日志序列 号所确定的设置为判断是否对目标数据页中的数据内容进行更新的条件。通过上述处理方式,使得数据库在执行DDL(Data Defination Language,数据库定义语言)操作之后,无需实时修改数据库中的数据,可通过被动访问分散更新数据,以降低数据库DDL操作瞬时压力,缩短数据库的响应时间。Based on this, in the embodiments of the present disclosure, passive access to the database can be set to trigger data changes, that is, after the data read and write instructions are obtained, it is determined whether the target data page in the database meets the data update condition. Wherein, the data update condition is determined based on the log serial number of the target data page and is set as a condition for judging whether to update the data content in the target data page. Through the above processing method, the database does not need to modify the data in the database in real time after executing the DDL (Data Destination Language) operation, and the data can be distributed and updated through passive access to reduce the transient pressure of the database DDL operation and shorten the response of the database time.
步骤S206,若目标数据页满足数据更新条件,则对目标数据页中的数据内容进行更新。In step S206, if the target data page meets the data update condition, the data content in the target data page is updated.
在本公开实施例中,首先,获取对数据库中的目标数据页执行读写操作时生成的数据读写指令,在获取到该数据读写指令之后,判断目标数据页是否满足数据更新条件,如果满足数据更新条件,则对目标数据页中的数据内容进行更新。通过上述描述可知,在本公开实施例中,通过在检测到数据读写指令的情况下,对满足更新条件的目标数据页进行更新的方式,能够提高数据库中数据的变更效率,进而解决通过相关技术方案对数据库中的数据进行变更时效率较慢的技术问题。In the embodiment of the present disclosure, firstly, the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed. The technical solution is the technical problem of slow efficiency when changing the data in the database.
在一个实施方式中,步骤S206,判断目标数据页是否满足数据更新条件,包括如下步骤:In one embodiment, step S206, judging whether the target data page meets the data update condition, includes the following steps:
步骤S2061,在目标数据页中获取目标数据页的最大日志序列号;Step S2061: Obtain the maximum log sequence number of the target data page in the target data page;
步骤S2062,在目标系统表中确定目标数据页对应的目标日志序列号,其中,目标系统表中包含数据库中各个数据表的最新元数据信息,以及最新元数据信息对应的操作日志的日志序列号;Step S2062: Determine the target log sequence number corresponding to the target data page in the target system table, where the target system table contains the latest metadata information of each data table in the database, and the log sequence number of the operation log corresponding to the latest metadata information ;
步骤S2063,若最大日志序列号和目标日志序列号不相同,则确定目标数据页满足数据更新条件,并对目标数据页中的数据内容进行更新。In step S2063, if the maximum log sequence number and the target log sequence number are not the same, it is determined that the target data page meets the data update condition, and the data content in the target data page is updated.
在本公开实施例中,预先新增了一个系统表Schema_version_table(简记SVT),即目标系统表(简称SVT表)。在该SVT表中设置为记录表的新schema信息及其对应的redo日志,构成三元组{N,LSN_N,新schema描述},其中,N为表空间ID,LSN_N为MLOG_TRX_DDL日志对应日志序列号LSN。In the embodiment of the present disclosure, a system table Schema_version_table (abbreviated as SVT), namely the target system table (abbreviated as SVT table) is added in advance. Set the new schema information of the record table and its corresponding redo log in the SVT table to form a triplet {N, LSN_N, new schema description}, where N is the table space ID and LSN_N is the log sequence number corresponding to the MLOG_TRX_DDL log LSN.
例如,在对数据库中的数据表执行DDL操作时,将生成一条redo日志MLOG_TRX_DDL,在本公开实施例中,可以将该redo日志的日志序列 号、以及该DDL操作所对应的新schema描述(即,最新元数据信息),以及表空间ID作为一个三元组记录在SVT表中,其中,表空间ID为数据库中各个数据表的表空间ID信息。需要说明的是,在本公开实施例中,DDL操作为对数据库中数据表的列执行删除或者添加的操作。For example, when a DDL operation is performed on a data table in the database, a redo log MLOG_TRX_DDL will be generated. In the embodiment of the present disclosure, the log sequence number of the redo log and the new schema description corresponding to the DDL operation (ie , The latest metadata information), and the table space ID is recorded as a triple in the SVT table, where the table space ID is the table space ID information of each data table in the database. It should be noted that, in the embodiment of the present disclosure, the DDL operation is an operation of deleting or adding a column of a data table in the database.
也就是说,在本公开实施例中,在SVT表中,记录了数据库中各个数据表在执行最近一次的DDL操作之后的新schema描述(最新元数据信息),以及新schema描述对应的操作日志(例如,redo日志)的日志序列号LSN。That is to say, in the embodiment of the present disclosure, the SVT table records the new schema description (the latest metadata information) of each data table in the database after the most recent DDL operation is performed, and the operation log corresponding to the new schema description (For example, redo log) log sequence number LSN.
基于此,在本公开实施例中,由于在数据页中记录了该数据页的日志序列号,因此,可以首先在目标数据页中获取该目标数据页的最大日志序列号,该最大日志序列号表示对目标数据页的最新DDL操作所对应的redo日志的日志序列号。Based on this, in the embodiment of the present disclosure, since the log sequence number of the data page is recorded in the data page, the maximum log sequence number of the target data page can be obtained in the target data page first, and the maximum log sequence number Indicates the log sequence number of the redo log corresponding to the latest DDL operation on the target data page.
之后,在SVT表中,确定该目标数据页所对应的目标日志序列号。在一实施方式中,可以通过表空间ID的方式确定该目标日志序列号。在一实施方式中,可以确定目标数据页的表空间ID信息,例如,确定出的该目标数据页的表空间ID信息为N。接下来,在目标系统表查找与表空间ID信息相对应的日志序列号,并将查找到的日志序列号作为日志序列号。通过上述描述可知,在SVT表中包含多个三元组,每个三元组包括:redo日志的日志序列号、以及该DDL操作所对应的新schema描述(即,最新元数据信息),以及表空间ID。基于此,在本公开实施中,可以在SVT表中查找与目标数据表的表空间ID信息相对应的目标日志序列号,例如,N。在查找到该目标日志序列号之后,可以将该最大日志序列号和目标日志序列号进行比较,如果最大日志序列号和目标日志序列号不一致,则确定目标数据页满足数据更新条件。After that, in the SVT table, determine the target log sequence number corresponding to the target data page. In one embodiment, the target log sequence number can be determined by the table space ID. In an embodiment, the table space ID information of the target data page may be determined. For example, the determined table space ID information of the target data page is N. Next, look up the log sequence number corresponding to the table space ID information in the target system table, and use the found log sequence number as the log sequence number. It can be seen from the above description that there are multiple triples in the SVT table, and each triple includes: the log sequence number of the redo log, and the new schema description (ie, the latest metadata information) corresponding to the DDL operation, and Table space ID. Based on this, in the implementation of the present disclosure, the target log sequence number corresponding to the table space ID information of the target data table can be searched in the SVT table, for example, N. After the target log sequence number is found, the maximum log sequence number can be compared with the target log sequence number. If the maximum log sequence number and the target log sequence number are inconsistent, it is determined that the target data page meets the data update condition.
需要说明的是,在SVT表中记录的是最新DDL操作的新schema描述和其所对应的日志序列号,因此,如果最大日志序列号和目标日志序列号不一致,则表明在对目标数据页执行最新DDL操作之后,还未对目标数据页中相对应的数据内容进行更新,因此,该目标数据页满足数据更新条件。It should be noted that what is recorded in the SVT table is the new schema description of the latest DDL operation and its corresponding log sequence number. Therefore, if the maximum log sequence number is inconsistent with the target log sequence number, it indicates that the target data page is being executed After the latest DDL operation, the corresponding data content in the target data page has not been updated. Therefore, the target data page satisfies the data update condition.
在确定出目标数据页满足数据更新条件之后,可以按照目标系统表中目标数据表的最新元数据信息对目标数据页中的数据进行更新。在一实施 方式中,可以根据表空间ID信息N在SVT表中确定其所对应的三元组,然后,在该三元组中确定最新元数据信息,并按照最新元数据信息对目标数据页中的数据进行更新。After it is determined that the target data page meets the data update condition, the data in the target data page can be updated according to the latest metadata information of the target data table in the target system table. In one embodiment, the corresponding triplet can be determined in the SVT table according to the table space ID information N, and then the latest metadata information is determined in the triplet, and the target data page is updated according to the latest metadata information. The data in is updated.
通过上述描述可知,在本公开实施例中,通过上述处理方式能够使得数据库在执行DDL操作之后,无需实时修改数据库中的数据,可通过被动访问分散更新数据,以降低数据库DDL操作瞬时压力,缩短数据库的响应时间。It can be seen from the above description that in the embodiments of the present disclosure, the above-mentioned processing method enables the database to perform DDL operations without real-time modification of the data in the database. The data can be distributed and updated through passive access to reduce the instantaneous pressure of database DDL operations and shorten The response time of the database.
在本公开实施例中,若数据库为数据库主库,则对至少一个数据库从库中相对应的数据页中的数据进行更新。In the embodiment of the present disclosure, if the database is the database master database, the data in the corresponding data page in the at least one database slave database is updated.
需要说明的是,在本公开实施例中,数据库主库和数据库从库均对应一个目标系统表SVT表,在该目标系统表中包含数据库中各个数据表所对应的三元组数据:{N,LSN_N,新schema描述},其中,N为表空间ID,LSN_N为MLOG_TRX_DDL日志对应日志序列号LSN。也即,在本公开实施例中,在对数据库主库中的目标数据页中的数据内容进行更新之后,还可以根据数据库从库的目标系统表对相应的数据页中的数据进行更新。并在至少一个数据库从库中相对应的数据页中的数据更新结束之后,向用户返回更新结束的信息。It should be noted that in the embodiment of the present disclosure, the database master database and the database slave database each correspond to a target system table SVT table, and the target system table contains the triplet data corresponding to each data table in the database: {N ,LSN_N,new schema description}, where N is the tablespace ID, and LSN_N is the MLOG_TRX_DDL log corresponding to the log sequence number LSN. That is, in the embodiment of the present disclosure, after the data content in the target data page in the database master library is updated, the data in the corresponding data page can also be updated according to the target system table of the database slave library. And after the update of the data in the corresponding data page in the at least one database from the database is completed, the end of the update information is returned to the user.
在一个实施方式中,该方法还包括如下过程:In one embodiment, the method further includes the following process:
步骤S1,在检测到对目标数据表的元数据修改操作时,生成元数据修改操作所对应的操作日志,得到目标操作日志;其中,在本公开实施例中,元数据修改操作包括:对目标数据表中数据列的删除操作,和/或,对目标数据表中数据列的新增操作。Step S1, when the metadata modification operation on the target data table is detected, an operation log corresponding to the metadata modification operation is generated to obtain the target operation log; wherein, in the embodiment of the present disclosure, the metadata modification operation includes: The deletion operation of the data column in the data table, and/or the new operation of the data column in the target data table.
步骤S2,基于目标操作日志对目标系统表进行更新。Step S2, update the target system table based on the target operation log.
需要说明的是,在本公开实施例中,步骤S1和步骤S2可以在步骤S202至步骤S206之前执行,还可以在步骤S202至步骤S206之后执行,本公开实施例对此不做具体限定。It should be noted that in the embodiment of the present disclosure, step S1 and step S2 can be performed before step S202 to step S206, and can also be performed after step S202 to step S206, which is not specifically limited in the embodiment of the present disclosure.
通过上述描述可知,在本公开实施例中,预先新增了一个系统表即,目标系统表(简称SVT表)。在该SVT表中设置为记录表的新schema信息及其对应的redo日志,构成三元组{N,LSN_N,新schema描述},其中, N为表空间ID,LSN_N为MLOG_TRX_DDL日志对应日志序列号LSN。From the above description, it can be known that in the embodiment of the present disclosure, a new system table, namely, the target system table (SVT table for short) is added in advance. Set the new schema information of the record table and its corresponding redo log in the SVT table to form a triplet {N, LSN_N, new schema description}, where N is the table space ID, and LSN_N is the log sequence number corresponding to the MLOG_TRX_DDL log LSN.
在本公开实施例中,数据库在检测到对目标数据表的元数据修改操作时(即,DDL操作),则生成该元数据修改操作所对应的redo日志(即,目标操作日志),之后,就可以基于目标操作日志对目标系统表进行更新。In the embodiment of the present disclosure, when the database detects a metadata modification operation on the target data table (that is, a DDL operation), it generates a redo log (that is, a target operation log) corresponding to the metadata modification operation, and then, The target system table can be updated based on the target operation log.
在一实施方式中,上述更新过程可以描述为:In an embodiment, the above update process can be described as:
首先,确定目标操作日志的日志序列号和目标操作日志对应的元数据信息;First, determine the log sequence number of the target operation log and the metadata information corresponding to the target operation log;
其次,将目标日志序列号更新为目标操作日志的日志序列号;Second, update the target log sequence number to the log sequence number of the target operation log;
最后,将目标系统表的最新元数据信息更新为目标操作日志对应的元数据信息。Finally, the latest metadata information of the target system table is updated to the metadata information corresponding to the target operation log.
在一实施方式中,在检测到用户对目标数据表的最新元数据修改操作之后,可以生成该最新的元数据修改操作所对应的操作日志,即,目标操作日志。然后,将目标系统表中的目标日志序列号(也即,上一次元数据修改操作所对应的操作日志的日志序列号)更新为目标操作日志的日志序列号(即,最新元数据修改操作的操作日志的日志序列号)。同时,还可以将目标系统表中的最新元数据信息(即,上一次元数据修改操作所对应的元数据信息)更新为目标操作日志对应的元数据信息(即,最新元数据修改操作对应的元数据信息)。In one embodiment, after detecting the user's latest metadata modification operation on the target data table, the operation log corresponding to the latest metadata modification operation, that is, the target operation log, may be generated. Then, update the target log sequence number in the target system table (that is, the log sequence number of the operation log corresponding to the last metadata modification operation) to the log sequence number of the target operation log (that is, the log sequence number of the latest metadata modification operation). The log sequence number of the operation log). At the same time, the latest metadata information in the target system table (that is, the metadata information corresponding to the last metadata modification operation) can be updated to the metadata information corresponding to the target operation log (that is, the latest metadata modification operation corresponds to Metadata information).
需要说明的是,在本公开实施例中,上述实施例中所描述的数据库可以为数据库主库和数据库从库,数据库主库和数据库从库均对应一个目标系统表SVT表,在该目标系统表中包含数据库中各个数据表所对应的三元组数据:{N,LSN_N,新schema描述},其中,N为表空间ID,LSN_N为MLOG_TRX_DDL日志对应日志序列号LSN。It should be noted that, in the embodiments of the present disclosure, the databases described in the above embodiments can be a database master database and a database slave database. Both the database master database and the database slave database correspond to a target system table SVT table. The table contains the triplet data corresponding to each data table in the database: {N, LSN_N, new schema description}, where N is the table space ID, and LSN_N is the log sequence number LSN corresponding to the MLOG_TRX_DDL log.
也就是说,在本公开实施例中,对数据库主库的目标系统表的更新过程和对数据库从库的目标系统表的更新过程相同,此处不再一一介绍。That is to say, in the embodiment of the present disclosure, the update process of the target system table of the database master library is the same as the update process of the target system table of the database slave library, and will not be described here.
在一实施方式中,当数据库主库按照上述所描述的过程对目标系统表进行更新之后,向数据库从库发送更新完成的消息,之后,数据库从库按照上述所描述的过程对目标系统表进行更新。在数据库主库和数据库从库均执行完更新操作之后,向客户端返回更新完成的消息。In one embodiment, after the database master database updates the target system table according to the above-described process, it sends an update completion message to the database slave database. After that, the database slave database performs the update on the target system table according to the process described above. renew. After both the database master library and the database slave database have performed the update operation, the update complete message is returned to the client.
通过上述描述可知,在本公开实施例中,在对数据库进行DDL操作(即,元数据修改操作)时,数据库主库或者数据库从库只需修改目标系统表的元数据信息即可,无需同步所有变更页数据,极大减少了主从库间同步数据量。It can be seen from the above description that, in the embodiment of the present disclosure, when performing DDL operations (ie, metadata modification operations) on the database, the database master database or the database slave database only needs to modify the metadata information of the target system table, without synchronization All changed page data greatly reduces the amount of data synchronized between the master and slave libraries.
实施例二:Embodiment two:
图3是根据本公开实施例的另一种数据库中数据更新方法的流程图,从图3中可以看出,该方法的处理过程描述如下:Fig. 3 is a flowchart of another method for updating data in a database according to an embodiment of the present disclosure. As can be seen from Fig. 3, the processing procedure of the method is described as follows:
(1)在数据页中记录数据页修改时的LSN标记,假设为LSN x;(1) Record the LSN mark when the data page is modified in the data page, assuming it is LSN x;
(2)当数据库中的数据表进行DDL操作(即,元数据修改操作)时,生成一条redo日志MLOG_TRX_DDL;(2) When the data table in the database performs a DDL operation (that is, a metadata modification operation), a redo log MLOG_TRX_DDL is generated;
(3)SVT(Schema_version_table)记录表的新schema信息(即,最新元数据信息)及其对应的redo日志,构成三元组{N,LSN_N,新schema信息}。其中,N为表空间ID,LSN_N为MLOG_TRX_DDL日志对应LSN;SVT表为数据库新增的一个目标系统表(简记SVT表);(3) The new schema information (that is, the latest metadata information) of the SVT (Schema_version_table) recording table and its corresponding redo log form a triple {N, LSN_N, new schema information}. Among them, N is the table space ID, LSN_N is the MLOG_TRX_DDL log corresponding to the LSN; SVT table is a new target system table (abbreviated as SVT table) in the database;
(4)获取对数据库中的数据页执行读写操作时生成的数据读写指令;(4) Obtain the data read and write instructions generated when performing read and write operations on data pages in the database;
(5)比较数据页的LSN x与SVT中记录的表的schema对应的LSN号;若数据页的LSN x与SVT中记录的表的schema对应的LSN号LSN_N不相等(例如,LSN_N>LSN x),则按照新schema信息对该数据页中的数据进行修改,从而生成新的数据页;(5) Compare the LSN x of the data page with the LSN number corresponding to the schema of the table recorded in the SVT; if the LSN x of the data page and the LSN number LSN_N corresponding to the schema of the table recorded in the SVT are not equal (for example, LSN_N>LSN x ), the data in the data page is modified according to the new schema information to generate a new data page;
其中,在本公开实施例中,数据库从库回放MLOG_TRX_DDL日志时,也会更新相对应的SVT表。数据库主库和数据库从库均对应一个目标系统表SVT表。Among them, in the embodiment of the present disclosure, when the database replays the MLOG_TRX_DDL log from the library, the corresponding SVT table will also be updated. Both the main database and the slave database correspond to a target system table SVT table.
实施例三:Embodiment three:
本公开实施例还提供了一种数据库中数据更新装置,该数据库中数据更新装置主要设置为执行本公开实施例上述内容所提供的数据库中数据更新方法,以下对本公开实施例提供的数据库中数据更新装置做具体介绍。The embodiment of the present disclosure also provides a device for updating data in a database. The device for updating data in the database is mainly configured to execute the method for updating data in the database provided by the above-mentioned content of the embodiment of the present disclosure. The update device will be introduced in detail.
图4是根据本公开实施例的一种数据库中数据更新装置的示意图,如图4所示,该数据库中数据更新装置主要包括获取单元10,判断单元20和更新单元30,其中:Fig. 4 is a schematic diagram of a data update device in a database according to an embodiment of the present disclosure. As shown in Fig. 4, the data update device in the database mainly includes an acquisition unit 10, a judgment unit 20 and an update unit 30, wherein:
获取单元10,设置为获取对数据库中的目标数据页执行读写操作时生成的数据读写指令;The acquiring unit 10 is configured to acquire data read and write instructions generated when a read and write operation is performed on a target data page in the database;
判断单元20,设置为在获取到数据读写指令之后,判断目标数据页是否满足数据更新条件,其中,数据更新条件为对目标数据页中的数据内容进行更新的条件;The judging unit 20 is configured to judge whether the target data page meets the data update condition after obtaining the data read and write instruction, where the data update condition is a condition for updating the data content in the target data page;
更新单元30,设置为若目标数据页满足数据更新条件,则对目标数据页中的数据内容进行更新。The update unit 30 is configured to update the data content in the target data page if the target data page meets the data update condition.
在本公开实施例中,首先,获取对数据库中的目标数据页执行读写操作时生成的数据读写指令,在获取到该数据读写指令之后,判断目标数据页是否满足数据更新条件,如果满足数据更新条件,则对目标数据页中的数据内容进行更新。通过上述描述可知,在本公开实施例中,通过在检测到数据读写指令的情况下,对满足更新条件的目标数据页进行更新的方式,能够提高数据库中数据的变更效率,进而解决通过相关的技术方案对数据库中的数据进行变更时效率较慢的技术问题。In the embodiment of the present disclosure, firstly, the data read and write instructions generated when the target data page in the database is read and written are acquired. After the data read and write instructions are acquired, it is determined whether the target data page satisfies the data update condition, if When the data update condition is met, the data content in the target data page is updated. It can be seen from the above description that in the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change efficiency in the database can be improved, and the data in the database can be changed. The technical solution is a slow technical problem when changing the data in the database.
在一实施方式中,判断单元设置为:在目标数据页中获取目标数据页的最大日志序列号;在目标系统表中确定目标数据页对应的目标日志序列号,其中,目标系统表中包含数据库中各个数据表的最新元数据信息,以及最新元数据信息对应的操作日志的日志序列号;若最大日志序列号和目标日志序列号不相同,则确定目标数据页满足数据更新条件,并对目标数据页中的数据内容进行更新。In one embodiment, the judging unit is configured to: obtain the maximum log sequence number of the target data page in the target data page; determine the target log sequence number corresponding to the target data page in the target system table, wherein the target system table contains the database The latest metadata information of each data table in each data table, and the log sequence number of the operation log corresponding to the latest metadata information; if the maximum log sequence number and the target log sequence number are not the same, it is determined that the target data page meets the data update conditions, and the target The data content in the data page is updated.
在一实施方式中,判断单元还设置为:按照目标系统表中目标数据表的最新元数据信息对目标数据页中的数据进行更新。In one embodiment, the judging unit is further configured to update the data in the target data page according to the latest metadata information of the target data table in the target system table.
在一实施方式中,判断单元还设置为:确定目标数据页的表空间ID信息;在目标系统表查找与表空间ID信息相对应的日志序列号,并将查找到的日志序列号作为日志序列号。In an embodiment, the judging unit is further configured to: determine the table space ID information of the target data page; search for the log sequence number corresponding to the table space ID information in the target system table, and use the found log sequence number as the log sequence No.
在一实施方式中,该装置还设置为:在检测到对目标数据表的元数据修改操作时,生成元数据修改操作所对应的操作日志,得到目标操作日志;基于目标操作日志对目标系统表进行更新。In one embodiment, the device is further configured to: when a metadata modification operation on the target data table is detected, generate an operation log corresponding to the metadata modification operation to obtain the target operation log; and compare the target system table based on the target operation log Update.
在一实施方式中,该装置还设置为:确定目标操作日志的日志序列号 和目标操作日志对应的元数据信息;将目标日志序列号更新为目标操作日志的日志序列号;将目标系统表的最新元数据信息更新为目标操作日志对应的元数据信息。In one embodiment, the device is further configured to: determine the log sequence number of the target operation log and the metadata information corresponding to the target operation log; update the target log sequence number to the log sequence number of the target operation log; The latest metadata information is updated to the metadata information corresponding to the target operation log.
在一实施方式中,元数据修改操作包括:对目标数据表中数据列的删除操作,和/或,对目标数据表中数据列的新增操作。In one embodiment, the metadata modification operation includes: a deletion operation on a data column in the target data table, and/or a new operation on a data column in the target data table.
在一实施方式中,该装置还设置为:若数据库为数据库主库,则对至少一个数据库从库中相对应的数据页中的数据进行更新;在至少一个数据库从库中相对应的数据页中的数据更新结束之后,向用户返回更新结束的信息。In one embodiment, the device is further configured to: if the database is the database master, update the data in the data page corresponding to the at least one database slave library; the data page corresponding to the at least one database slave library After the update of the data in the data is completed, the end of the update information is returned to the user.
本公开实施例所提供的装置,其实现原理及产生的技术效果和前述方法实施例相同,为简要描述,装置实施例部分未提及之处,可参考前述方法实施例中相应内容。The implementation principles and technical effects of the device provided in the embodiments of the present disclosure are the same as those of the foregoing method embodiments. For a brief description, for the parts not mentioned in the device embodiments, please refer to the corresponding content in the foregoing method embodiments.
实施例四:Embodiment four:
参见图5,本公开实施例还提供一种电子设备100,包括:处理器50,存储器51,总线52和通信接口53,处理器50、通信接口53和存储器51通过总线52连接;处理器50设置为执行存储器51中存储的可执行模块,例如计算机程序。5, an embodiment of the present disclosure also provides an electronic device 100, including: a processor 50, a memory 51, a bus 52 and a communication interface 53, the processor 50, the communication interface 53, and the memory 51 are connected by the bus 52; the processor 50 It is configured to execute an executable module stored in the memory 51, such as a computer program.
其中,存储器51可能包含高速随机存取存储器(RAM,Random Access Memory),也可能还包括非不稳定的存储器(non-volatile memory),例如至少一个磁盘存储器。通过至少一个通信接口53(可以是有线或者无线)实现该系统网元与至少一个其他网元之间的通信连接,可以使用互联网,广域网,本地网,城域网等。The memory 51 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
总线52可以是ISA总线、PCI总线或EISA总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示,图5中仅用一个双向箭头表示,但并不表示仅有一根总线或一种类型的总线。The bus 52 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one bidirectional arrow is used to indicate in FIG. 5, but it does not mean that there is only one bus or one type of bus.
其中,存储器51设置为存储程序,处理器50在接收到执行指令后,执行程序,前述本公开实施例任一实施例揭示的流过程定义的装置所执行的方法可以应用于处理器50中,或者由处理器50实现。Wherein, the memory 51 is configured to store a program, and the processor 50 executes the program after receiving an execution instruction. The method executed by the flow process definition apparatus disclosed in any of the foregoing embodiments of the present disclosure can be applied to the processor 50, Or implemented by the processor 50.
处理器50可能是一种集成电路芯片,具有信号的处理能力。在实现过 程中,上述方法的各步骤可以通过处理器50中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器50可以是通用处理器,包括中央处理器(Central Processing Unit,简称CPU)、网络处理器(Network Processor,简称NP)等;还可以是数字信号处理器(Digital Signal Processing,简称DSP)、专用集成电路(Application Specific Integrated Circuit,简称ASIC)、现成可编程门阵列(Field-Programmable Gate Array,简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本公开实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本公开实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成,或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器,闪存、只读存储器,可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器51,处理器50读取存储器51中的信息,结合其硬件完成上述方法的步骤。The processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor 50 or instructions in the form of software. The aforementioned processor 50 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP for short), etc. ), Application Specific Integrated Circuit (ASIC for short), Field-Programmable Gate Array (FPGA for short) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of the present disclosure may be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51, and completes the steps of the above method in combination with its hardware.
另外,在本公开实施例的描述中,除非另有明确的规定和限定,术语“安装”、“相连”、“连接”应做广义理解,例如,可以是固定连接,也可以是可拆卸连接,或一体地连接;可以是机械连接,也可以是电连接;可以是直接相连,也可以通过中间媒介间接相连,可以是两个元件内部的连通。对于本领域的普通技术人员而言,可以具体情况理解上述术语在本公开实施例中的具体含义。In addition, in the description of the embodiments of the present disclosure, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, they may be fixed connections or detachable connections. , Or integrally connected; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication between two components. For those of ordinary skill in the art, the specific meanings of the above-mentioned terms in the embodiments of the present disclosure can be understood in specific situations.
在本公开实施例的描述中,需要说明的是,术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系,仅是为了便于描述本公开实施例和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此不能理解为对本公开的限制。此外,术语“第一”、“第二”、“第三”仅设置为描述目的,而不能理解为指示或暗示相对重要性。In the description of the embodiments of the present disclosure, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" The orientation or positional relationship indicated by "" is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the embodiments of the present disclosure and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, It is constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present disclosure. In addition, the terms "first", "second", and "third" are only provided for descriptive purposes, and cannot be understood as indicating or implying relative importance.
本公开实施例所提供的进行数据库中数据更新方法的计算机程序产品,包括存储了处理器可执行的非易失的程序代码的计算机可读存储介质, 程序代码包括的指令可设置为执行前面方法实施例中的方法,具体实现可参见方法实施例,在此不再赘述。The computer program product for performing a method for updating data in a database provided by an embodiment of the present disclosure includes a computer-readable storage medium storing a non-volatile program code executable by a processor, and instructions included in the program code can be set to execute the foregoing method For the specific implementation of the method in the embodiment, please refer to the method embodiment, which will not be repeated here.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的系统、装置和单元的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described system, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.
在本公开所提供的几个实施例中,应该理解到,所揭露的系统、装置和方法,可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的,例如,单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,又例如,多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device, and method may be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other divisions in actual implementation. For example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections between devices or units through some communication interfaces, and may be in electrical, mechanical or other forms.
作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
另外,在本公开各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解,本公开的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本公开各个实施例方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure essentially or the part that contributes to the related technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several The instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods of the various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .
最后应说明的是:以上实施例,仅为本公开的具体实施方式,用以说 明本公开的技术方案,而非对其限制,本公开的保护范围并不局限于此,尽管参照前述实施例对本公开进行了详细的说明,本领域的普通技术人员应当理解:任何熟悉本技术领域的技术人员在本公开揭露的技术范围内,其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化,或者对其中部分技术特征进行等同替换;而这些修改、变化或者替换,并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围,都应涵盖在本公开的保护范围之内。因此,本公开的保护范围应以权利要求的保护范围为准。Finally, it should be noted that the above embodiments are only specific implementations of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than limit it. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing embodiments The present disclosure has been described in detail, and those of ordinary skill in the art should understand that any person skilled in the art can still modify or modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure. It is easy to think of changes, or equivalent replacements of some of the technical features; and these modifications, changes or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered by the protection of the present disclosure Within range. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.
工业实用性Industrial applicability
基于本公开实施例提供的数据库中数据更新方法、装置和电子设备,通过在检测到数据读写指令的情况下,对满足更新条件的目标数据页进行更新的方式,能够提高数据库中数据的变更效率,进而解决通过相关技术方案对数据库中的数据进行变更时效率较慢的技术问题。Based on the method, device and electronic device for updating data in the database provided by the embodiments of the present disclosure, by updating the target data page that meets the update condition when the data read and write instruction is detected, the data change in the database can be improved. Efficiency, and then solve the technical problem of slow efficiency when changing data in the database through related technical solutions.

Claims (11)

  1. 一种数据库中数据更新方法,包括:A method for updating data in a database, including:
    获取对数据库中的目标数据页执行读写操作时生成的数据读写指令;Obtain the data read and write instructions generated when the target data page in the database is read and written;
    在获取到所述数据读写指令之后,判断所述目标数据页是否满足数据更新条件,其中,所述数据更新条件为对所述目标数据页中的数据内容进行更新的条件;After obtaining the data read and write instruction, determine whether the target data page meets a data update condition, where the data update condition is a condition for updating data content in the target data page;
    若所述目标数据页满足所述数据更新条件,则对所述目标数据页中的数据内容进行更新。If the target data page meets the data update condition, the data content in the target data page is updated.
  2. 根据权利要求1所述的方法,其中,判断所述目标数据页是否满足数据更新条件包括:The method according to claim 1, wherein determining whether the target data page satisfies a data update condition comprises:
    在所述目标数据页中获取所述目标数据页的最大日志序列号;Acquiring the maximum log sequence number of the target data page in the target data page;
    在目标系统表中确定所述目标数据页对应的目标日志序列号,其中,所述目标系统表中包含所述数据库中各个数据表的最新元数据信息,以及所述最新元数据信息对应的操作日志的日志序列号;Determine the target log sequence number corresponding to the target data page in the target system table, where the target system table contains the latest metadata information of each data table in the database, and the operation corresponding to the latest metadata information The log sequence number of the log;
    若所述最大日志序列号和所述目标日志序列号不相同,则确定所述目标数据页满足所述数据更新条件,并对所述目标数据页中的数据内容进行更新。If the maximum log sequence number and the target log sequence number are not the same, it is determined that the target data page meets the data update condition, and the data content in the target data page is updated.
  3. 根据权利要求2所述的方法,其中,对所述目标数据页中的数据内容进行更新包括:The method according to claim 2, wherein updating the data content in the target data page comprises:
    按照所述目标系统表中目标数据表的最新元数据信息对所述目标数据页中的数据进行更新。The data in the target data page is updated according to the latest metadata information of the target data table in the target system table.
  4. 根据权利要求2所述的方法,其中,在目标系统表中确定所述目标数据页对应的目标日志序列号包括:The method according to claim 2, wherein determining the target log sequence number corresponding to the target data page in the target system table comprises:
    确定所述目标数据页的表空间ID信息;Determine the table space ID information of the target data page;
    在所述目标系统表查找与所述表空间ID信息相对应的日志序列号,并将查找到的日志序列号作为所述日志序列号。Look up the log sequence number corresponding to the table space ID information in the target system table, and use the searched log sequence number as the log sequence number.
  5. 根据权利要求2所述的方法,其中,所述方法还包括:The method according to claim 2, wherein the method further comprises:
    在检测到对所述目标数据表的元数据修改操作时,生成所述元数据修改操作所对应的操作日志,得到目标操作日志;When a metadata modification operation on the target data table is detected, an operation log corresponding to the metadata modification operation is generated to obtain a target operation log;
    基于所述目标操作日志对所述目标系统表进行更新。The target system table is updated based on the target operation log.
  6. 根据权利要求5所述的方法,其中,基于所述目标操作日志对所 述目标系统表进行更新包括:The method according to claim 5, wherein updating the target system table based on the target operation log comprises:
    确定所述目标操作日志的日志序列号和所述目标操作日志对应的元数据信息;Determining the log sequence number of the target operation log and metadata information corresponding to the target operation log;
    将所述目标日志序列号更新为所述目标操作日志的日志序列号;Updating the target log sequence number to the log sequence number of the target operation log;
    将所述目标系统表的最新元数据信息更新为所述目标操作日志对应的元数据信息。The latest metadata information of the target system table is updated to the metadata information corresponding to the target operation log.
  7. 根据权利要求5所述的方法,其中,所述元数据修改操作包括:The method according to claim 5, wherein the metadata modification operation comprises:
    对目标数据表中数据列的删除操作,和/或,对目标数据表中数据列的新增操作。Delete operations on data columns in the target data table, and/or add new operations on data columns in the target data table.
  8. 根据权利要求2所述的方法,其中,所述方法还包括:The method according to claim 2, wherein the method further comprises:
    若所述数据库为数据库主库,则对至少一个数据库从库中相对应的数据页中的数据进行更新;If the database is a database master database, update the data in the corresponding data page of at least one database slave database;
    在所述至少一个数据库从库中相对应的数据页中的数据更新结束之后,向用户返回更新结束的信息。After the update of the data in the corresponding data page of the at least one database from the database is completed, the update completion information is returned to the user.
  9. 一种数据库中数据更新装置,包括:A device for updating data in a database includes:
    获取单元,设置为获取对数据库中的目标数据页执行读写操作时生成的数据读写指令;The acquiring unit is configured to acquire the data read and write instructions generated when the target data page in the database is read and written;
    判断单元,设置为在获取到所述数据读写指令之后,判断所述目标数据页是否满足数据更新条件,其中,所述数据更新条件为对所述目标数据页中的数据内容进行更新的条件;The judging unit is configured to judge whether the target data page satisfies a data update condition after obtaining the data read and write instruction, wherein the data update condition is a condition for updating data content in the target data page ;
    更新单元,设置为若所述目标数据页满足所述数据更新条件,则对所述目标数据页中的数据内容进行更新。The update unit is configured to update the data content in the target data page if the target data page meets the data update condition.
  10. 一种电子设备,包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现上述所述权利要求1-8中任一项所述方法。An electronic device, comprising a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the processor implements the aforementioned claims 1-8 when the processor executes the computer program Any of the methods described.
  11. 一种具有处理器可执行的非易失的程序代码的计算机可读介质,所述程序代码使所述处理器执行上述所述权利要求1-8中任一项所述方法。A computer readable medium having non-volatile program code executable by a processor, the program code causing the processor to execute the method described in any one of the foregoing claims 1-8.
PCT/CN2021/080734 2020-05-28 2021-03-15 Method and device for updating data in database, and electronic device WO2021238341A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010471948.0 2020-05-28
CN202010471948.0A CN111639087B (en) 2020-05-28 2020-05-28 Data updating method and device in database and electronic equipment

Publications (1)

Publication Number Publication Date
WO2021238341A1 true WO2021238341A1 (en) 2021-12-02

Family

ID=72329762

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/080734 WO2021238341A1 (en) 2020-05-28 2021-03-15 Method and device for updating data in database, and electronic device

Country Status (2)

Country Link
CN (1) CN111639087B (en)
WO (1) WO2021238341A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111639087B (en) * 2020-05-28 2023-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment
CN114331135B (en) * 2021-12-29 2023-02-03 上海赛美特软件科技有限公司 Process flow management method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101295306A (en) * 2007-04-26 2008-10-29 国际商业机器公司 Operation method and corresponding device for modifying clause name in catalog server
CN103345501A (en) * 2013-06-27 2013-10-09 华为技术有限公司 Method and device for updating database
US20180144015A1 (en) * 2016-11-18 2018-05-24 Microsoft Technology Licensing, Llc Redoing transaction log records in parallel
CN110019066A (en) * 2017-09-21 2019-07-16 阿里巴巴集团控股有限公司 Data base processing method and device, system
CN111639087A (en) * 2020-05-28 2020-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10747776B2 (en) * 2012-12-04 2020-08-18 International Business Machines Corporation Replication control using eventually consistent meta-data
US9436724B2 (en) * 2013-10-21 2016-09-06 Sap Se Migrating data in tables in a database
JP2017162194A (en) * 2016-03-09 2017-09-14 富士通株式会社 Data management program, data management device, and data management method
US10671642B2 (en) * 2016-11-11 2020-06-02 International Business Machines Corporation Copying data changes to a target database
US20200050692A1 (en) * 2018-08-10 2020-02-13 Microsoft Technology Licensing, Llc Consistent read queries from a secondary compute node
CN109408588A (en) * 2018-09-07 2019-03-01 武汉达梦数据库有限公司 A kind of method of breakpoint transmission when database synchronization initialization loading
CN110442560B (en) * 2019-08-14 2022-03-08 上海达梦数据库有限公司 Log replay method, device, server and storage medium
CN110471909B (en) * 2019-08-26 2022-03-08 上海达梦数据库有限公司 Database management method, device, server and storage medium
CN110502507B (en) * 2019-08-29 2022-02-08 上海达梦数据库有限公司 Management system, method, equipment and storage medium of distributed database

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101295306A (en) * 2007-04-26 2008-10-29 国际商业机器公司 Operation method and corresponding device for modifying clause name in catalog server
CN103345501A (en) * 2013-06-27 2013-10-09 华为技术有限公司 Method and device for updating database
US20180144015A1 (en) * 2016-11-18 2018-05-24 Microsoft Technology Licensing, Llc Redoing transaction log records in parallel
CN110019066A (en) * 2017-09-21 2019-07-16 阿里巴巴集团控股有限公司 Data base processing method and device, system
CN111639087A (en) * 2020-05-28 2020-09-08 北京金山云网络技术有限公司 Data updating method and device in database and electronic equipment

Also Published As

Publication number Publication date
CN111639087B (en) 2023-09-08
CN111639087A (en) 2020-09-08

Similar Documents

Publication Publication Date Title
CN109906448B (en) Method, apparatus, and medium for facilitating operations on pluggable databases
US8065484B2 (en) Enhanced access to data available in a cache
US9171027B2 (en) Managing a multi-version database
WO2021238341A1 (en) Method and device for updating data in database, and electronic device
WO2016050112A1 (en) Data storage method, storage apparatus and storage system
US20160034582A1 (en) Computing device and method for executing database operation command
US8380663B2 (en) Data integrity in a database environment through background synchronization
US11294973B2 (en) Codeless information service for abstract retrieval of disparate data
US10762068B2 (en) Virtual columns to expose row specific details for query execution in column store databases
KR20150061945A (en) All-in-one data storage device having internal hardware filter, method thereof, and system having the data storage device
WO2015024406A1 (en) Data file management method and device
CN110543495A (en) cursor traversal storage method and device
WO2023202394A1 (en) Partition table creation method and apparatus, data writing method and apparatus for partition table, and data reading method and apparatus for partition table
US10324933B2 (en) Technique for processing query in database management system
US20070174329A1 (en) Presenting a reason why a secondary data structure associated with a database needs rebuilding
US20130060795A1 (en) Prepared statements to improve performance in database interfaces
WO2023098593A1 (en) Data processing method and apparatus
WO2022252322A1 (en) Method for synchronizing memory database and relational database of feature tag-based power grid monitoring system
US10803065B1 (en) Microcubes
CN114116907A (en) Database synchronization method and device, electronic equipment and storage medium
CN113918535A (en) Data reading method, device, equipment and storage medium
CN108694209B (en) Distributed index method based on object and client
EP3082050A1 (en) Mass data fusion storage method and system
CN112527911B (en) Data storage method, device, equipment and medium
US11609909B2 (en) Zero copy optimization for select * queries

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21812491

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21812491

Country of ref document: EP

Kind code of ref document: A1