CN111241190A - Page data processing method, device, equipment, medium and distributed system - Google Patents
Page data processing method, device, equipment, medium and distributed system Download PDFInfo
- Publication number
- CN111241190A CN111241190A CN201811445345.2A CN201811445345A CN111241190A CN 111241190 A CN111241190 A CN 111241190A CN 201811445345 A CN201811445345 A CN 201811445345A CN 111241190 A CN111241190 A CN 111241190A
- Authority
- CN
- China
- Prior art keywords
- page data
- page
- modified
- node
- modification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2308—Concurrency control
- G06F16/2336—Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
- G06F16/2343—Locking methods, e.g. distributed locking or locking implementation details
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a method, a device, equipment, a medium and a distributed system for processing page data, wherein the method comprises the following steps: loading a local page lock of the computing node of the acquired first page data; and modifying the first page data, and sending the modified first page data, so that the management node selects effective modified page data according to one or more modified first page data, and updates the first page data. According to the page data processing method provided by the embodiment of the invention, the interaction times among the nodes can be reduced, and the system processing performance is improved.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a method, an apparatus, a device, a medium, and a distributed system for processing page data.
Background
Distributed database systems typically include a plurality of nodes. When a node needs to modify a data page, in order to prevent the data page from being modified by other nodes at the same time to cause data disorder, a page lock needs to be added to the data page.
Under the distributed database system, when a page lock is added, another management node which is responsible for managing page lock information needs to be informed.
Specifically, when a node modifies a data page, it first needs to acquire a page lock of the data page from a management node, and then notifies the management node to release the page lock of the data page after the modified data page is submitted.
In the above process of modifying data, the node needs to interact with the management node twice. There is a delay of the order of microseconds to milliseconds for each interaction. If the number of modified data pages is large, the interaction between the computing nodes and the management node can increase the response time of the modified data, and further reduce the performance of the distributed database system.
Disclosure of Invention
The embodiment of the invention provides a page data processing method, a page data processing device, page data processing equipment, page data processing media and a distributed database system, which can reduce the response time of modifying data and further improve the performance of the distributed database system.
According to an aspect of the embodiments of the present invention, a method for processing page data is provided, including: loading a local page lock of the computing node of the acquired first page data; and modifying the first page data, and sending the modified first page data, so that the management node selects effective modified page data according to one or more modified first page data, and updates the first page data.
According to another aspect of the embodiments of the present invention, a method for processing page data is provided, including: receiving one or more modified first page data; selecting the first page data which is effective in modification from the one or more modified first page data; and updating the first page data by modifying the valid first page data.
According to still another aspect of the embodiments of the present invention, there is provided a node, including:
the control module is used for loading the local page lock of the computing node of the acquired first page data; and the modification module is used for modifying the first page data and sending the modified first page data so that the management node selects and modifies the effective first page data according to one or more modified first page data and updates the first page data.
According to still another aspect of the embodiments of the present invention, there is provided a node, including:
a receiving module, configured to receive one or more modified first page data; the judging module is used for selecting the first page data which is effectively modified from the one or more modified first page data; and the updating module is used for updating the first page data by modifying the effective first page data.
According to still another aspect of the embodiments of the present invention, there is provided a page data processing apparatus, including: a memory and a processor; the memory is used for storing programs; the processor is used for reading the executable program code stored in the memory to execute the processing method of the page data, and the processing method of the page data comprises the following steps: loading a local page lock of the computing node of the acquired first page data; and modifying the first page data, and sending the modified first page data, so that the management node selects effective modified page data according to one or more modified first page data, and updates the first page data.
According to still another aspect of the embodiments of the present invention, there is provided a page data processing apparatus, including: a memory and a processor; the memory is used for storing programs; the processor is used for reading the executable program code stored in the memory to execute the processing method of the page data, and the processing method of the page data comprises the following steps: receiving one or more modified first page data; selecting effective modified first page data from the one or more modified first page data; the first page data is updated by modifying the valid first page data.
According to still another aspect of the embodiments of the present invention, there is provided a distributed processing system including: the computing node is used for loading the local page lock of the computing node of the acquired first page data, modifying the first page data and sending the modified first page data; and the management node is used for receiving the one or more modified first page data, selecting the effective modified first page data from the one or more modified first page data, and updating the first page data by modifying the effective first page data.
According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to execute the above processing method of page data, where the processing method of page data includes: loading a local page lock of the computing node of the acquired first page data; and modifying the first page data, and sending the modified first page data, so that the management node selects effective modified page data according to one or more modified first page data, and updates the first page data.
According to another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, in which instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to execute the above processing method of page data, where the processing method of page data includes: receiving one or more modified first page data; selecting effective modified first page data from the one or more modified first page data; the first page data is updated by modifying the valid first page data.
According to the page data processing method, device, equipment, medium and distributed system in the embodiment of the invention, interaction between the computing node and the management node can be reduced, so that the response time for modifying the page data is reduced, and the performance of the distributed database system is further reduced.
Drawings
The present invention will be better understood from the following description of specific embodiments thereof taken in conjunction with the accompanying drawings, in which like or similar reference characters designate like or similar features.
FIG. 1 is a schematic diagram illustrating an application scenario according to an embodiment of the present invention;
fig. 2 is a timing diagram illustrating a processing method of page data according to an exemplary embodiment of the present invention;
fig. 3 is a timing diagram illustrating a processing method of page data according to another exemplary embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method of processing page data provided according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method of processing page data according to another embodiment of the present invention;
fig. 6 is a flowchart illustrating a processing method of page data according to an exemplary embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating a node structure according to one embodiment of the invention;
FIG. 8 is a diagram illustrating the structure of a management node according to one embodiment of the invention;
FIG. 9 is a block diagram illustrating an exemplary hardware architecture of a computing device capable of implementing the page data processing method and apparatus according to one embodiment of the present invention;
FIG. 10 is a block diagram illustrating an exemplary hardware architecture of a computing device capable of implementing a page data processing method and apparatus according to another embodiment of the present invention;
fig. 11 is a schematic structural diagram showing a distributed processing system according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
For a better understanding of the present invention, reference will now be made in detail to the embodiments of the present invention, which are illustrated in the accompanying drawings, and it should be noted that the embodiments are not intended to limit the scope of the present invention.
Fig. 1 is a schematic diagram illustrating an application scenario according to an embodiment of the present invention. As shown in fig. 1, the page data processing method according to the embodiment of the present invention may be applied to a distributed database system. In one embodiment, a distributed database system may include a management node, one or more compute nodes, such as a first compute node and a second compute node.
In one embodiment, a distributed database system includes a plurality of database systems distributed over a plurality of compute nodes. A distributed database system may provide shared storage, sharing of page data among multiple compute nodes, and page locks among nodes.
In one embodiment, a compute node may be a node with independent computing capabilities and an independent database system. As one example, each compute node may include a central processor, a database, and a terminal device.
In one embodiment, the management node is coupled to each of the plurality of computing nodes using a communications network.
In an embodiment of the present invention, the distributed database system needs to process a database transaction, the transaction is a logical work unit executed by the database in each computing node, and the database transaction may include a series of modifications performed by the logical work unit on a page, for example, a series of modifications includes, but is not limited to, modifying page data, and also includes other modifications made to the page.
In the description of the embodiments of the invention, the plurality of representations is 1 or more.
In one embodiment, a compute node may first send a page lock request to a management node, after the compute node acquires a page lock, lock and modify the page data locally, and then release the lock, but not notify the management node. When the database affairs of the modified page are submitted, the management nodes are notified uniformly, and the page lock is released. Because there is one interaction with the node managing page lock information during locking, there is one interaction after releasing the lock. Each interaction has a delay on the order of microseconds to milliseconds, which can slow down the response time of a transaction if the transaction modifies many pages.
The embodiment of the invention provides a page data processing method, which can be used for reducing the response time of modifying data and further improving the performance of a distributed database system. The following describes a method for processing page data according to some exemplary embodiments of the present invention with reference to fig. 2 and fig. 3.
Fig. 2 is a timing diagram illustrating a processing method of page data according to an exemplary embodiment of the present invention. As shown in fig. 2, in one embodiment, the method 100 for processing page data may include:
in step S01, the first compute node modifies the first page.
In this step, when the first compute node modifies the data stored in a certain page, a page lock may be set locally on the first compute node, that is, the local process of the first compute node needs to apply for the local page lock first, and then modify the data of the page, so as to avoid the page data confusion caused by the local process of the first compute node modifying the page at the same time, and reduce invalid modifications when the local node generates high-concurrency modifications to a certain extent.
In one embodiment, a first computing node applies for a local page lock of a first page, modifies the first page after obtaining the local page lock, and releases the local page lock of the first page after completing the modification of the first page.
In step S02, the first compute node sends the modified first page to the management node.
In step S03, the management node receives the modified first page, and if there is no other computing node to update the page at the same time, caches the modified first page.
As shown in FIG. 2, the management node may receive the modified page from one or more of the compute nodes within a predetermined specified time period. Because the probability of simultaneously modifying the same page among a plurality of computing nodes is low, if the management node only receives a certain page modified by one computing node in a specified time period, the modified page is cached, and the page data updated by the page is obtained.
In this embodiment, the management node caches the modified first page for other computing nodes to read the modified first page.
In step S04, the management node determines to modify the valid first page and generates page failure information of the first page.
In this step, if the management node receives only the first page modified by the first computing node, it is determined that the first page modified by the first computing node is a first page whose modification is valid.
In step S05, the management node sends page invalidation information of the first page to other computing nodes.
In one embodiment, the management node may send the invalidation information for the first page to other computing nodes in real-time or asynchronously. As an example of the asynchronous transmission, when determining to modify a valid first page, the management node triggers an event that generates page failure information of the first page, and transmits the page failure information of the first page to other computing nodes except the first computing node.
In step S06, the other computing node receives the page failure information of the first page.
In one embodiment, when the other computing node receives the page failure information of the first page, if the first page is being modified, the other computing node needs to undo the page data modified for the first page and request the updated first page from the management node to modify the updated first page.
In one embodiment, when the other computing nodes receive the page failure information of the first page, if the first page does not need to be modified, the other computing nodes may not request the updated first page from the management node, so as to avoid resource overhead and time overhead caused by unnecessary modification.
In this embodiment, if the other compute nodes receive page fault information for the first page, the latest version of the first page may be loaded from the management node before the first page is modified, reducing the probability of generating invalid modifications.
The following describes, with reference to fig. 3, a processing procedure when the management node receives a plurality of modified first pages within a specified time period by using a specific embodiment.
Fig. 3 is a timing diagram illustrating a processing method of page data according to another exemplary embodiment of the present invention. The same reference numerals are used for the same or equivalent steps in fig. 3 and 2. As shown in fig. 3, the method 200 for processing page data is substantially the same as the method 100 for processing page data, except that the method 200 for processing page data further includes:
in step S11, the second compute node modifies the first page.
In one embodiment, when the first computing node and the second computing node modify the first page, the system time for modifying the first page may be recorded respectively.
The system time of the first page includes one or more of: modifying a starting time point of the first page, modifying an ending time point of the first page, modifying a starting time point of a database transaction corresponding to the first page, and modifying an ending time point of a database transaction corresponding to the first page.
The system time is global system time, so the recorded system time is comparable among different computing nodes.
In one embodiment, the system time to modify the first page may be recorded in the modified first page.
Other details of the second computing node modifying the first page are the same as the specific details of the first computing node modifying the first page described in the above embodiment, and are not described herein again.
In step S12, the second compute node sends the modified first page to the management node.
Step S13, the management node receives the plurality of modified first pages, selects a first page that is effectively modified, and caches the first page that is effectively modified.
The following describes, by using a specific embodiment, a specific step of the management node selecting a first page that is valid for modification from the received multiple modified first pages.
In one embodiment, when the management node receives the plurality of modified first pages, the management node may select page data that is valid for modification from the plurality of modified first pages based on the system time for modifying the first pages.
In a specific embodiment, the management node may select the modified first page that takes the longest modification time as the first page that is valid for modification by determining the modification time period of the modified first page.
As an example, the modification elapsed time may be a time elapsed from a starting time point of the database transaction corresponding to the modification of the first page to an ending time point of the modification of the first page.
As a specific example, the first computing node modifies the database transaction corresponding to the first page to have a starting time point of 21 minutes 5.000001 seconds at 10 o 'clock of the day and a terminating time point of 21 minutes 5.000011 seconds at 10 o' clock of the day. Thus, it takes 10 microseconds for the first node to modify the first page.
The second computing node modifies the starting time point of the database transaction corresponding to the first page to be 21 minutes 5.000003 seconds at 10 points of the day, and modifies the ending time point of the first page to be 21 minutes 5.000012 seconds at 10 points of the day. Therefore, it takes 8 microseconds for the second compute node to modify the first page.
And determining the modified first page from the first computing node as a first page with effective modification because the management node receives two modified first pages in a specified time period and because the first node takes longer time to modify the first page.
In this specific embodiment, if the time consumed by a certain computing node to modify the first page is the longest, it indicates that the computing node may involve other modification processing on the first page in the transaction processing corresponding to the first page, and in order to avoid loss of modified data of the first page to the greatest extent, the modified first page corresponding to the modification time that takes the longest time is taken as the first page that is valid for modification.
In another embodiment, the management node may select the modified first page corresponding to the earliest modification time point as the first page effective for modification.
As an example, the earliest modification time point may be a start time point of modifying the first page or a start time point of modifying a database transaction to which the first page corresponds.
As a specific example, the starting time point of the database transaction corresponding to the first computing node modifying the first page is 21 minutes 5.000001 seconds at 10 o 'clock of the day, and the starting time point of the database transaction corresponding to the second computing node modifying the first page is 21 minutes 5.000003 seconds at 10 o' clock of the day.
Therefore, the starting time point of the database transaction corresponding to the first page modified by the first computing node is earlier than the starting time point of the database transaction corresponding to the first page modified by the second computing node. Thus, the management node may select the modified first page from the first compute node as the first page for which the modification is valid.
As a specific example, the first computing node modifies the first page to have a starting point in time of 5.000010 seconds at 10 o 'clock of the day, and the second computing node modifies the first page to have a starting point in time of 5.000004 seconds at 10 o' clock of the day.
It can be seen that the second computing node modifies the starting time point of the first page earlier than the first computing node modifies the starting time point of the first page. Thus, the management node may select the modified first page from the second compute node as the first page for which the modification is valid.
In this embodiment, if the starting time of a computing node modifying the first page is the earliest, it indicates that the computing node has the greatest possibility of modifying the content of the page data of the first page. In order to avoid the loss of the modified data of the page data to the maximum extent, the modified first page corresponding to the earliest modification starting time point is selected as the first page which is effective in modification.
In another embodiment, if the system time corresponding to the modified first page received by the management node is the same, the modified valid first page may be randomly selected, or the modified valid first page may be selected according to the valid sequence of the preset node weight or the preset node priority of the calculation node.
As can be seen from the embodiment, the system time is the same and includes any one of the following items: the time consumption for modifying the first page among different computing nodes is the same, the starting time point of the database transaction corresponding to the modified first page is the same, and the starting time point of the modified first page is the same.
As an example, if the first computing node priority is greater than the second computing node priority, and the second computing node priority is greater than the third computing node priority.
According to the sequence of the priorities of the computing nodes from large to small, selecting a first page modified by a first computing node with the highest priority as a first page effective to modification;
or according to the sequence of the priorities of the computing nodes from small to large, selecting the modified first page of the third computing node with the minimum priority as the first page with effective modification.
As can be seen from the above description of the embodiments, when determining to modify the valid first page, the management node has different selection policies, and the result of determining the valid first page may be different. The user may customize the specific policy in determining to modify the valid first page.
With continued reference to fig. 3, at step S04, the management node determines to modify the valid first page and generates page failure information for the first page.
In this step, if the management node receives a plurality of modified first pages, according to the method described in the above embodiment, a first page that is valid for modification is selected, and page failure information of the first page is generated.
In step S05, if the first page with the selected modification being valid comes from the first computing node, the management node sends page failure information of the first page to other computing nodes except the first node.
At step S06, the other computing node receives page failure information for the first page.
Other details of step S05 and step S06 are consistent with the details of step S05 and step S06 in the method for modifying a page described in conjunction with fig. 2, and will not be described herein again.
In this embodiment of the present invention, when the first computing node sends the modified first page to the management node, if the management node has determined that the modified first page from the second computing node is a modified valid first page according to one or more modified first pages received within a specified time period, the management node may determine that the modification of the first page by the first computing node is invalid. The management node may send page failure information for the first page to the first compute node. After receiving the page failure information of the first page, the first computing node can read the first page again from the management node, and modify page data according to the read local first page again.
In the processing process of the modified data, one computing node does not need to apply a page lock to the management node before modifying the page data of the first page to perform interaction with the management node once, and when the database transaction for modifying the page data of the first page is submitted, the computing node is informed to release the page lock and perform interaction with the management node once again; the first page may be modified first, and when the modified first page is submitted to the management node, it is determined whether the modified first page of the computing node is a first page whose modification is valid. Because the probability of changing the same page between different computing nodes is very small, the probability that the modified first page submitted by one computing node is determined as the effective modified first page is very large, and therefore, the interaction times of the computing nodes and the management nodes caused by page locking in the processing process are reduced by one time under most conditions. Therefore, the response time of database transactions is shortened, and the system performance is improved.
Fig. 4 is a flowchart illustrating a method for processing page data according to an embodiment of the present invention. In one embodiment, the processing method of page data shown in fig. 4 may be applied to a compute node. As shown in fig. 4, the method 400 for processing page data includes:
step S410, loading a local page lock of the computing node of the acquired first page data;
step S420, modify the first page data, and send the modified first page data, so that the management node selects the page data that is valid for modification according to one or more modified first page data, and updates the first page data.
In one embodiment, modifying the first page data and sending the modified first page data comprises:
modifying the first page data and sending the modified first page data;
receiving a message that the modified first page data is invalid;
the first page data is rolled back to a state prior to modifying the first page data based on the received message.
In one embodiment, the first page data is obtained from the management node according to the received message that the first page data is invalid.
In one embodiment, the page lock local to the compute node includes lock information in the compute node memory.
In the embodiment, when the number of the pages modified by the database transaction of the computing node is large, the response time of the transaction is slowed down.
Fig. 5 is a flowchart of a page data processing method according to another embodiment of the present invention. In one embodiment, the processing method of the page data shown in fig. 5 may be applied to a management node. As shown in fig. 5, a method 500 for processing page data includes:
step S510, the management node receives one or more modified first page data;
step S520, selecting effective modified first page data from the one or more modified first page data;
in step S530, the first page data is updated by modifying the valid first page data.
In an embodiment, the method 500 for processing page data may further include:
generating first page data failure information; and sending the first page data failure information to the computing nodes except the computing node corresponding to the modified effective first page data.
In an embodiment, the step of selecting a modified valid first page data from the one or more modified first page data may specifically include:
and selecting the first page data which is effective in modification from the one or more modified first page data according to the system time for modifying the first page data.
In one embodiment, selecting, from the one or more modified first page data, a first page data that is valid for modification according to a system time for modifying the first page data includes: and taking the modified first page data which takes the longest modification time as the effective modified first page data in the one or more modified first page data, wherein the modification time comprises the time consumed from the starting time point of the database transaction corresponding to the modified first page data to the modification ending time point corresponding to the modified first page data.
In one embodiment, selecting, from the one or more modified first page data, a first page data that is valid for modification according to a system time for modifying the first page data includes: and in the one or more modified first page data, taking the modified page data with the earliest modification time point as the first page data which is effective for modification, wherein the modification time point comprises the starting time point of the database transaction corresponding to the modified first page data or the corresponding modification starting time point.
In one embodiment, selecting the first page data with valid modification from the one or more modified first page data includes: and randomly selecting or selecting the first page data which is effective for modification from the one or more modified first page data according to the priority of the data node.
In an embodiment, the method 500 for processing page data may further include:
the first page data failure information is sent periodically or in a single time.
In this embodiment, when the same page is modified more concurrently, the management node may decide to modify the valid page and modify the invalid page according to a custom policy, generate page failure information, and notify other computing nodes through the page failure information, so that the other nodes read the latest page version, reduce the collision probability, and improve the processing efficiency of the page data.
Fig. 6 illustrates a flowchart of a processing method of page data according to an exemplary embodiment of the present invention. As shown in fig. 6, the management node, the first computing node Member1, and the second computing node Member2 in the embodiment of the present invention may operate in a clustered database architecture.
In one embodiment, the clustered database architecture may be, for example, a DB2 puresecale data architecture, and the management node may be a management component of the DB2 puresecale, such as a cluster cache facility CF, for managing shared resources among members, where the shared resources include a Global Buffer Pool (GBP) and a Global Lock Manager (GLM).
In this embodiment, each compute node may receive the client's connection and process the information for the application, and each compute node has its own local storage containing buffer pools and lock lists, among other things.
With continued reference to fig. 6, in the clustered database architecture, the step of modifying the data page by the compute node in the prior art includes:
step S61, when the second computing node needs to modify a data page, such as page 501, it sends a lock request to the management node;
step S62, the first computing node modifies the data page, and the management node informs the first computing node to release the lock;
step S63, after the first compute node finishes modifying, the lock is released, and the modified data page is transmitted to the global buffer pool GBP of the management node, and the global buffer pool caches the modified data page for being read by other compute nodes.
In step S64, the management node sends the updated data page to the second computing node.
In step S65, the second computing node caches the updated data page in the local buffer pool, and the application reads the updated data page in the local buffer pool to determine whether the data page needs to be modified continuously.
In order to reduce the response time of the management node to the computing node for modifying the page data and reduce the number of interactions with the management node when modifying the page, according to the page data processing method in the embodiment of the present invention, when the second computing node needs to modify a local data page, the method may include the following steps:
firstly, a second computing node loads a local page lock;
secondly, the second computing node modifies the data page and sends the modified data page to the management node, so that the management node can select a modified effective data page from the received at least one modified page data and update the data page.
In the embodiment of the invention, if the data page selected by the management node and effectively modified comes from the second computing node, the management node sends a modified page failure message to other computing nodes except the second computing node, and the other computing nodes rollback the data page to a state before modifying the first page data according to the modified page failure message.
Finally, if the second compute node needs to modify the page, the latest data page is loaded from the management node.
It can be seen from the foregoing embodiments that, in the method for processing page data according to the embodiments of the present invention, when a computing node modifies page data, the computing node does not need to apply for lock and release lock twice in interaction with a management node, but loads a local page lock, modifies a page, and transmits the modified page to the management node, so that the management node selects an effective page. The page data processing method can reduce the interaction times of the computing node and the management node caused by page locking by one time under most conditions, thereby improving the system performance and the processing efficiency.
Fig. 7 shows a schematic diagram of a node structure according to an embodiment of the invention. As shown in fig. 7, node 700 may include:
the control module 710 is configured to load a local page lock of the compute node of the acquired first page data;
and the modifying module 720 is configured to modify the first page data and send the modified first page data, so that the management node selects the first page data that is valid for modification according to one or more modified first page data, and updates the first page data.
Other details of the node according to the embodiment of the present invention are similar to the method for processing the page data by the computing node according to the embodiment of the present invention described above with reference to fig. 1 to 5, and are not repeated herein.
Fig. 8 shows a schematic structural diagram of a management node according to an embodiment of the present invention. As shown in fig. 8, node 800 may include:
a receiving module 810 for receiving one or more modified first page data;
a determining module 820, configured to select a first page data that is valid for modification from the one or more modified first page data;
an update module 830 for updating the first page data by modifying the valid first page data.
Other details of the node according to the embodiment of the present invention are similar to the method for processing the page data by the management node according to the embodiment of the present invention described above with reference to fig. 1 to 5, and are not repeated herein.
Fig. 9 is a block diagram illustrating an exemplary hardware architecture of a computing device capable of implementing the page data processing method and apparatus according to an embodiment of the present invention.
Computing device 900 includes input device 901, input interface 902, central processor 903, memory 904, output interface 905, and output device 906. The input interface 902, the central processing unit 903, the memory 904, and the output interface 905 are connected to each other through a bus 910, and the input device 901 and the output device 906 are connected to the bus 910 through the input interface 902 and the output interface 905, respectively, and further connected to other components of the computing device 900. Specifically, the input device 901 receives input information from the outside, and transmits the input information to the central processor 903 through the input interface 902; central processor 903 processes input information based on computer-executable instructions stored in memory 904 to generate output information, stores the output information temporarily or permanently in memory 904, and then transmits the output information to output device 906 via output interface 905; output device 906 outputs the output information external to computing device 900 for use by a user.
In one embodiment, the computing device 900 shown in FIG. 9 may be implemented as a processing system for page data that may include: a memory configured to store a program; and the processor is configured to run a program stored in the memory to execute the method for processing the page data by the data node described in the above embodiment in conjunction with fig. 1 to 5.
Fig. 10 is a block diagram illustrating an exemplary hardware architecture of a computing device capable of implementing a page data processing method and apparatus according to another embodiment of the present invention.
As shown in fig. 10, computing device 100 includes an input device 101, an input interface 102, a central processing unit 103, a memory 104, an output interface 105, and an output device 106. The input interface 102, the central processing unit 103, the memory 104, and the output interface 105 are connected to each other through a bus 1010, and the input device 101 and the output device 106 are connected to the bus 1010 through the input interface 102 and the output interface 105, respectively, and further connected to other components of the computing device 100. Specifically, the input device 101 receives input information from the outside and transmits the input information to the central processing unit 103 through the input interface 102; the central processor 103 processes the input information based on computer-executable instructions stored in the memory 104 to generate output information, stores the output information temporarily or permanently in the memory 104, and then transmits the output information to the output device 106 through the output interface 105; output device 106 outputs the output information to the exterior of computing device 100 for use by the user.
In one embodiment, the computing device 100 shown in FIG. 10 may be implemented as a processing system for page data that may include: a memory configured to store a program; a processor configured to execute the program stored in the memory to execute the method for processing the page data by the management node described in the above embodiments with reference to fig. 1 to 4 and fig. 7.
FIG. 11 shows a block diagram of a distributed processing system according to an embodiment of the invention. As shown in fig. 11, a distributed processing system 1100 may include:
the computing node 1110 is configured to load a local page lock of the computing node of the acquired first page data, modify the first page data, and send the modified first page data.
The management node 1120 is configured to receive one or more modified first page data, select a first page data that is valid for modification from the one or more modified first page data, and update the first page data by modifying the valid first page data.
Other details of the distributed processing system according to the embodiment of the present invention are similar to the method for processing page data by the compute node and the management node in the distributed database system according to the embodiment of the present invention described above with reference to fig. 1 to 5, and are not described herein again.
According to an embodiment of the invention, the process described above with reference to the flow chart may be implemented as a computer software program. For example, embodiments of the invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network, and/or installed from a removable storage medium.
It is to be understood that the invention is not limited to the particular arrangements and instrumentality described in the above embodiments and shown in the drawings. For convenience and brevity of description, detailed description of a known method is omitted here, and for the specific working processes of the system, the module and the unit described above, reference may be made to corresponding processes in the foregoing method embodiments, which are not described herein again.
According to an embodiment of the invention, the process described above with reference to the flow chart may be implemented as a computer software program. For example, embodiments of the invention include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network, and/or installed from a removable storage medium.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions which, when run on a computer, cause the computer to perform the method described in the various embodiments above. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), among others.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (18)
1. A page data processing method comprises the following steps:
loading a local page lock of the computing node of the acquired first page data;
and modifying the first page data, and sending the modified first page data, so that the management node selects effective modified page data according to one or more modified first page data, and updates the first page data.
2. The method for processing page data according to claim 1, wherein said modifying said first page data and sending the modified first page data includes:
modifying the first page data and sending the modified first page data;
receiving the modified first page data failure message;
and rolling back the first page data to a state before the first page data is modified according to the received message.
3. A page data modifying method according to claim 1,
the first page data is obtained from the management node according to the received message that the first page data is invalid.
4. A method for processing page data according to claim 1, wherein the page lock local to the compute node comprises lock information in a memory of the compute node.
5. A page data processing method comprises the following steps:
receiving one or more modified first page data;
selecting the first page data which is effective in modification from the one or more modified first page data;
and updating the first page data by modifying the valid first page data.
6. The page data processing method according to claim 5, further comprising:
generating first page data failure information;
and sending the first page data failure information to the computing nodes except the computing node corresponding to the modified effective first page data.
7. The method for processing page data according to claim 5, wherein said selecting the first page data with valid modification from the one or more modified first page data includes:
and selecting and modifying effective first page data in one or more modified first page data according to the system time for modifying the first page data.
8. The method for processing page data according to claim 7, wherein the selecting, from the one or more modified first page data, the first page data that is valid for modification according to the system time for modifying the first page data includes:
among the one or more modified first page data, the modified first page data which takes the longest modification time is taken as the first page data which is valid for modification,
the modification consumed time comprises the time consumed from the starting time point of the database transaction corresponding to the modified first page data to the modification ending time point corresponding to the modified first page data.
9. The method for processing page data according to claim 7, wherein the selecting, from the one or more modified first page data, the first page data that is valid for modification according to the system time for modifying the first page data includes:
among the one or more modified first page data, the modified page data whose modification time point is the earliest is taken as the first page data whose modification is valid,
the modification time point comprises a starting time point of a database transaction corresponding to the modified first page data or a corresponding modification starting time point.
10. The page data processing method according to claim 5 or 7, wherein the selecting, from the one or more modified first page data, a first page data whose modification is valid includes:
and randomly selecting or selecting the first page data which is effectively modified from the one or more modified first page data according to the priority of the computing node.
11. The page data processing method according to claim 6, further comprising:
and periodically or singly sending the first page data failure information.
12. A node, comprising:
the control module is used for loading the local page lock of the computing node of the acquired first page data;
and the modification module is used for modifying the first page data and sending the modified first page data so that the management node selects and modifies the effective first page data according to one or more modified first page data and updates the first page data.
13. A node, comprising:
a receiving module, configured to receive one or more modified first page data;
the judging module is used for selecting the first page data which is effectively modified from the one or more modified first page data;
and the updating module is used for updating the first page data by modifying the effective first page data.
14. A distributed processing system, comprising:
the computing node is used for loading the local page lock of the computing node of the acquired first page data, modifying the first page data and sending the modified first page data;
and the management node is used for receiving one or more modified first page data, selecting the first page data which is effective in modification from the one or more modified first page data, and updating the first page data through the effective first page data.
15. A device for processing page data, comprising:
the memory is used for storing executable program codes;
the processor is used for reading the executable program code stored in the memory to execute the page data processing method of any one of claims 1 to 4.
16. A device for processing page data, comprising:
the memory is used for storing executable program codes;
the processor is used for reading the executable program code stored in the memory to execute the page data processing method of any one of claims 5 to 11.
17. A computer-readable storage medium characterized by comprising instructions that, when executed on a computer, cause the computer to execute the processing method of page data according to any one of claims 1 to 4.
18. A computer-readable storage medium characterized by comprising instructions that, when executed on a computer, cause the computer to execute the method of processing page data according to any one of claims 5 to 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811445345.2A CN111241190A (en) | 2018-11-29 | 2018-11-29 | Page data processing method, device, equipment, medium and distributed system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811445345.2A CN111241190A (en) | 2018-11-29 | 2018-11-29 | Page data processing method, device, equipment, medium and distributed system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111241190A true CN111241190A (en) | 2020-06-05 |
Family
ID=70877472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811445345.2A Pending CN111241190A (en) | 2018-11-29 | 2018-11-29 | Page data processing method, device, equipment, medium and distributed system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111241190A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5226143A (en) * | 1990-03-14 | 1993-07-06 | International Business Machines Corporation | Multiprocessor system includes operating system for notifying only those cache managers who are holders of shared locks on a designated page by global lock manager |
| US20060010170A1 (en) * | 2004-07-08 | 2006-01-12 | International Business Machines Corporation | Method and system using virtual replicated tables in a cluster database management system |
| CN104573112A (en) * | 2015-01-30 | 2015-04-29 | 华为技术有限公司 | Page query method and data processing node for OLTP cluster database |
| CN106575238A (en) * | 2014-09-26 | 2017-04-19 | 华为技术有限公司 | Concurrency control in shared storage architecture supporting on-page implicit locks |
| CN108073656A (en) * | 2016-11-17 | 2018-05-25 | 杭州华为数字技术有限公司 | A kind of method of data synchronization and relevant device |
-
2018
- 2018-11-29 CN CN201811445345.2A patent/CN111241190A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5226143A (en) * | 1990-03-14 | 1993-07-06 | International Business Machines Corporation | Multiprocessor system includes operating system for notifying only those cache managers who are holders of shared locks on a designated page by global lock manager |
| US20060010170A1 (en) * | 2004-07-08 | 2006-01-12 | International Business Machines Corporation | Method and system using virtual replicated tables in a cluster database management system |
| CN106575238A (en) * | 2014-09-26 | 2017-04-19 | 华为技术有限公司 | Concurrency control in shared storage architecture supporting on-page implicit locks |
| CN104573112A (en) * | 2015-01-30 | 2015-04-29 | 华为技术有限公司 | Page query method and data processing node for OLTP cluster database |
| CN108073656A (en) * | 2016-11-17 | 2018-05-25 | 杭州华为数字技术有限公司 | A kind of method of data synchronization and relevant device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105700939B (en) | The method and system of Multi-thread synchronization in a kind of distributed system | |
| CN107943594B (en) | Data acquisition method and device | |
| US6792510B1 (en) | System and method for updating a cache | |
| US8301779B2 (en) | Mechanisms for obtaining access to shared resources using a single timestamp technique | |
| US10083199B1 (en) | Data migration in active data store | |
| CN108090058B (en) | High-concurrency activity interaction method | |
| US7328213B2 (en) | Transaction processing method, transaction control apparatus and program thereof | |
| CN108694199A (en) | Data synchronization unit, method, storage medium and electronic equipment | |
| US9189303B2 (en) | Shadow queues for recovery of messages | |
| US9836331B2 (en) | Shared resource updating | |
| US9104714B2 (en) | Incremental optimistic locking of data distributed on multiple nodes to avoid transaction deadlock | |
| US10713378B2 (en) | Techniques for eventually consistent inheritance propagation for nested content item namespaces | |
| CN112052264B (en) | Business data query method and device, electronic equipment and readable storage medium | |
| CN111741110B (en) | Message pushing system and method | |
| CN115544044A (en) | Data consistency keeping method, device, equipment and storage medium | |
| Pandey et al. | RAPID: A real time commit protocol | |
| CN112965837A (en) | Configuration and service hot overload updating method, device, computer equipment and storage medium | |
| CN112819638A (en) | Transaction weight judging method, device, equipment and computer readable medium | |
| CN111241190A (en) | Page data processing method, device, equipment, medium and distributed system | |
| CN108037897B (en) | Metadata diffusion method, computer device and readable storage medium | |
| CN114500416A (en) | Delivery method and delivery system for at most one message delivery | |
| CN112783613B (en) | Method and device for scheduling units | |
| CN112182003A (en) | Data synchronization method and device | |
| US11281542B2 (en) | System and method for backup generation for deployments | |
| Al-Jumah et al. | Implementation and modeling of two-phase locking concurrency control—a performance study |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |