CN117251119B - Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium - Google Patents

Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium Download PDF

Info

Publication number
CN117251119B
CN117251119B CN202311526544.7A CN202311526544A CN117251119B CN 117251119 B CN117251119 B CN 117251119B CN 202311526544 A CN202311526544 A CN 202311526544A CN 117251119 B CN117251119 B CN 117251119B
Authority
CN
China
Prior art keywords
node
mask
bitmap
bitmap data
local
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311526544.7A
Other languages
Chinese (zh)
Other versions
CN117251119A (en
Inventor
马明慧
朱红玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Priority to CN202311526544.7A priority Critical patent/CN117251119B/en
Publication of CN117251119A publication Critical patent/CN117251119A/en
Application granted granted Critical
Publication of CN117251119B publication Critical patent/CN117251119B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1458Management of the backup or restore process
    • G06F11/1464Management of the backup or restore process for networked environments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of computer data storage, and discloses a bitmap processing method, a device, computer equipment, a control system and a storage medium, wherein the bitmap processing method is applied to any node in a multi-node control system, and the method comprises the following steps: when an offline event is received, acquiring first domain attribution information of a strip; setting a first mask and a second mask according to the first domain attribution information; performing first preprocessing on the first local bitmap data by using a first mask so as to enable the first local bitmap data to complete first reconstruction; performing second preprocessing on the first received bitmap data by using a second mask so as to enable the first received bitmap data to complete second reconstruction; and combining the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment. Therefore, reconstruction in two nodes needing to be combined can be completed, and the defect of loss of the strip information after the combination is overcome.

Description

Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium
Technical Field
The present invention relates to the field of computer data storage technologies, and in particular, to a bitmap processing method, a bitmap processing device, a computer device, a control system, and a storage medium.
Background
RAID (Redundant Array of Independent Disks ) refers to an array of a series of hard disks. Management of fault tolerant operations and recovery of underlying disks is performed in RAID. At the software layer, RAID for operation management of the hard disk array is commonly called soft RAID, namely a software system for protecting consistency of disk data realized at the software layer, and background tasks such as initialization, reconstruction, check reconstruction, slow disk repair, inspection and the like are formed in the process of maintaining consistency of disk information by RAID at the layer so as to judge and monitor abnormal conditions and correct and recover data after the abnormal conditions. The background task refers to a task for maintaining consistency of disk data, which is executed in the background by RAID (redundant array of independent disks) except tasks such as read-write io and the like.
During the execution of these background tasks, RAID records the execution of these background tasks by managing a memory space, and for some background tasks such as initialization, reconstruction, etc., recording whether all stripes on an array are executed is generally performed using bitmap (bitmap), where bitmap refers to a data structure that saves a memory space storing indicating data information by indicating a state (0 or 1) of one level with one 1bit in a data basic type (used int/used long), where unsigned is an integer type representing 0 or more in a computer programming language; unsigned long int, referred to as unsigned long integer in the C language, is one of the integer (integer type) variables. Currently, in the bitmap processing process, there is a situation that stripe information is lost.
Disclosure of Invention
In view of the above, the present invention provides a bitmap processing method, apparatus, computer device, control system and storage medium, so as to solve the problem of stripe information loss during bitmap processing.
In a first aspect, an embodiment of the present invention provides a bitmap processing method, which is applied to any node in a multi-node control system, and the method includes the following steps: when an offline event is received, acquiring first domain attribution information of a strip; setting a first mask and a second mask according to the first domain attribution information; performing first preprocessing on the first local bitmap data by using a first mask so as to enable the first local bitmap data to complete first reconstruction; performing second preprocessing on the first received bitmap data by using a second mask so as to enable the first received bitmap data to complete second reconstruction; and combining the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment to obtain second local bitmap data.
Because the execution progress of the reconstruction in the two nodes needing to be combined is not the same, when the task is repeatedly executed, the execution area is marked along with the progress of another task, and the bitmap space is multiplexed, the situation of strip information loss exists; therefore, the embodiment of the invention sets the first mask and the second mask according to the domain attribution information, respectively performs the first preprocessing on the first local bitmap data by using the first mask, and performs the second preprocessing on the first received bitmap data by using the second mask, thereby completing reconstruction execution in two nodes needing to be combined, and overcoming the defect of stripe information loss after combination.
In an alternative embodiment, the bitmap processing method further includes the steps of: acquiring a metadata header; judging whether the metadata heads are the same; when the metadata headers are the same, it is determined that an offline event is received.
Thus, the offline event can be accurately identified.
In an alternative embodiment, the bitmap processing method further includes the steps of: when the metadata header is different, it is determined that a regression event is received.
Thus, regression events can be accurately identified.
In an alternative embodiment, the bitmap processing method further includes the steps of: after receiving the regression event, after receiving bitmap data with preset length sent by the current merging node; acquiring progress information of each node combined with the regression node; determining whether the current merging node is the node with the fastest progress according to the progress information of each node; and when the current merging node is the node with the fastest progress, merging the current merging node and the regression node according to a preset first merging mode.
When the node is regressed, the node which is about to become the own node after the reorganization and the node which becomes the partner node send the bitmap information of the whole quantity of the own node to the regressed node, and the sending logic is sequentially and partially sent, and the two nodes send the bitmaps at different speeds and different rates, so that the scene of interaction of the first merging and the second merging exists, and the mode adopted by the first merging and the second merging is the same, so that the proper merging mode can be obtained by judging whether the current merging node is the node with the fastest progress.
In an alternative embodiment, merging the current merging node with the regression node according to a preset first merging mode includes: acquiring second domain attribution information of the strip; setting a third mask according to the second domain attribution information; performing third preprocessing on the second received bitmap data by using a third mask; and taking the second received data after the third pretreatment as third local bitmap data of the regression node.
The second local data bitmap can be subjected to third preprocessing by utilizing a third mask, so that reconstruction execution in the current merging node is completed, and the defect of loss of the merged stripe information is overcome.
In an alternative embodiment, setting the third mask according to the second domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1 and setting the bitmap of the stripes of domains not belonging to the node to 0, thereby obtaining a third mask; and/or setting the bitmap of the stripes belonging to the domains of the node to 0 and setting the bitmap of the stripes not belonging to the domains of the node to 1 to obtain a fourth mask; and inverting the fourth mask to obtain a third mask.
Therefore, an accurate third mask can be obtained, third preprocessing is carried out on the second received bitmap data by using the third mask, and reconstruction execution in the current merging node is further completed, so that the defect of loss of the strip information after merging is overcome.
In an alternative embodiment, performing the third preprocessing of the second received bitmap data with the third mask includes: the second received bitmap data is bitwise ored with a third mask.
By adopting the processing mode, the reconstruction in the current merging node can be completed, so that the defect of loss of the strip information after merging is overcome.
In an alternative embodiment, the bitmap processing method further includes the following steps: and when the current merging node is not the node with the fastest progress, merging the current merging node and the regression node according to a preset second merging mode.
As described above, when the nodes regress, because the node which is about to become the own node and the node which becomes the part node after the reorganization will send the bitmap information of the whole amount of the node to the regressive node, and the sending logic is all sent in sequence, because the two nodes send the bitmaps with different progress and different speed, the scene of interaction between the first merging and the second merging exists, and because the modes adopted by the first merging and the second merging are the same, whether the current merging node is the node with the fastest progress can be judged, so that a proper merging mode can be obtained.
In an alternative embodiment, merging the current merging node with the regression node according to a preset second merging mode includes: acquiring third domain attribution information of the strip; setting a fifth mask and a sixth mask according to the third domain attribution information; performing fourth preprocessing on fourth local bitmap data by using a fifth mask; performing fifth preprocessing on the third received bitmap data by using a sixth mask; and combining the fourth local bitmap data subjected to the fourth pretreatment with the third receiving bitmap data subjected to the fifth pretreatment.
When the regression event is received and the regression event is the second merging, the fifth mask can be utilized to perform the third preprocessing on the fourth local bitmap data, and the sixth mask can be utilized to perform the third preprocessing on the third received bitmap data, so that the reconstruction in two nodes needing to be merged can be completed, and the defect of the loss of the strip information after the merging is overcome.
In an alternative embodiment, setting the first mask and the second mask according to the first domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 0 and setting the bitmap of the stripes of domains not belonging to the node to 1 to obtain a first mask; bit inverting the first mask to obtain a second mask; or alternatively; setting the fifth mask and the sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 0 and setting the bitmap of the stripes of domains not belonging to the node to 1 to obtain a fifth mask; the fifth mask is bit-wise inverted to obtain a sixth mask.
Therefore, the accurate first mask and the second mask can be obtained, and the reconstruction execution in the current merging node is further completed, so that the defect of loss of the merged stripe information is overcome; and/or, the accurate fifth mask and sixth mask can be obtained, so that the reconstruction in the current merging node is completed, and the defect of loss of the information of the merged stripes is overcome.
In an alternative embodiment, setting the first mask and the second mask according to the first domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain a second mask; bit inverting the second mask to obtain a first mask; and/or; setting the fifth mask and the sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1 and setting the bitmap of the stripes of domains not belonging to the node to 0, thereby obtaining a sixth mask; the sixth mask is bit-wise inverted to obtain a fifth mask.
Therefore, the accurate first mask and the second mask can be obtained, and the reconstruction execution in the current merging node is further completed, so that the defect of loss of the merged stripe information is overcome; and/or, the accurate fifth mask and sixth mask can be obtained, so that the reconstruction in the current merging node is completed, and the defect of loss of the information of the merged stripes is overcome.
In an alternative embodiment, first preprocessing the first local bitmap data with the first mask includes: performing bit OR operation on the first local bitmap data and the first mask; and/or performing fourth preprocessing on fourth local bitmap data with the fifth mask includes: the fourth local bitmap data is bitwise ored with a fifth mask.
By adopting the processing mode, the reconstruction in the current merging node can be completed, so that the defect of loss of the strip information after merging is overcome.
In an alternative embodiment, performing the second preprocessing of the first received bitmap data using the second mask includes: performing bit OR operation on the first received bitmap data and the second mask; and/or performing fifth preprocessing on the third received bitmap data with the sixth mask includes: the third received bitmap data is bitwise ored with a sixth mask.
By adopting the processing mode, the reconstruction in the current merging node can be completed, so that the defect of loss of the strip information after merging is overcome.
In an alternative embodiment, combining the first local bitmap data after the first preprocessing with the first received bitmap data after the second preprocessing includes: performing bit-wise and operation on the first local bitmap data subjected to the first pretreatment and the first received bitmap data subjected to the second pretreatment; and/or combining the fourth local bitmap data subjected to the fourth preprocessing with the third received bitmap data subjected to the fifth preprocessing includes: and performing bit-wise AND operation on the fourth local bitmap data subjected to the fourth pretreatment and the third receiving bitmap data subjected to the fifth pretreatment.
By adopting the processing mode, the reconstruction in the current merging node can be completed, so that the defect of loss of the strip information after merging is overcome.
In an alternative embodiment, obtaining the first domain home information for the stripe includes: acquiring a preset band attribution mapping relation; and acquiring first domain attribution information in a band attribution mapping relation according to a preset band length.
Therefore, the first domain attribution information of the strip can be accurately and conveniently obtained.
In an alternative embodiment, before the first preprocessing of the first local bitmap data with the first mask, further comprising: a first reconstruction is performed on the first local bitmap data.
This prevents the data consistency from being affected by uninitialization after the home wner change of the strip.
In an alternative embodiment, the first reconstructing the first local bitmap data includes: for the owner stripe, marking/initializing whether to be reconstructed according to the logic of yes/no need; the non-owner stripes are all initialized.
Thereby preventing unaitialization after the home wner change of the stripe from affecting data consistency.
In a second aspect, an embodiment of the present invention further provides a bitmap processing apparatus, which is applied to any node in a multi-node control system, where the apparatus includes a first acquisition module, a mask setting module, a local data bitmap preprocessing module, a received bitmap data preprocessing module, and a merging module; when an offline event is received, a first acquisition module is used for acquiring first domain attribution information of the strip; the mask setting module is used for setting a first mask and a second mask according to the first domain attribution information; the local data bitmap preprocessing module is used for performing first preprocessing on the first local bitmap data by utilizing a first mask so as to enable the first local bitmap data to complete first reconstruction; the received bitmap data preprocessing module is used for performing second preprocessing on the first received bitmap data by utilizing a second mask so as to enable the first received bitmap data to complete second reconstruction; the merging module is used for merging the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment to obtain second local bitmap data.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory and a processor, where the memory and the processor are communicatively connected to each other, and the memory stores computer instructions, and the processor executes the computer instructions, so as to execute the bitmap processing method of the first aspect or any implementation manner corresponding to the first aspect.
In a fourth aspect, embodiments of the present invention also provide a multi-controller system, including the computer device of the third aspect.
In a fifth aspect, an embodiment of the present invention further provides a computer readable storage medium, where computer instructions are stored on the computer readable storage medium, where the computer instructions are configured to cause a computer to execute the bitmap processing method of the first aspect or any implementation manner corresponding to the first aspect.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a merge mode after a node fails in a multi-controller system;
FIG. 2 is a schematic diagram of a merging effect of executing background tasks once in the case of initializing to all 1's after a node in a multi-controller system fails in the related art;
FIG. 3 is a schematic diagram of merging effect in the case of marking an execution area and multiplexing bitmap space with another task when the task is repeatedly executed after a failure occurs in one node in the multi-controller system in the related art;
FIG. 4 is a flow chart of a bitmap processing method according to an embodiment of the present invention;
FIG. 5 is a flow chart of another bitmap processing method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of merging effect in the case where an execution area is marked and a bitmap space is multiplexed as another task proceeds when the task is repeatedly executed after a failure occurs in one node in the multi-controller system according to an embodiment of the present invention;
FIG. 7 is a flow chart of yet another bitmap processing method according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a merging mode after one node in the multi-controller system is regressed;
FIG. 9 is a schematic diagram of a first merge and a second merge after regression of a node in a multi-controller system;
FIG. 10 is a block diagram of a bitmap processing apparatus according to an embodiment of the present invention;
Fig. 11 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Since in the multi-controller system, each node (a controller in the multi-controller system) maintains bitmap information of its own jurisdiction, when the background task is executed, metadata (data information indicating the execution condition of the background task, most bitmap data) information of a stripe or the like of the execution completion of the background task is synchronized to the opposite node, so as to ensure redundancy of the information. When the controller fails, in order to prevent the metadata information from being lost, the memory management module can combine all the metadata information of the failed node onto the surviving node, and similarly, when the offline node is online again, the metadata information is combined onto the offline node which is online again from the surviving node, so that the metadata information is prevented from being lost, the accuracy and the reliability are ensured, and the accurate and efficient execution of the background task is ensured.
As shown in fig. 1, four nodes of node0 (node 0), node1 (node 1), node2 (node 2), and node3 (node 4) are included in the multi-controller system. As described above, when the background task is executed, the metadata information of the stripe executed by the node is synchronized to the peer node (partner) to ensure redundancy of the information, and a domain (domain) is formed between the node initiating metadata synchronization and the node receiving metadata synchronization, and in fig. 1, an arrow indicates a metadata synchronization direction. In FIG. 1, node0 and node1 constitute a domain; node1 and node2 comprise a domain; node2 and node3 form a domain; node3 and node0 constitute a domain. Where a stripe refers to a memory unit in an array area of RAID management, here as a unit of 1bit indication in bitmap management.
As shown in fig. 1, in domain composed of node0 and node1, node0 is a node initiating metadata synchronization, and node1 is a node receiving metadata synchronization, that is, bitmap information of a stripe whose execution is completed by the node0 is recorded in addition to bitmap information of a stripe whose execution is completed by the node 1.
In domains composed of node1 and node2, node1 is a node that initiates metadata synchronization, and node2 is a node that receives metadata synchronization. That is, in node2, bitmap information of a stripe whose execution by node1 is completed is recorded in addition to bitmap information of a stripe whose execution by the node is completed.
In domains composed of node 2 and node 3, node 2 is a node that initiates metadata synchronization, and node 3 is a node that receives metadata synchronization. That is, in node 3, bitmap information of a stripe whose execution by node 2 is completed is recorded in addition to bitmap information of a stripe whose execution by the node is completed.
In domain composed of node 3 and node 0, node 3 is a node that initiates metadata synchronization, and node 0 is a node that receives metadata synchronization. That is, in node 0, bitmap information of a stripe whose execution by node 3 is completed is recorded in addition to bitmap information of a stripe whose execution by the node is completed.
As shown in fig. 1, when node 3 fails, since bitmap information of a stripe completed by executing node 3 is recorded on node 0, node 0 takes over domain of original node 3 after domain reorganization (also referred to as domain reorganization) after node 3 goes offline.
As described above, the original node 3 has recorded thereon bitmap information of the stripe whose execution by the present node is completed and bitmap information of the stripe whose execution by the node 2 is completed. After the node 3 is offline and domain reorganization is performed, the bitmap information of the stripe completed by the original node 3 is recorded on the node 0, so that the bitmap information of the stripe completed by the original node 3 is not required to be processed, but the bitmap information of the stripe completed by the original node 2 is not recorded on the node 0, and the bitmap information of the stripe completed by the original node 2 is required to be combined on the original node 0 in domain reorganization, so that metadata loss caused by node failure is avoided.
Note that, in the multi-controller system, a full-volume bitmap is stored for each node. For example, in node 0, the own node area stores bitmap information of a band corresponding to node 0, and the non-own node area stores bitmap information of a band corresponding to node 1, a band corresponding to node2, and a band corresponding to node 3. In the node2, the owner area holds bitmap information of a band corresponding to the node2, and the non-owner area holds bitmap information of a band corresponding to the node 0, a band corresponding to the node 1, and a band corresponding to the node 3.
The following describes a process of merging bitmap information of a stripe, which has been executed by the original node2, onto the original node 0, taking a failure of the node 3 as an example.
As shown in fig. 2, node x may be understood as node 0 and node y may be understood as node 2. The bitmaps of node 0 and node2 each include an own area and a non own area, wherein the own area records the bitmap information of the band completed by the node, and the non own area records the bitmap information of the band completed by other nodes in the multi-controller system except the node.
In fig. 2, only partial bitmaps of node 0 and node 3 are shown due to the spatial limit. Only bitmap information of a band corresponding to the node 0 (i.e., the owner area) and bitmap information of a band corresponding to the node2 are displayed for the node 0; only the band bitmap information (i.e., the owner area) corresponding to the node2 and the bitmap information of the band corresponding to the node 0 are displayed for the node 2.
In the multi-controller system, the execution condition of the stripes of the array area under the task is indicated by initializing the bitmap of the whole quantity to be 1 and setting the bit corresponding to the executed stripes to be 0 in the execution process of the background task. In fig. 2, the owner area in node 0, which belongs to domain of node 0, so the band execution during task execution is marked in the owner area; before domain reorganization, the stripe corresponding to node 2 does not belong to domain of node 0, so that the stripe execution condition in the task execution process does not need to be marked in node 0, and therefore, as shown in fig. 2, the bitmap is 1 in the area outside the owner in node 0.
Also, the owner area in node 2, which belongs to domain of node 2, so that the band execution situation during the task execution is marked in the owner area; before domain reorganization, the stripe corresponding to node 0 does not belong to domain of node 2, so that the stripe execution condition in the task execution process does not need to be marked in node 2, and therefore, as shown in fig. 2, the bitmap is 1 in the area outside the owner in node 2.
As above, after node 3 fails, node 0 takes over the domain of original node 3 (including the bitmap information of the stripe that node 3 performed completed and the bitmap information of the stripe that node 2 performed completed). As described above, since the bitmap information of the stripe whose execution is completed by the node 3 is originally recorded on the node 0, it is unnecessary to process the bitmap information of the stripe whose execution is completed by the node 3, but since the bitmap information of the stripe whose execution is completed by the node 2 is not recorded on the node 0, it is necessary to merge the bitmap information of the stripe whose execution is completed by the node 2 onto the node 0 in domain reorganization to avoid metadata loss caused by a node failure.
The process of merging bitmap information of the stripe completed by the execution of node 2 onto node 0 is: the corresponding position is bit-wise and. In fig. 2, the lowermost line is bitmap information on node 0 after merging.
It should be noted that, the above merging method is only applicable to the case of executing the background task once when initializing to all 1's, but cannot satisfy the correctness of merging when marking the execution area and multiplexing the bitmap space as another task proceeds when the task is repeatedly executed.
In fig. 3, the node x may be understood as the node 0 and the node y may be understood as the node 2, as in fig. 2. The bitmaps of node 0 and node 2 each include an own area and a non own area, wherein the own area records the bitmap information of the band completed by the node, and the non own area records the bitmap information of the band completed by other nodes in the multi-controller system except the node.
In a multi-controller system, when a task is completed, bitmap information of each node in the multi-to-multi-controller system is required to be reconstructed.
As shown in fig. 3, for the owner area of node 0, for the stripe that needs to be reconstructed, no initialization is marked; for the strip that does not need to be reconstructed, then flag initialization (i.e., flag whether/initialize the strip of owner according to yes/no need to reconstruct logic); all bands are initialized for the other areas in node 0 except for owner. Wherein initialization refers to bit map set 1 for the stripe. In fig. 3, the progress of the execution of the reconstruction in node 0 is marked.
Also, for the owner region of node 2, for the stripe that needs to be reconstructed, no initialization is marked; for the strip that does not need to be reconstructed, then flag initialization (i.e., flag whether/initialize the strip of owner according to yes/no need to reconstruct logic); all bands are initialized for the other areas in node 2 except for the owner. Wherein initialization refers to bit map set 1 for the stripe. In fig. 3, the progress of the execution of the reconstruction in node 2 is marked.
As shown in fig. 3, since the execution progress of the reconstruction in the node 0 is different from the execution progress of the reconstruction in the node 2, if the node 3 fails at this time, after domain reorganization, the bitmap information of the stripe of which the execution of the node 2 is completed needs to be merged onto the node 0, and the merging result is the last line in fig. 3, and since the execution progress of the reconstruction in the node 0 is different from the execution progress of the reconstruction in the node 2, the positions to be initialized marked on the node 2 are covered in the bitmaps of the last position and the last position after merging. That is, if node 3 fails, after domain reorganization, the merging result of merging bitmap information of a stripe performed by node 2 onto node 0 may be affected by the reorganization, resulting in loss of stripe information.
Based on this, there is provided a bitmap processing method embodiment according to an embodiment of the present invention, it should be noted that the steps shown in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different from that herein.
In this embodiment, a bitmap processing method is provided, which is applied to any node in a multi-node control system. Fig. 4 is a flowchart of a bitmap processing method according to an embodiment of the present invention, as shown in fig. 4, the flowchart includes the steps of:
step S401: and when an offline event is received, acquiring first domain attribution information of the strip.
In a multi-node control system, all memory space memories are visible to all nodes, a home mapping relation of each memory space basic unit stripe (stride) exists on each node, meanwhile, the mode that the nodes respectively manage home parts is carried out by dividing domains, so that a mapping relation of the nodes and domains exists, two nodes (master-slave nodes) form a domain, and a master node manages the write io of the stripe of the memory space divided by the domain.
Specifically, domain attribution information of the strip is obtained by acquiring a preset attribution mapping relation of the strip.
In an alternative embodiment, obtaining the first domain home information for the stripe includes: acquiring a preset band attribution mapping relation; and acquiring first domain attribution information in a band attribution mapping relation according to a preset band length.
As described above, in fig. 1, node 0 takes over domain of original node 3 after domain reorganization after offline event is received and domain reorganization is performed, so in an alternative embodiment, the first domain attribution information of the stripe is obtained after offline event is received.
Step S402: the first mask and the second mask are set according to the first domain home information.
In the embodiment of the invention, the bitmaps in the two nodes needing to be combined can be processed by setting the first mask and the second mask, so that the reconstruction in the two nodes needing to be combined is completed.
Step S403: the first local bitmap data is subjected to first preprocessing by using the first mask so that the first local bitmap data completes first reconstruction.
Step S404: and performing second preprocessing on the first received bitmap data by using the second mask so as to enable the first received bitmap data to complete second reconstruction.
Step S405: and combining the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment to obtain second local bitmap data.
As described above, since the execution progress of the reconstruction in the two nodes that need to be merged is not the same, there is a case where the stripe information is lost in the case where the execution area is marked and the bitmap space is multiplexed as another task proceeds when the task is repeatedly executed; therefore, the embodiment of the invention sets the first mask and the second mask according to the domain attribution information, respectively performs the first preprocessing on the first local bitmap data by using the first mask, and performs the second preprocessing on the first received bitmap data by using the second mask, thereby completing reconstruction execution in two nodes needing to be combined, and overcoming the defect of stripe information loss after combination.
It should be noted that the embodiment of the present invention may also be applied to more than two background tasks with interdependence, and by setting a specific 0/1 setting manner, the bitmap information of each background task may still be ensured to be accurate in merging results after a fault occurs, so as to save scheduling and space resources of each task.
In this embodiment, a bitmap processing method is provided, which may be used in a computer device. FIG. 5 is a flow chart of another bitmap processing method according to an embodiment of the present invention, as shown in FIG. 5, the flow includes the steps of:
step S501: and when an offline event is received, acquiring domain attribution information of the strip.
That is, after receiving the offline event, the domain attribution information of the number of the stripes on the minimum memory unit is queried in batches,
step S502: the first mask and the second mask are set according to the domain attribution information.
In an alternative embodiment, setting the first mask and the second mask according to the domain home information comprises the steps of:
step S5021: setting the bitmap of the stripes belonging to the domains of the node to 0 and setting the bitmap of the stripes not belonging to the domains of the node to 1, thereby obtaining a first mask.
Step S5022: and inverting the first mask by bits to obtain a second mask.
That is, by setting the mask owner_mask by distinguishing the domain attribution status, when setting the first mask owner_mask, the queried stripe position of the domain attributing to the node is 0, and the stripe position of the domain not attributing to the node is 1; and inverting the first mask owner_mask to obtain a second mask receiver_mask.
Step S503: the first local bitmap data is bitwise ored with a first mask.
That is, when the metadata is merged in a 32-bit unsigned integer type of structure unit, the local metadata bitmap is bitwise or operated with the first mask owner_mask.
Illustratively, in FIG. 6, the first row represents a partial data bitmap of node 0 and the second row represents a partial data bitmap of node 2. As shown in fig. 3, since the execution schedule of the reconfiguration in the node 0 is not the same as the execution schedule of the reconfiguration in the node 2, at this time, if the node 3 fails, after domain reorganization, the bitmap information of the stripe of which the execution of the node 2 is completed needs to be merged onto the node 0, so the data bitmap in the node 0 may be understood as the first local bitmap data, and the data bitmap in the node 2 may be understood as the first receiving bitmap data.
In fig. 6, the third line shows the result of bitwise or operation of the partial data bitmap of node 0 of the first line and the first mask.
Step S504: the first received bitmap data is bitwise ored with the second mask.
That is, when the metadata is merged in the unsigned integer unit of 32 bits, a bitwise or operation of the received metadata bitmap with the second mask receiver_mask is performed.
In fig. 6, the fourth line shows a result of performing a bitwise or operation on the partial data bitmap of node 2 of the second line and the second mask.
Step S505: and performing bit-wise AND operation on the first local bitmap data subjected to the first pretreatment and the first receiving bitmap data subjected to the second pretreatment.
In fig. 6, the fifth line indicates a result obtained by performing bitwise and operation on the bitmap of the third line and the bitmap of the fourth line.
As shown in fig. 6, in the embodiment of the present invention, by setting a first mask and a second mask according to domain attribution information, the first mask is used to perform first preprocessing on the first local bitmap data, and the second mask is used to perform second preprocessing on the first received bitmap data, so that reconstruction in two nodes that need to be combined can be performed, and the defect that the information of the combined stripe is lost is overcome.
The following explanation is given to the operation procedure for avoiding the merge conflict caused by the difference of the progress before and after the merge, which is generated in the metadata bitmap merge procedure:
(1) Bitmap for the owner section: according to the mechanism of the io and the background task partition execution, merging with the bitmap part of the non-owner of other nodes when merging; in this scenario, the bitmap of the marking situation that may interfere with the owner part on the other nodes is only affected when the position is 0 (i.e. the progress of another task affecting the bitmap of the present task is not executed to the current position, so the current position is the initial value of 0), and is not affected when it is 1 (a & 1=a; 0&A = 0;A is an arbitrary value (0/1)), so for this merging scenario, an or operation is executed on the non-owner part and 1, so that the bitmap of the owner part can be ensured to remain the same;
(2) Bitmap for non-owner part: the non-owner partial area is defined as an other area outside the area taken over by the domain taken over by the node and the partner node of the node, and the bitmap of the part is set to be 1 for the node, so as to avoid the situation that no strip is formatted after the attribution of the strip changes; the bitmap of the owner part as an offline node/accepted node must be merged with the non-owner part of the node, this part of the processing is the same as 1, and also ensures that the own partial bitmap of the accepted node remains intact;
as can be seen from fig. 6, the metadata merging result obtained by the embodiment of the present invention is the correct merging result, and the embodiment of the present invention ensures the accuracy of merging metadata after the failure of the composite task flow, for example, the execution of the mark initialization task in the reconstruction execution, in the merging process, thereby reducing the consumption of the space resources of the newly added bitmap and the scheduling resources of the newly added task under other policies.
Therefore, the embodiment of the invention realizes the accuracy of the bitmap information of the background tasks with mutual dependency relationship through the data operation logic on the basis of the bitmap structure, and does not need to consume resources in the task execution scheduling process to update and correct the execution condition according to the actual condition. The embodiment of the invention solves the problem of fault merging of bitmap information of background tasks with mutual dependency, but not the problem of fault merging of bitmaps of single background tasks and the problem of single merging of bitmaps of tasks without dependency.
In this embodiment, a bitmap processing method is provided, which may be used in a computer device. FIG. 7 is a flow chart of yet another bitmap processing method according to an embodiment of the present invention, in which not only offline events but also regression events are included.
As shown in fig. 1 and 8, since node 0 takes over domain of node 3 when node 3 fails, after node 3 regresses, node 0 needs to revert back domain of node 3, i.e., send data of the domain to node 3; and node 2 needs to backup its metadata information to node 3.
Fig. 7 is a flowchart of still another bitmap processing method according to an embodiment of the present invention, as shown in fig. 7, the flowchart includes the steps of:
step S701: a metadata header is obtained.
This is because the status of an event can be judged by the metadata header. Specifically, when the metadata heads are the same, the state of the event is offline over, that is, an offline event is received; when the metadata heads are different, the state of the event is regression back, namely, a regression event is received.
Step S702: judging whether the metadata heads are the same; when the metadata heads are the same, it is determined that an offline event is received, and the process proceeds to step S703; when the metadata heads are different, determining that a regression event is received, and proceeding to step S708;
Step S703: and acquiring first domain attribution information of the strip. This step is the same as step S501 described above and will not be described in detail here.
Step S704: the first mask and the second mask are set according to the first domain home information.
In an alternative set of embodiments, setting the first mask and the second mask according to domain home information includes:
step S7041: setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain a second mask;
step S7042: and inverting the second mask by bits to obtain the first mask.
That is, the first mask may be set first in step S504, and then the first mask is inverted to obtain the second mask; or the second mask can be set first, and then the second mask is inverted to obtain the first mask.
Step S705: the first local bitmap data is bitwise ored with a first mask. This step is the same as step S503 described above and will not be described in detail here.
Step S706: the first received bitmap data is bitwise ored with the second mask. This step is the same as step S504 described above and will not be described in detail here.
Step S707: and performing bit-wise AND operation on the first local bitmap data subjected to the first pretreatment and the first receiving bitmap data subjected to the second pretreatment. This step is the same as step S505 described above, and will not be described in detail here.
Step S708: whether bitmap data with preset length sent by the current merging node is received or not, and when bitmap data with preset length sent by the current merging node is received, step S709 is shifted to; otherwise, continuing to receive bitmap data with preset length sent by the current merging node.
Step S709: and acquiring progress information of each node combined with the regression node.
Step S710: and determining whether the current merging node is the node with the fastest progress according to the progress information of each node. When the current merging node is the node with the fastest progress, the step S711 is shifted to; when the current node is not the fastest node, the process proceeds to step S715.
When the node returns, the node which is about to become the own node after the reorganization and the node which becomes the part of the node send the bitmap information of the whole node to the returning node, and the sending logic is sequentially and partially sent.
In the process of judging whether the current merging node is the node merging the first time and the regression node, a mode of recording the progress of the bitmap sending by the two nodes is adopted, and the node with more progress can be understood to be the node merging the first time and the regression node.
Step S711: and obtaining the second domain attribution information of the strip.
In an alternative embodiment, obtaining the second domain home information for the stripe includes: acquiring a preset band attribution mapping relation; and acquiring second domain attribution information in a band attribution mapping relation according to the preset band length.
That is, domain attribution information of the number of stripes on the minimum memory unit is queried in batches.
Step S712: and setting a third mask according to the second domain attribution information.
In an alternative embodiment, setting the third mask according to the second domain home information includes: setting the bitmap of the stripes belonging to the domains of the node to 0 and setting the bitmap of the stripes not belonging to the domains of the node to 1, thereby obtaining a third mask.
In another alternative embodiment, setting the third mask according to the second domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1 and setting the bitmap of the stripes of domains not belonging to the node to 0, thereby obtaining a fourth mask; and inverting the fourth mask to obtain a third mask.
That is, when the fourth mask is set by setting the fourth mask_mask by distinguishing the domain attribution status, the position of the band of the domain attributed to the node is found to be 0, and the band position of the domain not attributed to the node is found to be 1; and inverting the third owner_mask to obtain a third mask receiver_mask.
Step S713: and performing third preprocessing on the second received bitmap data by using a third mask.
In an alternative embodiment, performing the third preprocessing of the second received bitmap data with the third mask includes: the second received bitmap data is bitwise ored with a third mask.
That is, when the metadata is merged in the unsigned integer unit of 32 bits, the bit-wise or operation of the received metadata bitmap with the third mask receiver_mask is performed.
Step S714: and taking the second received data after the third pretreatment as third local bitmap data of the regression node.
That is, the received bitmap information under the currently processed merge area is assigned to the local bitmap.
Step S715: and obtaining third domain attribution information of the strip.
In an alternative embodiment, obtaining third domain attribution information for a stripe comprises: acquiring a preset band attribution mapping relation; and acquiring third domain attribution information in a band attribution mapping relation according to a preset band length.
Step S716: the fifth mask and the sixth mask are set according to the third domain home information.
In an alternative embodiment, setting the fifth mask and the sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 0 and setting the bitmap of the stripes of domains not belonging to the node to 1 to obtain a fifth mask; the fifth mask is bit-wise inverted to obtain a sixth mask.
In another alternative embodiment, setting the fifth mask and the sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1 and setting the bitmap of the stripes of domains not belonging to the node to 0, thereby obtaining a sixth mask; the sixth mask is bit-wise inverted to obtain a fifth mask.
Step S717: fourth preprocessing is performed on the fourth local bitmap data using the fifth mask.
In an alternative embodiment, fourth preprocessing the fourth local bitmap data with the fifth mask includes: the fourth local bitmap data is bitwise ored with a fifth mask.
Step S718: fifth preprocessing is performed on the third reception bitmap data using the sixth mask.
In an alternative embodiment, fifth preprocessing the third received bitmap data with the sixth mask includes: the third received bitmap data is bitwise ored with a sixth mask.
Step S719: and combining the fourth local bitmap data subjected to the fourth pretreatment with the third receiving bitmap data subjected to the fifth pretreatment.
In an alternative embodiment, combining the fourth local bitmap data after the fourth preprocessing with the third received bitmap data after the fifth preprocessing includes: and performing bit-wise AND operation on the fourth local bitmap data subjected to the fourth pretreatment and the third receiving bitmap data subjected to the fifth pretreatment.
For example, as shown in fig. 9, node 1 and node 2 respectively give bitmap information of the entire amount of the present node to the regression node.
The number of bitmaps transmitted at a time is 100. When the regression node receives 0-100 bitmaps sent by one node, progress information of each node combined with the regression node is obtained, and according to the progress information of each node recorded in the record 1, the progress of the node 1 is larger than the progress of the node 2, so that the 0-100 bitmaps of the node 1 are combined with the regression node for the first time; when the regression node receives 0-100 bitmaps sent by another node, according to the progress information of each node recorded in the record 2, the progress of the node 1 leads/equals to the node 2, which indicates that the regression node has been combined with the node 1 for the first time, so that the regression node is combined with the node 2 for the second time.
When the regression node receives 100-200 bitmaps sent by one node, progress information of each node combined with the regression node is obtained, and according to the progress information of each node recorded in the record 3, the progress of the node 2 is larger than the progress of the node 1, so that the 100-200 bitmaps of the node 2 are combined with the regression node for the first time; when the regression node receives 100-200 bitmaps sent by another node, according to the progress information of each node recorded in the record 4, the progress of the node 2 leads/equals to the node 1, which indicates that the regression node has been combined with the node 2 for the first time, so that the regression node is combined with the node 1 for the second time.
The embodiment also provides a bitmap processing device, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
The present embodiment provides a bitmap processing apparatus, as shown in fig. 10, including:
A first obtaining module 1001, configured to obtain first domain attribution information of a stripe when an offline event is received;
a mask setting module 1002, configured to set a first mask and a second mask according to the first domain home information;
a local data bitmap preprocessing module 1003, configured to perform a first preprocessing on the first local bitmap data with a first mask, so that the first local bitmap data completes a first reconstruction;
a received bitmap data preprocessing module 1004, configured to perform a second preprocessing on the first received bitmap data using a second mask, so that the first received bitmap data completes a second reconstruction;
the merging module 1005 is configured to merge the first local bitmap data after the first preprocessing with the first received bitmap data after the second preprocessing to obtain second local bitmap data.
In an alternative embodiment, the bitmap processing apparatus further includes a second acquisition module and a metadata header processing module. The second acquisition module is used for acquiring the metadata head; the metadata head processing module is used for judging whether the metadata heads are the same or not; when the metadata headers are the same, it is determined that an offline event is received.
In an alternative embodiment, the metadata header processing module is further configured to determine that a regression event is received when the metadata headers are different.
In an optional implementation manner, the bitmap processing apparatus further includes a third obtaining module and a progress fastest node determining module. After receiving the regression event, after receiving bitmap data with preset length sent by the current merging node; the third acquisition module is used for acquiring the progress information of each node combined with the regression node; the progress fastest node determining module is used for determining whether the current merging node is the node with the fastest progress according to the progress information of each node; when the current merging node is the node with the fastest progress, the first obtaining module 1001 is further configured to obtain second domain attribution information of the stripe; mask setting module 1002 is also to: setting a third mask according to the second domain attribution information; the received bitmap data preprocessing module 1004 is further configured to perform third preprocessing on the second received bitmap data with a third mask; the merging module 1005 is further configured to use the third preprocessed second received data as third local bitmap data of the regression node.
In an alternative embodiment, the mask setting module 1002 is configured to set the bitmap of the stripe belonging to the domain of the node to 1, and set the bitmap of the stripe not belonging to the domain of the node to 0, to obtain the third mask.
In an alternative embodiment, the received bitmap data preprocessing module 1004 is further configured to bitwise or the second received bitmap data with the third mask.
In an alternative embodiment, when the current merge node is not the fastest node, the first obtaining module 1001 is configured to obtain third domain attribution information of the stripe; a mask setting module 1002, configured to set a fifth mask and a sixth mask according to the third domain home information; a local data bitmap preprocessing module 1003, configured to perform fourth preprocessing on fourth local bitmap data with a fifth mask; a received bitmap data preprocessing module 1004, configured to perform fifth preprocessing on the third received bitmap data with a sixth mask; a merging module 1005, configured to merge the fourth local bitmap data after the fourth preprocessing with the third received bitmap data after the fifth preprocessing.
In an alternative embodiment, mask setting module 1002 is specifically configured to: setting the bitmap of the stripes of domains belonging to the node to 0 and setting the bitmap of the stripes of domains not belonging to the node to 1 to obtain a first mask; bit inverting the first mask to obtain a second mask; and/or setting the bitmap of the stripes belonging to the domains of the node to 0 and setting the bitmap of the stripes not belonging to the domains of the node to 1, so as to obtain a fifth mask; the fifth mask is bit-wise inverted to obtain a sixth mask.
In an alternative embodiment, mask setting module 1002 is specifically configured to: setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain a second mask; bit inverting the second mask to obtain a first mask; and/or setting the bitmap of the stripes of domains belonging to the node to 1 and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain a sixth mask; the sixth mask is bit-wise inverted to obtain a fifth mask.
In an alternative embodiment, the local data bitmap preprocessing module 1003 is specifically configured to: performing bit OR operation on the first local bitmap data and the first mask; and/or, performing bit OR operation on the fourth local bit map data and the fifth mask.
In an alternative embodiment, the received bitmap data preprocessing module 1004 is specifically configured to: performing bit OR operation on the first received bitmap data and the second mask; and/or, performing bit OR operation on the third received bitmap data and the sixth mask.
In an alternative embodiment, the merging module 1005 is specifically configured to: performing bit-wise and operation on the first local bitmap data subjected to the first pretreatment and the first received bitmap data subjected to the second pretreatment; and/or performing bit-wise AND operation on the fourth local bitmap data subjected to the fourth pretreatment and the third receiving bitmap data subjected to the fifth pretreatment.
In an alternative embodiment, the first obtaining module 1001 is specifically configured to: acquiring a preset band attribution mapping relation; and acquiring first domain attribution information in a band attribution mapping relation according to a preset band length.
In an alternative embodiment, the bitmap processing apparatus further includes a reconstruction module, where the reconstruction module is specifically configured to, before the first preprocessing of the first local bitmap data with the first mask: a first reconstruction is performed on the first local bitmap data.
In an alternative embodiment, the reconstruction module is specifically configured to: for the owner stripe, marking/initializing whether to be reconstructed according to the logic of yes/no need; the non-owner stripes are all initialized.
The bitmap processing apparatus in this embodiment is presented in the form of functional units, where a unit refers to an ASIC circuit, a processor and a memory executing one or more software or fixed programs, and/or other devices that can provide the above-described functions.
Further functional descriptions of the above respective modules and units are the same as those of the above corresponding embodiments, and are not repeated here.
The embodiment of the invention also provides a computer device which is provided with the bitmap processing device shown in the figure 10.
Further, the embodiment of the invention also provides a multi-controller system, which comprises the computer equipment.
Referring to fig. 11, fig. 11 is a schematic structural diagram of a computer device according to an alternative embodiment of the present invention, as shown in fig. 11, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 11.
The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.
Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.
The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created from the use of the computer device of the presentation of a sort of applet landing page, and the like. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.
The computer device also includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor controller or programmable hardware includes a storage element that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the above embodiments.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims (21)

1. A bitmap processing method, applied to any node in a multi-node control system, the method comprising:
when an offline event is received, acquiring first domain attribution information of a strip;
setting a first mask and a second mask according to the first domain attribution information;
performing first preprocessing on the first local bitmap data by using the first mask so as to enable the first local bitmap data to complete first reconstruction;
performing second preprocessing on the first received bitmap data by using the second mask so as to enable the first received bitmap data to complete second reconstruction;
combining the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment to obtain second local bitmap data;
the first preprocessing of the first local bitmap data using the first mask includes: performing bit-wise OR operation on the first local bitmap data and the first mask;
The second preprocessing of the first received bitmap data using the second mask includes: and performing bit OR operation on the first receiving bitmap data and the second mask.
2. The method as recited in claim 1, further comprising:
acquiring a metadata header;
judging whether the metadata heads are the same;
and when the metadata heads are the same, determining that the offline event is received.
3. The method as recited in claim 2, further comprising:
and when the metadata heads are different, judging that a regression event is received.
4. A method according to claim 3, further comprising:
after receiving the regression event, after receiving bitmap data with preset length sent by the current merging node; acquiring progress information of each node combined with the regression node;
determining whether the current merging node is the node with the fastest progress according to the progress information of each node;
and when the current merging node is the node with the fastest progress, merging the current merging node and the regression node according to a preset first merging mode.
5. The method of claim 4, wherein merging the current merge node with the regression node in a preset first merge mode comprises:
Acquiring second domain attribution information of the strip;
setting a third mask according to the second domain attribution information;
performing third preprocessing on the second received bitmap data by using the third mask;
and taking the second received data after the third pretreatment as third local bitmap data of the regression node.
6. The method of claim 5, wherein the setting a third mask according to the second domain home information comprises:
setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain the third mask;
and/or;
setting the bitmap of the stripes of domains belonging to the node to 0 and setting the bitmap of the stripes of domains not belonging to the node to 1 to obtain a fourth mask; and inverting the fourth mask to obtain the third mask.
7. The method of claim 5, wherein the third preprocessing the second received bitmap data with the third mask comprises:
and performing bit OR operation on the second received bitmap data and the third mask.
8. The method as recited in claim 4, further comprising:
And when the current merging node is not the node with the fastest progress, merging the current merging node and the regression node according to a preset second merging mode.
9. The method of claim 8, wherein the merging the current merge node with the regression node in a second predetermined merge mode comprises:
acquiring third domain attribution information of the strip;
setting a fifth mask and a sixth mask according to the third domain attribution information;
performing fourth preprocessing on fourth local bitmap data by using the fifth mask;
performing fifth preprocessing on third received bitmap data by using the sixth mask;
and combining the fourth local bitmap data subjected to the fourth pretreatment with the third receiving bitmap data subjected to the fifth pretreatment.
10. The method according to claim 9, wherein:
the setting a first mask and a second mask according to the first domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 0, and setting the bitmap of the stripes of domains not belonging to the node to 1, so as to obtain the first mask; bit inverting the first mask to obtain the second mask;
And/or;
setting a fifth mask and a sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 0, and setting the bitmap of the stripes of domains not belonging to the node to 1, so as to obtain the fifth mask; and inverting the fifth mask by bits to obtain the sixth mask.
11. The method according to claim 9, wherein:
the setting a first mask and a second mask according to the first domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain the second mask; bit inverting the second mask to obtain the first mask;
and/or;
the setting of the fifth mask and the sixth mask according to the third domain home information includes: setting the bitmap of the stripes of domains belonging to the node to 1, and setting the bitmap of the stripes of domains not belonging to the node to 0, so as to obtain the sixth mask; and inverting the sixth mask by bits to obtain the fifth mask.
12. The method according to claim 9, wherein:
the fourth preprocessing of the fourth local bitmap data with the fifth mask includes: and performing bit OR operation on the fourth local bit map data and the fifth mask.
13. The method according to claim 9, wherein:
the fifth preprocessing of the third received bitmap data with the sixth mask includes: and performing bit OR operation on the third receiving bitmap data and the sixth mask.
14. The method according to claim 9, wherein:
the merging the first local bitmap data after the first preprocessing and the first receiving bitmap data after the second preprocessing comprises the following steps: performing bit-wise and operation on the first local bitmap data subjected to the first pretreatment and the first received bitmap data subjected to the second pretreatment;
and/or the number of the groups of groups,
the combining the fourth local bitmap data after the fourth preprocessing with the third received bitmap data after the fifth preprocessing includes: and performing bit-wise AND operation on the fourth local bitmap data subjected to the fourth pretreatment and the third receiving bitmap data subjected to the fifth pretreatment.
15. The method of claim 1, wherein the obtaining the first domain home information for the stripe comprises:
acquiring a preset band attribution mapping relation;
and acquiring the first domain attribution information from the band attribution mapping relation according to a preset band length.
16. The method of claim 1, further comprising, prior to first preprocessing the first local bitmap data with the first mask:
and carrying out first reconstruction on the first local bitmap data.
17. The method of claim 16, wherein the first reconstructing the first local bitmap data comprises:
for the owner stripe, marking/initializing whether to be reconstructed according to the logic of yes/no need;
the non-owner stripes are all initialized.
18. A bitmap processing apparatus applied to any node in a multi-node control system, the apparatus comprising:
the first acquisition module is used for acquiring first domain attribution information of the strip when receiving an offline event;
a mask setting module, configured to set a first mask and a second mask according to the first domain home information;
the local data bitmap preprocessing module is used for performing first preprocessing on first local bitmap data by utilizing the first mask so as to enable the first local bitmap data to complete first reconstruction;
a received bitmap data preprocessing module, configured to perform second preprocessing on first received bitmap data using the second mask, so that the first received bitmap data completes second reconstruction;
The merging module is used for merging the first local bitmap data subjected to the first pretreatment with the first receiving bitmap data subjected to the second pretreatment to obtain second local bitmap data;
the local data bitmap preprocessing module is specifically configured to: performing bit-wise OR operation on the first local bitmap data and the first mask;
the receiving bitmap data preprocessing module is specifically configured to: and performing bit OR operation on the first receiving bitmap data and the second mask.
19. A computer device, comprising:
a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the bitmap processing method of any one of claims 1 to 17.
20. A multi-controller system comprising the computer device of claim 19.
21. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the bitmap processing method of any one of claims 1 to 17.
CN202311526544.7A 2023-11-16 2023-11-16 Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium Active CN117251119B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311526544.7A CN117251119B (en) 2023-11-16 2023-11-16 Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311526544.7A CN117251119B (en) 2023-11-16 2023-11-16 Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium

Publications (2)

Publication Number Publication Date
CN117251119A CN117251119A (en) 2023-12-19
CN117251119B true CN117251119B (en) 2024-02-13

Family

ID=89137231

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311526544.7A Active CN117251119B (en) 2023-11-16 2023-11-16 Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium

Country Status (1)

Country Link
CN (1) CN117251119B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7065212B1 (en) * 2000-10-27 2006-06-20 Matsushita Electric Industrial Co., Ltd. Data hiding in communication
CN115543696A (en) * 2022-11-30 2022-12-30 苏州浪潮智能科技有限公司 Disk array reconstruction task acceleration method, system, storage medium and equipment

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9146685B2 (en) * 2013-02-11 2015-09-29 International Business Machines Corporation Marking local regions and providing a snapshot thereof for asynchronous mirroring
US10181208B2 (en) * 2016-02-10 2019-01-15 Microsoft Technology Licensing, Llc Custom heatmaps
US10162563B2 (en) * 2016-12-02 2018-12-25 International Business Machines Corporation Asynchronous local and remote generation of consistent point-in-time snap copies

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7065212B1 (en) * 2000-10-27 2006-06-20 Matsushita Electric Industrial Co., Ltd. Data hiding in communication
CN115543696A (en) * 2022-11-30 2022-12-30 苏州浪潮智能科技有限公司 Disk array reconstruction task acceleration method, system, storage medium and equipment

Also Published As

Publication number Publication date
CN117251119A (en) 2023-12-19

Similar Documents

Publication Publication Date Title
US20230333930A1 (en) Distributed Erasure Coded Virtual File System
CN104052576B (en) Data recovery method based on error correcting codes in cloud storage
US7849223B2 (en) Virtually synchronous Paxos
US9367598B2 (en) Merging an out of synchronization indicator and a change recording indicator in response to a failure in consistency group formation
CN109491609B (en) Cache data processing method, device and equipment and readable storage medium
US5737763A (en) Incremental disk backup
US9996434B2 (en) Data mirror volume verification
US7761431B2 (en) Consolidating session information for a cluster of sessions in a coupled session environment
CN111046024B (en) Data processing method, device, equipment and medium for shared storage database
CN107451013B (en) Data recovery method, device and system based on distributed system
CN108932249B (en) Method and device for managing file system
CN112506710B (en) Distributed file system data restoration method, device, equipment and storage medium
CN115268784A (en) RAID (redundant array of independent disks) verification write protection method and system based on four-control storage system
CN112000513A (en) Computer and VPD data operation method, device and storage medium thereof
CN117251119B (en) Bitmap processing method, bitmap processing device, computer equipment, control system and storage medium
WO2019109257A1 (en) Log management method, server and database system
CN110990197B (en) Optimization method of application-level multi-layer check point based on supercomputer
CN109542665A (en) Power loss recovery method, method for writing data and device
CN113312205A (en) Data verification method and device, storage medium and computer equipment
CN108628701B (en) Cache data protection method and device
CN109344012A (en) Data reconstruction control method, device and equipment
US11269550B2 (en) Storage system and history information management method
CN110460478B (en) Alarm test method, alarm test system, alarm manufacturing equipment and network management system
CN114518845A (en) Data storage method, device, medium and equipment
JP2020181533A (en) Data collection system and data collection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant