CN116204124B - Data processing method and system based on conflict lock and electronic equipment - Google Patents
Data processing method and system based on conflict lock and electronic equipment Download PDFInfo
- Publication number
- CN116204124B CN116204124B CN202310159056.0A CN202310159056A CN116204124B CN 116204124 B CN116204124 B CN 116204124B CN 202310159056 A CN202310159056 A CN 202310159056A CN 116204124 B CN116204124 B CN 116204124B
- Authority
- CN
- China
- Prior art keywords
- write request
- range
- data block
- write
- lock
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title abstract description 20
- 238000000034 method Methods 0.000 claims abstract description 56
- 238000012545 processing Methods 0.000 claims abstract description 51
- 230000015654 memory Effects 0.000 claims abstract description 25
- 238000004590 computer program Methods 0.000 claims abstract description 6
- 230000004044 response Effects 0.000 claims description 16
- 230000000903 blocking effect Effects 0.000 abstract description 16
- 238000004891 communication Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000013500 data storage Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000002829 reductive effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000000414 obstructive effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The specification discloses a data processing method, a system and electronic equipment based on conflict lock, which can simplify blocking processing logic, reduce program complexity and improve processing efficiency. The method comprises the following steps: after the data volume generates a snapshot, a target data block in the data volume receives a write request; determining whether the write request requires performing a redirect-on-write operation; if the redirection operation during writing needs to be executed, setting a first range lock for the writing request, and setting a locking range of the first range lock as the target data block; and if the redirection operation on writing is not required to be executed, setting a second range lock for the writing request. The system comprises a request receiving module, a redirection judging module and a range lock setting module. The electronic device comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the data processing method when executing the program.
Description
Technical Field
The present disclosure relates to the field of data storage technologies, and in particular, to a data processing method, system and electronic device based on conflict locks.
Background
In the computer field, using locks for conflict handling is a common approach for concurrent requests. In computer data storage technology, two kinds of blocking conditions are often considered to perform conflict processing, one kind of conflict lock needs to be set for a read-Write request range conflict in a data volume, and the other kind of conflict lock needs to be set for a conflict between a Redirect-On-Write (ROW) operation and a subsequent Write request after a snapshot is generated On the data volume. In the data storage technology, two different types of blocking conditions are considered to set conflict locks, so that the computer processing program is complex in logic, high in program complexity and low in processing efficiency.
Disclosure of Invention
In view of this, the embodiments of the present disclosure provide a data processing method, system and electronic device based on conflict lock, which can simplify the blocking processing logic, reduce the complexity of the program, and improve the processing efficiency.
In a first aspect, embodiments of the present disclosure provide a method for processing data based on a conflicting lock. The method is applied to a data volume and comprises the following steps:
after the data volume generates a snapshot, a target data block in the data volume receives a write request;
determining whether the write request requires performing a redirect-on-write operation;
responding to the write request to execute a redirect-on-write operation, setting a first range lock for the write request, and setting a locking range of the first range lock as the target data block;
and setting a second range lock for the write request in response to the write request without performing a redirect-on-write operation, wherein the locking range of the second range lock is determined according to the corresponding data range of the write request in the target data block.
Optionally, determining whether the write request requires performing a redirect-on-write operation includes:
determining whether the write request is a first write request received by the target data block after snapshot generation;
responding to the first write request received by the target data block after the write request is generated by the snapshot, and comparing the data range corresponding to the write request with the whole data range of the target data block;
and determining that the write request needs to execute a redirect-on-write operation in response to the data range corresponding to the write request being a part of the overall data range of the target data block.
Optionally, after setting the first range lock for the write request, the method further includes: and adding a redirection mark for the target data block.
Optionally, before determining whether the write request requires performing a redirect-on-write operation, the method further includes:
determining whether the target data block has the redirection flag;
in response to the presence of the redirect-on-write flag, it is determined that the write request does not require performing a redirect-on-write operation.
Optionally, after the data volume generates the snapshot again, the method clears the redirection flag corresponding to the target data block.
Optionally, after setting the first range lock for the write request, the method further includes:
determining a data block corresponding to the write request in a snapshot volume corresponding to the data volume;
and determining data to be written in the write request, merging the data to be written with the data in the data block corresponding to the write request, and then writing the merged data into the target data block.
Optionally, after merging the data to be written with the data in the data block corresponding to the write request, the method writes the data into the target data block, the method further includes: releasing the first range lock.
Optionally, after setting the second range lock for the write request, the method further includes:
determining, in the target data block, whether there is a previous request being executed;
in response to there being a prior request being executed, determining whether a range conflict exists between the second range lock and a range lock corresponding to the prior request;
and in response to the existence of the range conflict, adding the write request to a to-be-executed list, and executing the write request after the range lock corresponding to the previous request is released.
In a second aspect, embodiments of the present specification also provide a conflict lock-based data processing system, the system being applied to a data volume, comprising:
the request receiving module is used for controlling a target data block in the data volume to receive a write request after the data volume generates the snapshot;
a redirection determination module configured to determine whether the write request requires performing a redirect-on-write operation;
the range lock setting module is used for setting a first range lock for the write request when the write request needs to execute a redirect-on-write operation, and setting the locking range of the first range lock as the target data block;
the range lock setting module is further configured to set a second range lock for the write request when the write request does not need to perform a redirect-on-write operation, where a locking range of the second range lock is determined according to a data range corresponding to the write request in the target data block.
In a third aspect, embodiments of the present specification also provide a data processing electronic device based on a conflict lock, including a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the data processing method based on a conflict lock as described in the first aspect when executing the program.
From the above, it can be seen that the data processing method, system and electronic device based on conflict lock provided in the embodiments of the present disclosure have the following beneficial technical effects:
in the conflict lock-based data processing method, it is first determined whether a request for the write request received by the target data block requires performing a redirect-on-write (ROW) operation. For the write request needing to execute the ROW operation, a first range lock is set for the write request, and the locking range of the first range lock is set to be the whole data range of the target data block, so that the range lock of other write requests conflicts with the first range lock, and the blocking processing between the write request needing to execute the ROW operation and other write requests can be realized. In the data processing method, only one mechanism of range conflict lock is utilized in the mode, and the processing of two different blocking conditions of read-write request range conflict and redirection and write request conflict during writing is realized at the same time, so that the blocking processing logic is greatly simplified, the program complexity is reduced, and the processing efficiency is improved.
Drawings
The features and advantages of the present invention will be more clearly understood by reference to the accompanying drawings, which are illustrative and should not be construed as limiting the invention in any way, in which:
FIG. 1 illustrates a schematic diagram of a method of conflict lock-based data processing provided in one or more alternative embodiments of the present disclosure;
FIG. 2 illustrates a method diagram of determining whether a write request requires performing a redirect-on-write operation in a data processing method based on a conflicting lock provided in one or more alternative embodiments of the present disclosure;
FIG. 3 illustrates a method diagram of processing the write request in a data processing method based on a conflict lock provided in one or more alternative embodiments of the present disclosure;
FIG. 4 depicts a schematic diagram of a data processing system based on a conflict lock provided by one or more alternative embodiments of the present specification;
FIG. 5 illustrates a schematic diagram of a data processing electronic device based on a conflict lock provided in one or more alternative embodiments of the present specification.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
In the computer field, using locks for conflict handling is a common approach for concurrent requests. In computer data storage technology, it is often necessary to consider two obstructive conditions for conflict handling.
One is to set a conflict lock for read-write request range conflicts in a data volume.
In the technical field of data storage, in order to prevent data disorder caused by simultaneous operation of multiple concurrent read-write requests on a data volume by a same block area, a range lock is used to lock the read-write requests, so that blocking processing is realized. That is, if the data range of the subsequent read-write request overlaps with the area before the write request to the area is not processed, the requests are blocked in the range lock, and the subsequent conflicting read-write requests are executed only after the previous write request is processed and the range lock is released.
Another is that after a snapshot is generated On a data volume, a conflict lock needs to be set for performing a Redirect-On-Write (ROW) operation to conflict with a subsequent Write request.
In the case of snapshots generated for a data volume using the incremental snapshot mode of redirection-on-write, there is also a blocking between the first write request received after the snapshot was generated by the data volume and the subsequent other write requests.
When the first received write request needs to execute the ROW operation and the write range of the subsequent other write requests also falls in the data block corresponding to the first write request under the condition that the data volume just completes the snapshot, the subsequent write request can be executed after the first write request is completed even if the subsequent write request does not have the write range conflict with the first write request. In this case, a special conflict lock needs to be set for the first write request.
In the data storage technology, two different types of blocking conditions are considered to set conflict locks, so that the computer processing program is complex in logic, high in program complexity and low in processing efficiency.
Aiming at the problems, the technical scheme of the embodiment of the specification aims to provide a data processing method, which adopts the same type of conflict lock setting for different types of blocking conditions, ensures that the blocking processing logic can be simplified, the program complexity can be reduced, and the processing efficiency can be improved while the processing of different blocking conditions is realized.
In view of the above, in one aspect, embodiments of the present disclosure provide a method for processing data based on a conflict lock.
As shown in fig. 1, one or more alternative embodiments of the present disclosure provide a data processing method based on a conflict lock, applied to a data volume, where the method includes:
s1: after the data volume generates the snapshot, a write request is received by a target data block in the data volume.
The data volume may include a plurality of data blocks, and any of the data modules in the data volume may receive a write request as the target data block. The write request may be directed to some or all of the data range in the target data block.
For example, the size of the target data block is 512 bytes, which may be represented as a range of 0 to 512, and a range of 10 to 20 into which data to be written in the write request may be written.
S2: a determination is made as to whether the write request requires performing a redirect-on-write operation.
After receiving the Write request, it is necessary to first determine whether the Write request requires performing a Redirect-On-Write (ROW) operation.
As shown in FIG. 2, a method diagram of determining whether the write request requires performing a redirect-on-write operation is provided. As a specific embodiment, the method for determining whether the write request needs to perform a redirect-on-write operation may include the following implementation steps:
s201: determining whether the write request is a first write request received by the target data block after snapshot generation.
S202: and in response to the write request being a first write request received by the target data block after snapshot generation, comparing the data range corresponding to the write request with the whole data range of the target data block to determine whether the data range of the write request is a part of the whole data range of the target data block.
S203: and determining that the write request needs to execute a redirect-on-write operation in response to the data range corresponding to the write request being a part of the overall data range of the target data block.
For a write request received by the target data block after the snapshot is generated by the data volume, when the corresponding data range is only a part of the total data range of the target data block, it may be determined that the write request needs to perform a ROW operation.
S3: and responding to the write request to execute a redirect-on-write operation, setting a first range lock for the write request, and setting the locking range of the first range lock as the target data block.
And when the target data block processes the write request, receiving other write requests pointing to the target data block, wherein whether the data range corresponding to the other write requests coincides with the data range of the write request needing to execute the ROW operation or not, the locking range of the range lock corresponding to the other write requests conflicts with the locking range of the first range lock. Therefore, in the parallel processing process, the target data block firstly processes the write request for performing the ROW operation, and after the write request processing is completed, other write requests are processed.
S4: and setting a second range lock for the write request in response to the write request without performing a redirect-on-write operation, wherein the locking range of the second range lock is determined according to the corresponding data range of the write request in the target data block.
In the conflict lock-based data processing method, it is first determined whether a request for the write request received by the target data block requires performing a redirect-on-write (ROW) operation. For the write request needing to execute the ROW operation, a first range lock is set for the write request, and the locking range of the first range lock is set to be the whole data range of the target data block, so that the range lock of other write requests conflicts with the first range lock, and the blocking processing between the write request needing to execute the ROW operation and other write requests can be realized. In the data processing method, only one mechanism of range conflict lock is utilized in the mode, and the processing of two different blocking conditions of read-write request range conflict and redirection and write request conflict during writing is realized at the same time, so that the blocking processing logic is greatly simplified, the program complexity is reduced, and the processing efficiency is improved.
When it is determined that the write request needs to perform the redirect-on-write operation, no other write requests subsequently received by the target data block need to perform the redirect-on-write operation, and no further determination may be made on the other write requests.
Based on this, in the data processing method provided in one or more optional embodiments of the present specification, a redirection flag is further added to the target data block after the first range lock is set for the write request. Before determining whether the write request requires performing a redirect-on-write operation, it is first determined whether the target data block has the redirect flag.
If the target data block has the redirection mark, which indicates that it has been determined that a certain write request needs to perform redirection when writing, it may be directly determined that the redirection operation when writing is not needed by the write request that is recently received by the target data block, and a second range lock may be set for the write request.
According to the data processing method based on the conflict lock, after the fact that a certain write request needs to execute the ROW operation is determined for the first time, the mark is added to the target data block, the fact that the subsequent received write request does not need to execute the ROW operation can be directly determined, and therefore workload can be greatly reduced, and processing efficiency is improved.
Further, in some optional embodiments, after the data volume generates the snapshot again, the redirection flag corresponding to the target data block is cleared. The method can timely clear the redirection mark, can ensure the accuracy of judgment of the writing request after the snapshot is regenerated, and avoids the mark from interfering the judgment result.
One or more optional embodiments of the present disclosure provide a method for processing data based on a conflict lock, which further includes processing the write request after setting a first range lock for the write request. As a specific embodiment, the write request may be processed using the following scheme: determining a data block corresponding to the write request in a snapshot volume corresponding to the data volume; and determining data to be written in the write request, merging the data to be written with the data in the data block corresponding to the write request, and then writing the merged data into the target data block. Further, after the data to be written is merged with the data in the data block corresponding to the write request and then written into the target data block, the first range lock is released, so that the orderly processing of other subsequent read-write requests is facilitated.
One or more optional embodiments of the present disclosure provide a method for processing data based on a conflict lock, which further includes processing the write request after setting a second range lock for the write request. As shown in fig. 3, as a specific embodiment, the write request may be processed using the following scheme:
s301: in the target data block, it is determined whether there is a previous request being executed.
S302: in response to there being a prior request being executed, a determination is made as to whether a range conflict exists between the second range lock and a range lock corresponding to the prior request.
Wherein the prior request being performed may be a write request that requires a ROW operation to be performed. For a previous request requiring a ROW operation, the range lock is the first range lock, and the corresponding locking range is the full range of the target data block. In this case, the second range lock collides with the first range lock at a range.
S303: and in response to the existence of the range conflict, adding the write request to a to-be-executed list, and executing the write request after the range lock corresponding to the previous request is released.
It should be noted that the methods of one or more embodiments of the present description may be performed by a single device, such as a computer or server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of one or more embodiments of the present description, the devices interacting with each other to accomplish the methods.
It should be noted that the foregoing describes specific embodiments of the present invention. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Based on the same inventive concept, the embodiments of the present disclosure also provide a data processing system based on a conflict lock, corresponding to the method of any of the embodiments described above.
With reference to FIG. 4, the conflict lock-based data processing system is applied to data volumes. The system comprises:
the request receiving module is used for controlling a target data block in the data volume to receive a write request after the data volume generates the snapshot;
a redirection determination module configured to determine whether the write request requires performing a redirect-on-write operation;
the range lock setting module is used for setting a first range lock for the write request when the write request needs to execute a redirect-on-write operation, and setting the locking range of the first range lock as the target data block;
the range lock setting module is further configured to set a second range lock for the write request when the write request does not need to perform a redirect-on-write operation, where a locking range of the second range lock is determined according to a data range corresponding to the write request in the target data block.
In one or more embodiments of the present disclosure, the redirection determination module is further configured to determine whether the write request is a first write request received by the target data block after snapshot generation; when the write request is the first write request received by the target data block after snapshot generation, comparing the data range corresponding to the write request with the whole data range of the target data block; and when the data range corresponding to the write request is a part of the whole data range of the target data block, determining that the write request needs to execute a redirect-on-write operation.
One or more alternative embodiments of the present specification provide a conflict lock-based data processing system that further includes a tagging module. The marking module is configured to add a redirection mark to the target data block after the range lock setting module sets a first range lock for the write request.
In a data processing system based on a conflict lock provided in one or more alternative embodiments of the present specification, the redirection determination module is further configured to determine whether the target data block has the redirection flag before determining whether the write request requires performing a redirect-on-write operation; when the redirect-on-write flag is present, it is determined that the write request does not require performing a redirect-on-write operation.
In one or more alternative embodiments of the present disclosure, the marking module is further configured to clear the redirection mark corresponding to the target data block after the data volume generates the snapshot again.
One or more alternative embodiments of the present disclosure provide a data processing system based on a conflict lock, further comprising a write request processing module. The write request processing module is configured to determine, after the range lock setting module sets a first range lock for the write request, a data block corresponding to the write request in a snapshot volume corresponding to a data volume; and determining data to be written in the write request, merging the data to be written with the data in the data block corresponding to the write request, and then writing the merged data into the target data block.
In a data processing system based on a conflict lock provided in one or more alternative embodiments of the present disclosure, the write request processing module is further configured to release the first range lock after merging the data to be written with the data in the data block corresponding to the write request and then writing the merged data into the target data block.
In a data processing system based on a conflict lock provided in one or more alternative embodiments of the present disclosure, the write request processing module is further configured to determine, after the range lock setting module sets a second range lock for the write request, whether there is a previous request that is being executed in the target data block; determining if a range conflict exists between the second range lock and a range lock corresponding to the prior request in the case that the prior request is being executed; and when the range conflict exists, adding the write request to a to-be-executed list, and executing the write request after the range lock corresponding to the previous request is released.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in one or more pieces of software and/or hardware when implementing one or more embodiments of the present description.
The device of the foregoing embodiment is configured to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Fig. 5 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), microprocessor, application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.
The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The electronic device of the foregoing embodiment is configured to implement the corresponding method in the foregoing embodiment, and has the beneficial effects of the corresponding method embodiment, which is not described herein.
Based on the same inventive concept, the present disclosure also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the data processing method according to any of the embodiments above, corresponding to the method of any of the embodiments above.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The storage medium of the foregoing embodiments stores computer instructions for causing the computer to perform the data processing method according to any of the foregoing embodiments, and has the advantages of the corresponding method embodiments, which are not described herein.
It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiment method may be implemented by a computer program to instruct related hardware, where the program may be stored in a computer readable storage medium, and the program may include the above-described embodiment method when executed. Wherein the storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a random access Memory (RandomAccessMemory, RAM), a Flash Memory (Flash Memory), a Hard Disk (HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present application.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the disclosure, including the claims, is limited to these examples; combinations of features of the above embodiments or in different embodiments are also possible within the spirit of the present disclosure, steps may be implemented in any order, and there are many other variations of the different aspects of one or more embodiments described above which are not provided in detail for the sake of brevity.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The present disclosure is intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Any omissions, modifications, equivalents, improvements, and the like, which are within the spirit and principles of the one or more embodiments of the disclosure, are therefore intended to be included within the scope of the disclosure.
Claims (9)
1. A method of conflict lock-based data processing, for application to a data volume, the method comprising:
after the data volume generates a snapshot, a target data block in the data volume receives a write request;
determining whether the write request requires performing a redirect-on-write operation; the determining whether the write request requires performing a redirect-on-write operation includes: determining whether the write request is a first write request received by the target data block after snapshot generation; responding to the first write request received by the target data block after the write request is generated by the snapshot, and comparing the data range corresponding to the write request with the whole data range of the target data block; determining that the write request needs to execute a redirect-on-write operation in response to the data range corresponding to the write request being a portion of the overall data range of the target data block; the data range is the operation range of the write request to the target data block;
responding to the write request to execute a redirect-on-write operation, setting a first range lock for the write request, and setting a locking range of the first range lock as the target data block;
and setting a second range lock for the write request in response to the write request without performing a redirect-on-write operation, wherein the locking range of the second range lock is determined according to the corresponding data range of the write request in the target data block.
2. The method of claim 1, further comprising, after setting a first range lock for the write request: and adding a redirection mark for the target data block.
3. The method of claim 2, further comprising, prior to determining whether the write request requires performing a redirect-on-write operation:
determining whether the target data block has the redirection flag;
in response to the presence of the redirect-on-write flag, it is determined that the write request does not require performing a redirect-on-write operation.
4. The method of claim 2, wherein the redirection flag corresponding to the target data block is cleared after the data volume regenerates a snapshot.
5. The method of claim 1, further comprising, after setting a first range lock for the write request:
determining a data block corresponding to the write request in a snapshot volume corresponding to the data volume;
and determining data to be written in the write request, merging the data to be written with the data in the data block corresponding to the write request, and then writing the merged data into the target data block.
6. The method of claim 5, further comprising, after merging the data in the data block corresponding to the write request with the data to be written, writing the data to the target data block: releasing the first range lock.
7. The method of claim 1, further comprising, after setting a second range lock for the write request:
determining, in the target data block, whether there is a previous request being executed;
in response to there being a prior request being executed, determining whether a range conflict exists between the second range lock and a range lock corresponding to the prior request;
and in response to the existence of the range conflict, adding the write request to a to-be-executed list, and executing the write request after the range lock corresponding to the previous request is released.
8. A conflict lock-based data processing system for application to a data volume, the system comprising:
the request receiving module is used for controlling a target data block in the data volume to receive a write request after the data volume generates the snapshot;
a redirection determination module configured to determine whether the write request requires performing a redirect-on-write operation; the determining whether the write request requires performing a redirect-on-write operation includes: determining whether the write request is a first write request received by the target data block after snapshot generation; responding to the first write request received by the target data block after the write request is generated by the snapshot, and comparing the data range corresponding to the write request with the whole data range of the target data block; determining that the write request needs to execute a redirect-on-write operation in response to the data range corresponding to the write request being a portion of the overall data range of the target data block;
the range lock setting module is used for setting a first range lock for the write request when the write request needs to execute a redirect-on-write operation, and setting the locking range of the first range lock as the target data block; the data range is the operation range of the write request to the target data block;
the range lock setting module is further configured to set a second range lock for the write request when the write request does not need to perform a redirect-on-write operation, where a locking range of the second range lock is determined according to a data range corresponding to the write request in the target data block.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 7 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310159056.0A CN116204124B (en) | 2023-02-23 | 2023-02-23 | Data processing method and system based on conflict lock and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310159056.0A CN116204124B (en) | 2023-02-23 | 2023-02-23 | Data processing method and system based on conflict lock and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116204124A CN116204124A (en) | 2023-06-02 |
| CN116204124B true CN116204124B (en) | 2024-03-22 |
Family
ID=86509035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310159056.0A Active CN116204124B (en) | 2023-02-23 | 2023-02-23 | Data processing method and system based on conflict lock and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116204124B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430314A (en) * | 2015-12-01 | 2016-03-23 | 浙江宇视科技有限公司 | Video data storage and access control method and device |
| CN108460045A (en) * | 2017-02-21 | 2018-08-28 | 中兴通讯股份有限公司 | A kind of processing method and distributed block storage system of snapshot |
| CN115470040A (en) * | 2022-08-26 | 2022-12-13 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for testing re-deleted fingerprint threshold based on snapshot |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10831398B2 (en) * | 2016-09-19 | 2020-11-10 | International Business Machines Corporation | Storage device efficiency during data replication |
| US11307935B2 (en) * | 2020-03-31 | 2022-04-19 | EMC IP Holding Company LLC | Management of volume snapshots in a data storage system |
| US11567704B2 (en) * | 2021-04-29 | 2023-01-31 | EMC IP Holding Company LLC | Method and systems for storing data in a storage pool using memory semantics with applications interacting with emulated block devices |
-
2023
- 2023-02-23 CN CN202310159056.0A patent/CN116204124B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105430314A (en) * | 2015-12-01 | 2016-03-23 | 浙江宇视科技有限公司 | Video data storage and access control method and device |
| CN108460045A (en) * | 2017-02-21 | 2018-08-28 | 中兴通讯股份有限公司 | A kind of processing method and distributed block storage system of snapshot |
| CN115470040A (en) * | 2022-08-26 | 2022-12-13 | 苏州浪潮智能科技有限公司 | Method, device, equipment and medium for testing re-deleted fingerprint threshold based on snapshot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116204124A (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101266580B1 (en) | Indexed register access for memory device | |
| CN104424030B (en) | Method and device for sharing memory by multi-process operation | |
| CN110781016B (en) | Data processing method, device, equipment and medium | |
| KR102545689B1 (en) | Computing system with buffer and method of operation thereof | |
| CN112395093A (en) | Multithreading data processing method and device, electronic equipment and readable storage medium | |
| CN114529417A (en) | Method for executing transaction, block chain, main node and slave node | |
| CN104461730A (en) | Virtual resource allocation method and device | |
| US20090198849A1 (en) | Memory Lock Mechanism for a Multiprocessor System | |
| US20120059997A1 (en) | Apparatus and method for detecting data race | |
| CN112925606A (en) | Memory management method, device and equipment | |
| US8726101B2 (en) | Apparatus and method for tracing memory access information | |
| US20090198695A1 (en) | Method and Apparatus for Supporting Distributed Computing Within a Multiprocessor System | |
| CN111913807B (en) | Event processing method, system and device based on multiple storage areas | |
| CN116204124B (en) | Data processing method and system based on conflict lock and electronic equipment | |
| CN113127438B (en) | Method, apparatus, server and medium for storing data | |
| CN115905040B (en) | Counter processing method, graphics processor, device and storage medium | |
| EP3249540B1 (en) | Method for writing multiple copies into storage device, and storage device | |
| US20090198916A1 (en) | Method and Apparatus for Supporting Low-Overhead Memory Locks Within a Multiprocessor System | |
| US11663012B1 (en) | Systems and methods for detecting coroutines | |
| CN115269199A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN114679465A (en) | Resource operation method and device, electronic equipment and storage medium | |
| CN103235745A (en) | Address conflict detection method and device | |
| US10416923B1 (en) | Fast backup solution for cluster shared volumes shared across a cluster of nodes using extent sets as parallel save streams | |
| CN111209062A (en) | Information acquisition method and device, electronic equipment and computer storage medium | |
| CN107645541B (en) | Data storage method and device and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |