CN102681892A - Key-Value type write-once read-many lock pool software module and running method thereof - Google Patents
Key-Value type write-once read-many lock pool software module and running method thereof Download PDFInfo
- Publication number
- CN102681892A CN102681892A CN2012101507211A CN201210150721A CN102681892A CN 102681892 A CN102681892 A CN 102681892A CN 2012101507211 A CN2012101507211 A CN 2012101507211A CN 201210150721 A CN201210150721 A CN 201210150721A CN 102681892 A CN102681892 A CN 102681892A
- Authority
- CN
- China
- Prior art keywords
- lock
- worm
- logic
- read
- key
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a Key-Value type write-once read-many lock pool software module and a running method thereof. The module comprises an application program calling interface, a write-once read-many operation logical layer, a Hash managing module, a dynamic memory pool module and a system physical lock resource which are coupled with one another and work in a linked way; and a plenty of write-once read-many logical locks are provided for developing other application programs. The running method comprises the following steps of: calling an application program calling interface by taking any data as a Key through an application program; making a lock access request including adding write lock, unlocking write lock, adding read lock, unlocking read lock and inquiring a load read-write state; connecting the application program calling interface to the access request; and transferring to a write-once read-many operation logical layer to realize service. According to the module and the method, a large quantity of write-once read-many lock resources can be provided for an application layer to use by using a small quantity of operating system resources, an efficient ''small lock'' application model is realized, and the high-performance development requirement in a multi-task environment in the software industry is met.
Description
Technical field
The present invention relates to lock pond software module and method in the parallel computation field, be specifically related to a kind of Key-Value type WORM lock pond software module and operation method thereof.
Background technology
Lock is a kind of valuable source that multiple task operating system provides; Common have various ways such as critical section, semaphore, mutex; Its Core Feature is in the parallel Programming field; When two or more threads face when visiting some resources (internal memory, Socket, IO etc.) possible simultaneously, this resource of provisional declaration is being taken (Lock) by certain thread sometime, prevents the conflict property visit of other threads; To guarantee that each visit action all is able to intactly accomplish, and avoids program bug in the authority that oneself needs.
At the large server development field; Owing to will tackle the high speed of a large amount of clients, concurrent greatly, mass data visit; The visit of resource such as internal data module configured, communication module and change very dense; Need the lock resource of efficient and magnanimity to support, could guarantee the integrality and the security of data access, and the efficient that guarantees resource access.
Traditional latching operation program design mode generally is divided into two kinds:
1, definition one is " big lock "; Protect a complete data structure; Such as all data in a lock protection one tree type data structure, formation, the storehouse, all visits to this data structure all need realize that this takies seldom operating-system resources through this lock; But the lock collision is too intensive in the time of can causing big concurrent visit, and access efficiency is very low.
2, definition is much " little lock "; Subdata that segment, very among a small circle in the protected data structure; Only protect a slice " leaf " in the tree form data structure such as a lock; Lock disposes according to 1:1 quantity with data resource in the program, and this can reduce the lock collision and promote access efficiency.But because lock itself also is the resource of operating system, in the tissue of mass data, system resources consumption can be very big, can cause system to be collapsed because of inadequate resource, or program is compelled that the upper limit is set, and can't protect too many resource.
Above-mentioned two kinds of schemes are all imperfect, or performance is low, or serious waste of resources, and the two is difficult to compromise.At present the software industry lacks that a kind of high efficiency, low system resource takies, to the lock salvo of mass data.This has seriously restricted the performance and the scale of high concurrent mass data application server.
In addition, the lock resource that legacy operating system provides, its locking method is " rough " relatively, and the two kinds of modes of operation of (Lock) and release (Unlock) that only lock are not segmented the concrete data access behavior of applications.This causes locking at every turn all is unconditional " exclusive ", can't be concurrent, and performance is not high.
But in real work, data access has " reading ", " writing " two kinds of demands usually, and research shows; Under " writing " mode, need the lock protection of " exclusive " really, prevent bug; And under " reading " mode,, can allow multiple " reading " concurrent because protected data resource is not modified; Promptly allow a plurality of threads to read certain resource simultaneously, to promote the concurrent efficient of program.
This needs WORM lock logic, i.e. the application layer latching operation behavior of " segmentation ", clearly difference " writing action " and " reading to move ", in addition different processing, the lock protective capability of realization " unique writing " and " concurrent reading ".
At present, most of multiple task operating systems do not provide " WORM lock " resource, and the software industry does not have unified standard yet, need the programmer in development, to realize voluntarily.But because WORM lock logic development difficulty is bigger, industry programmer's level is uneven again, often goes wrong in doing in fact, has influenced the program development work efficiency.
The software industry relatively lacks comparatively general at present, has application development interface API higher abstractness, that the close friend is provided, can lock the resource solution by WORM cross-platform, that stride language call.
Summary of the invention
In order to overcome the shortcoming that above-mentioned prior art exists; The object of the present invention is to provide a kind of Key-Value type WORM lock pond software module and operation method thereof; Can use operating-system resources (physical locks resource, internal memory etc.) seldom; Provide magnanimity " WORM logic lock " resource provisioning to use,, satisfy the high-performance development requirement under multitask environment of software industry to realize " little lock " application model efficiently with layer.
In order to achieve the above object, the technical scheme that the present invention adopted is:
A kind of Key-Value type WORM lock pond software module comprises intercoupling the following submodule of linkage work:
Application call interface: define the method for operating of this module, supply other computer programming languages to call, realize function output;
WORM operation logic layer: the WORM algorithm logic is provided, for each logic lock of realizing provides the WORM operating characteristic;
The Hash administration module: accept the Key key assignments of arbitrary data types, quick indexing is searched corresponding WORM logic lock, realizes the accurate visit of a plurality of WORM logic locks;
Dynamic memory pool module: realize the dynamic application and the release of inner WORM logic lock data;
System physical lock resource: to the lock resource of operating system application, it adds unlock method this modular program run time call, and the multithreading safeguard protection of this module self function is provided.
Said lock pond software module provides magnanimity WORM logic lock to supply other application developments to use; Said WORM logic lock comprises two characteristics, writes zone bit and a read counter, can store, describe the operation state values of " unique writing " and " multiple reading " for one; Its core data belongs to internal storage data; Can also can be applied for dynamically and release that its logical definition is following by index management in the Hash administration module by dynamic memory pool module:
1) writing flag set is 1, and expression this logic lock at present is in " writing " state, forbids that other " writing " or " reading " action gets into, and promptly can't be provided with once more and write zone bit is 1, and read counter perhaps superposes;
2) read counter stack is not 0, and expression has " reading " action at present, does not allow other " writing " action to get into, and promptly can't be provided with and write zone bit is 1, but other " reading " action can be concurrent, and this value that is embodied in read counter constantly increases.
The operation method of a kind of Key-Value type WORM lock pond software module comprises the steps:
Step 1: application program with arbitrary data as Key; Call this software module application call interface; The lock access request is proposed; Described request comprises adding to be write lock, separates and write lock, read lock, separate read lock and inquiry lock read-write state, and the application call interface is received access request, forwards WORM operation logic layer to and realizes service;
Step 2: WORM operation logic layer method of service is following:
Move for " add and write lock " or " read lock "; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; If do not exist; Then to WORM logic lock of dynamic memory pool module application, push the Hash administration module then and carry out index management, this moment, this WORM logic lock was regarded as existing; If exist, then directly add latching operation, return result;
For " separate and write lock " or " separating read lock " action, whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in, and does not exist directly and returns; Do not do anything; Exist and then carry out unlocking operation, check after release finishes whether it also has other locking states, if do not have; The index that then this WORM logic of deletion is locked from the Hash administration module, and to this WORM logic lock resource of dynamic memory pool module release;
For reading the lock state action; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; Do not exist and directly return the lock unused state, existence is then read the interior real data mode of WORM logic lock and is returned;
Above-mentioned everything is all accomplished same the lock system physical under the protection, is not interrupted during with the operation of assurance everything, guarantees the multithreading security of this software module self.
Described operation method comprises following application call interface and relevant WORM logical layer program circuit:
Write lock status poll interface: the lock state of writing of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read and wherein write sign, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Read lock status poll interface: the read counter state of value of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read wherein read counter, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Trial adds writes lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create WORM logic lock, be set to vacation if create the failure rreturn value, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having set up, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added read lock or write lock, and lock if added, rreturn value is set to vacation if find or create success
F) if can lock, then write sign and be set to 1 and realize locking, rreturn value is set to very simultaneously
G) separate this modular system physical locks
H) inspection rreturn value, if be false, expression locks not successful, function directly returns vacation, finishes
I) if rreturn value is true, expression is write lock and is added success, then unlimited endless loop need be set, and constantly calls read lock status poll interface, detects whether all read locks are all removed, and is 0 up to obtaining read counter, and the side jumps out circulation, returns true;
Confirm to add and write lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, circular wait, till successfully locking, the specific procedure flow process is following:
A) unlimited endless loop is set, constantly calls and attempt adding writing lock interface
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Lock interface is write in releasing: remove for the WORM logic lock of Key appointment and write lock, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, setting is write and is masked as 0, destroys WORM logic lock, should lock index from the deletion of Hash administration module simultaneously, discharges resource
E) separate this modular system physical locks, return;
Attempt the read lock interface: for the WORM logic lock read lock of Key appointment, write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create, if create the failure rreturn value vacation is set, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having created, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added to write lock that if lock, rreturn value is set to vacation if find or create success
F) if can lock, then realized locking in read counter+1, rreturn value is set to very simultaneously
G) separate this modular system physical locks, return rreturn value;
Confirm the read lock interface: for the WORM logic lock read lock of Key appointment,, this WORM logic lock writes lock if having been added, circular wait, till successfully adding read lock, the specific procedure flow process is following:
A) unlimited endless loop is set, constantly calls the trial read lock
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Remove the read lock interface: remove read lock for the WORM logic lock of Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, read counter-1, if read counter at this moment=0, and destroy WORM logic lock, should lock index from the deletion of Hash administration module simultaneously, discharge resource
E) separate this modular system physical locks, return.
The application call interface of said lock pond software module supplies all to have the computer programming language routine call of the API of call operation system ability, comprises C, C++, compilation, Java, C#, VB.
Said lock pond software module provides the version of different operating system, realizes the spanning operation system platform function, can under Windows, Linux and some other type of Unix platform, work, and module product can be organized as following form, realizes function output:
.h header file defines (application development interface declaration definition) (Windows, Linux, Unix like)
.dll dynamic link library (Windows)
.so dynamic link library (Linux, Unix like)
.obj Object linking file (Windows, Linux, Unix like)
.lib static link library (Windows, Linux, Unix like)
The present invention compares with prior art, has following advantage:
The common operating system of its function ratio of WORM logic lock of the present invention's definition is locked powerful; Be characterized in segmenting outside action type,, realized " unique writing " principle of complete exclusiveness for " writing " operation to the data resource; Be equivalent to the Lock that locks of operating system lock this moment; And, allow " concurrent reading " conditionally for " reading " operation, then can improve the concurrent work efficiency of multi-thread software system.
The dynamic memory management principle that cooperates the C/C++ language again; The present invention defines WORM logic lock to be had only and just really is instantiated as effective internal storage data in use; In time discharge back Installed System Memory when not using; Then can use memory cost seldom, realize that the WORM logic lock programs of magnanimity is used.
Description of drawings
Fig. 1 is for writing lock status poll process flow diagram.
Fig. 2 is a read lock status poll process flow diagram.
Fig. 3 writes the lock process flow diagram for attempting adding.
Fig. 4 writes the lock process flow diagram for confirming to add.
Fig. 5 writes the lock process flow diagram for removing.
Fig. 6 is for attempting the read lock process flow diagram.
Fig. 7 is for confirming the read lock process flow diagram.
Fig. 8 is for removing the read lock process flow diagram.
Embodiment
Below in conjunction with embodiment and accompanying drawing the present invention is done further explain.
A kind of Key-Value type of the present invention WORM lock pond software module comprises intercoupling the following submodule of linkage work:
Application call interface: define the method for operating of this module, supply other computer programming languages to call, realize function output;
WORM operation logic layer: the WORM algorithm logic is provided, for each logic lock of realizing provides the WORM operating characteristic;
The Hash administration module: accept the Key key assignments of arbitrary data types, quick indexing is searched corresponding WORM logic lock, realizes the accurate visit of a plurality of WORM logic locks;
Dynamic memory pool module: realize the dynamic application and the release of inner WORM logic lock data;
System physical lock module: to the lock resource of operating system application, call it in the program and add unlock method, the multithreading safeguard protection of this module self function is provided.
Said lock pond software module provides magnanimity WORM logic lock to supply other application developments to use; Said WORM logic lock comprises two characteristics, writes zone bit and a read counter, can store, describe the operation state values of " unique writing " and " multiple reading " for one; Its core data belongs to internal storage data; Can also can be applied for dynamically and release that its logical definition is following by index management in the Hash administration module by dynamic memory pool module:
1) writing flag set is 1, and expression this logic lock at present is in " writing " state, forbids that other " writing " or " reading " action gets into, and promptly can't be provided with once more and write zone bit is 1, and read counter perhaps superposes;
2) read counter stack is not 0, and expression has " reading " action at present, does not allow other " writing " action to get into, and promptly can't be provided with and write zone bit is 1, but other " reading " action can be concurrent, and this value that is embodied in read counter constantly increases;
The operation method of Key-Value type WORM lock of the present invention pond software module comprises the steps:
Step 1: application program with arbitrary data as Key; Call this software module application call interface; The lock access request is proposed; Described request comprises adding to be write lock, separates and write lock, read lock, separate read lock and inquiry lock read-write state, and the application call interface is received access request, forwards WORM operation logic layer to and realizes service;
Step 2: WORM operation logic layer method of service is following:
Move for " add and write lock " or " read lock "; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; If do not exist; Then to WORM logic lock of dynamic memory pool module application, push the Hash administration module then and carry out index management, this moment, this WORM logic lock was regarded as existing; If exist, then directly add latching operation, return result;
For " separate and write lock " or " separating read lock " action, whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in, and does not exist directly and returns; Do not do anything; Exist and then carry out unlocking operation, check after release finishes whether it also has other locking states, if do not have; The index that then this WORM logic of deletion is locked from the Hash administration module, and to this WORM logic lock resource of dynamic memory pool module release;
For reading the lock state action; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; Do not exist and directly return the lock unused state, existence is then read the interior real data mode of WORM logic lock and is returned;
Above-mentioned everything is all accomplished same the lock system physical under the protection, is not interrupted during with the operation of assurance everything, guarantees the multithreading security of this software module self.
Described operation method comprises following application call interface and relevant WORM logical layer program circuit:
Write lock status poll interface: the lock state of writing of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is as shown in Figure 1:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read and wherein write sign, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Read lock status poll interface: the read counter state of value of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is as shown in Figure 2:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read wherein read counter, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Trial adds writes lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is as shown in Figure 3:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create WORM logic lock, be set to vacation if create the failure rreturn value, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having set up, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added read lock or write lock, and lock if added, rreturn value is set to vacation if find or create success
F) if can lock, then write sign and be set to 1 and realize locking, rreturn value is set to very simultaneously
G) separate this modular system physical locks
H) inspection rreturn value, if be false, expression locks not successful, function returns vacation, directly finishes
I) if rreturn value is true, expression is write lock and is added success, then unlimited endless loop need be set, and constantly calls read lock status poll interface, detects whether all read locks are all removed, and is 0 up to obtaining read counter, and the side jumps out circulation, returns true;
Confirm to add and write lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, circular wait, till successfully locking, the specific procedure flow process is as shown in Figure 4:
A) unlimited endless loop is set, constantly calls and attempt adding writing lock interface Try2EnableWrite
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Lock interface is write in releasing: remove for the WORM logic lock of Key appointment and write lock, the specific procedure flow process is as shown in Figure 5:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, being provided with and writing zone bit is 0, destroys WORM logic lock, is somebody's turn to do the lock index from the deletion of Hash administration module simultaneously, discharges resource
E) separate this modular system physical locks, return;
Attempt the read lock interface: for the WORM logic lock read lock of Key appointment, write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is as shown in Figure 6:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create, if create the failure rreturn value vacation is set, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having created, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added to write lock that if lock, rreturn value is set to vacation if find or create success
F) if can lock, then realized locking in read counter+1, rreturn value is set to very simultaneously
G) separate this modular system physical locks, return rreturn value;
Confirm the read lock interface: for the WORM logic lock read lock of Key appointment,, this WORM logic lock writes lock if having been added, circular wait, till successfully adding read lock, the specific procedure flow process is as shown in Figure 7:
A) unlimited endless loop is set, constantly calls and attempt read lock Try2AddRead
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Remove the read lock interface: remove read lock for the WORM logic lock of Key appointment, the specific procedure flow process is as shown in Figure 8:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, read counter-1, if read counter at this moment=0, and destroy WORM logic lock, should lock index from the deletion of Hash administration module simultaneously, discharge resource
E) separate this modular system physical locks, return.
The application call interface of said lock pond software module supplies all to have the computer programming language routine call of the API of call operation system ability, comprises C, C++, compilation, Java, C#, VB.
Said lock pond software module provides the version of different operating system, realizes the spanning operation system platform function, can under Windows, Linux and some other type of Unix platform, work, and module product can be organized as following form, realizes function output:
.h header file defines (application development interface declaration definition) (Windows, Linux, Unix like)
.dll dynamic link library (Windows)
.so dynamic link library (Linux, Unix like)
.obj Object linking file (Windows, Linux, Unix like)
.lib static link library (Windows, Linux, Unix like)
Development procedure of the present invention:
The present invention utilizes the structure of C language, defines a WORM logic lock, preserves WORM logic relevant information.
Utilize the compiling macro definition of C language, the access function of the different lock of abstract different operating system platform resource provides the standard calls method, realizes cross-platform transfer ability.
Exploitation C language function provides the WORM logic relevant standard access function: add and separate " writing " and lock and add and separate " reading " lock, status poll function etc.All functions all can be directed against above-mentioned WORM logic latching operation.
Exploitation dynamic memory management module, i.e. memory pool, this step can omit; Directly use malloc, the free function of C language; Perhaps the new of C Plus Plus, delete function realize that the memory pool that the present invention adopts the inventor to design voluntarily only is to reuse internal memory in order to reclaim; Avoid memory fragmentation, satisfy server 7*24 hour long-play demand.This is not a right emphasis of the present invention, repeats no more.
Exploitation Hash administration module, this step also can omit, and directly uses the Map class of C Plus Plus to realize that the Hash pond module that the present invention adopts the inventor to design voluntarily is in order to keep bigger flexibility ratio and high efficiency.This is not a right emphasis of the present invention, repeats no more.
Aforementioned WORM logic is locked the Value that indicator index is defined as the Hash administration module, and each sheet " leaf " that makes the Hash administration module all is the index of a WORM logic lock.Key does not then limit, and application layer can use arbitrary data as Key, in the Hash pond, looks for the WORM logic lock of correspondence, realizes operation.
Arbitrary data can be one section binary data, a C language structure, a character string; An integer, or other data are planned by application layer procedure person; As long as guarantee uniqueness, promptly Key can only corresponding one locks logic, and two different Key must corresponding different logical lock.
System physical is locked in design one, and with the interlock of above-mentioned logic lock, the operation of retrieval, WORM lock body of guaranteeing the Hash pond is all through this system physical lock completion, and making all lock pond operations itself is multithreading safety.
Design standards application call interface, the visit behavior of standard software module of the present invention provides the unified interface of calling, and supplies the different application layer to use.
Pay close attention to professional platform independence in the programming procedure; Do not allow to call the distinctive function of each operating system; The power function that only allows use standard C/C Plus Plus to provide passes through different compilers again, is compiled as various binary code modules such as obj, lib, dll, so; Allow application layer cross-platform, stride the language compiling link and call, realize function output.
Application example:
The present invention Rython in real time/the historical data base plateform system in, used a plurality of instances.
In application-specific instance, need do the lock protection, therein with the data resource secure access of guaranteeing height and giving to different data paragraph in the thousands of data files.Through calculating, the quantity size of lock demand is 400,000,000 locks.Undoubtedly, traditional " little lock " model will cause serious system resource bottleneck under so big scale, can not realize, and the performance requirement of system does not allow to use inefficient " big lock " model.
Process is to application of the present invention, and the logic lock that system has obtained up to 400,000,000 supplies to use, and support the concurrent reading and writing visit above 150,000 times/second simultaneously, and the actual system resource that takies seldom, realized the demand.
Claims (6)
1. a Key-Value type WORM is locked the pond software module, it is characterized in that: comprise intercoupling the following submodule of linkage work:
Application call interface: define the method for operating of this module, supply other computer programming languages to call, realize function output;
WORM operation logic layer: the WORM algorithm logic is provided, for each logic lock of realizing provides the WORM operating characteristic;
The Hash administration module: accept the Key key assignments of arbitrary data types, quick indexing is searched corresponding WORM logic lock, realizes the accurate visit of a plurality of WORM logic locks;
Dynamic memory pool module: realize the dynamic application and the release of inner WORM logic lock data;
System physical lock module: to the lock resource of operating system application, call it in the program and add unlock method, the multithreading safeguard protection of this module self function is provided.
2. according to the said lock of claim 1 pond software module; It is characterized in that: provide magnanimity WORM logic lock to supply other application developments to use; Said WORM logic lock comprises two characteristics, writes zone bit and a read counter, can store, describe the operation state values of " unique writing " and " multiple reading " for one; Its core data belongs to internal storage data; Can also can be applied for dynamically and release that its logical definition is following by index management in the Hash administration module by dynamic memory pool module:
1) writing flag set is 1, and expression this logic lock at present is in " writing " state, forbids that other " writing " or " reading " action gets into, and promptly can't be provided with once more and write zone bit is 1, and read counter perhaps superposes;
2) read counter stack is not 0, and expression has " reading " action at present, does not allow other " writing " action to get into, and promptly can't be provided with and write zone bit is 1, but other " reading " action can be concurrent, and this value that is embodied in read counter constantly increases.
3. the operation method of the said lock of claim 1 pond software module is characterized in that: comprise the steps:
Step 1: application program with arbitrary data as Key; Call this software module application call interface; The lock access request is proposed; Described request comprises adding to be write lock, separates and write lock, read lock, separate read lock and the lock state is read in inquiry, and the application call interface is received access request, forwards WORM operation logic layer to and realizes service;
Step 2: WORM operation logic layer method of service is following:
Move for " add and write lock " or " read lock "; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; If do not exist; Then to WORM logic lock of dynamic memory pool module application, push the Hash administration module then and carry out index management, this moment, this WORM logic lock was regarded as existing; If exist, then directly add latching operation, return result;
For " separate and write lock " or " separating read lock " action, whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in, and does not exist directly and returns; Do not do anything; Exist and then carry out unlocking operation, check after release finishes whether it also has other locking states, if do not have; The index that then this WORM logic of deletion is locked from the Hash administration module, and to this WORM logic lock resource of dynamic memory pool module release;
For reading the lock state action; Whether WORM operation logic layer is at first retrieved in the Hash administration module according to Key has corresponding WORM logic to be latched in; Do not exist and directly return the lock unused state, existence is then read the interior real data mode of WORM logic lock and is returned;
Above-mentioned everything is all accomplished same the lock system physical under the protection, is not interrupted during with the operation of assurance everything, guarantees the multithreading security of this software module self.
4. operation method according to claim 3 is characterized in that: comprise following application call interface and relevant WORM logical program flow process:
Write lock status poll interface: the lock state of writing of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read and wherein write sign, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Read lock status poll interface: the read counter state of value of the WORM logic lock of inquiry Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if find, read wherein read counter, assignment is given rreturn value
D) if do not find, rreturn value directly is set to 0, expression does not lock
E) separate this modular system physical locks, return rreturn value;
Trial adds writes lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create WORM logic lock, be set to vacation if create the failure rreturn value, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having set up, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added read lock or write lock, and lock if added, rreturn value is set to vacation if find or create success
F) if can lock, then write sign and be set to 1 and realize locking, rreturn value is set to very simultaneously
G) separate this modular system physical locks
H) inspection rreturn value, if be false, expression locks not successful, function returns vacation, directly finishes.
I) if rreturn value is true, expression is write lock and is added success, then unlimited endless loop need be set, and constantly calls read lock status poll interface, detects whether all read locks are all removed, and is 0 up to obtaining read counter, and the side jumps out circulation, returns true;
Confirm to add and write lock interface: lock to add for the WORM logic of Key appointment and write lock, read or write lock if this WORM logic lock has added, circular wait, till successfully locking, the specific procedure flow process is following:
A) unlimited endless loop is set, constantly calls and attempt adding writing lock interface
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Lock interface is write in releasing: remove for the WORM logic lock of Key appointment and write lock, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, setting is write and is masked as 0, destroys this WORM logic lock, should lock index from the deletion of Hash administration module, the release resource
E) separate this modular system physical locks, return;
Attempt the read lock interface: for the WORM logic lock read lock of Key appointment, write lock if this WORM logic lock has added, vacation is returned in failure, or returns after successfully locking very, and the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C) if do not find, attempt to create, if create the failure rreturn value vacation is set, jump to g)
D) if create successfully, this WORM logic lock index and Key are made up the Key-Value pairing, be pushed to the Hash administration module and manage, if Hash pushes failure, then destroy the WORM logic lock of just having created, rreturn value is set to vacation, jumps to g)
E), check whether this lock has added to write lock that if lock, rreturn value is set to vacation if find or create success
F) if can lock, then realized locking in read counter+1, rreturn value is set to very simultaneously
G) separate this modular system physical locks, return rreturn value;
Confirm the read lock interface: for the WORM logic lock read lock of Key appointment,, this WORM logic lock writes lock if having been added, circular wait, till successfully adding read lock, the specific procedure flow process is following:
A) unlimited endless loop is set, constantly calls the trial read lock
B) then circulation continuation of failure
C) circulation is then jumped out in success, returns true;
Remove the read lock interface: remove read lock for the WORM logic lock of Key appointment, the specific procedure flow process is following:
A) add this modular system physical locks
B) carry out Hash according to Key and search, seek corresponding WORM logic lock
C), do not do anything if do not find
D) if find, read counter-1, if read counter at this moment=0, and destroy this WORM logic lock, should lock index from the deletion of Hash administration module, the release resource
E) separate this modular system physical locks, return.
5. the application call interface of the said lock of claim 1 pond software module supplies all to have the computer programming language routine call of the API of call operation system ability, comprises C, C++, compilation, Java, C#, VB.
6. the said lock of claim 1 pond software module provides the version of different operating system; Realize the spanning operation system platform function; Can under Windows, Linux and some other type of Unix platform, work, module product can be organized as following form, realizes function output:
.h header file defines (application development interface declaration definition) (Windows, Linux, Unix like)
.dll dynamic link library (Windows)
.so dynamic link library (Linux, Unix like)
.obj Object linking file (Windows, Linux, Unix like)
.lib static link library (Windows, Linux, Unix like).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210150721.1A CN102681892B (en) | 2012-05-15 | 2012-05-15 | Key-Value type write-once read-many lock pool software module and running method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210150721.1A CN102681892B (en) | 2012-05-15 | 2012-05-15 | Key-Value type write-once read-many lock pool software module and running method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102681892A true CN102681892A (en) | 2012-09-19 |
| CN102681892B CN102681892B (en) | 2014-08-20 |
Family
ID=46813861
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210150721.1A Expired - Fee Related CN102681892B (en) | 2012-05-15 | 2012-05-15 | Key-Value type write-once read-many lock pool software module and running method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102681892B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572568A (en) * | 2013-10-15 | 2015-04-29 | 阿里巴巴集团控股有限公司 | Read lock operation method, write lock operation method and system |
| CN105243101A (en) * | 2015-09-17 | 2016-01-13 | 浪潮(北京)电子信息产业有限公司 | Method and system for multiple controllers to perform concurrent access to same data compressed file |
| CN105511969A (en) * | 2015-11-25 | 2016-04-20 | 中国船舶工业系统工程研究院 | Inter-process thread mutex method |
| CN108140009A (en) * | 2015-10-13 | 2018-06-08 | 微软技术许可有限责任公司 | B-tree key assignments manager of the distributed freedom formula based on RDMA |
| CN108762940A (en) * | 2018-04-12 | 2018-11-06 | 武汉斗鱼网络科技有限公司 | Multi-threaded Access Methods and device |
| CN112631742A (en) * | 2020-12-30 | 2021-04-09 | 上海金卓科技有限公司 | Resource access authority management device, method and system |
| CN112929278A (en) * | 2021-02-24 | 2021-06-08 | 深圳市吉祥腾达科技有限公司 | Method and system for multi-core parallel fast forwarding of data packets by network communication equipment |
| CN113220458A (en) * | 2021-05-26 | 2021-08-06 | 西安热工研究院有限公司 | Real-time scheduling method and device for realizing load balance of database connection pool |
| US11556395B2 (en) * | 2020-01-24 | 2023-01-17 | Microsoft Technology Licensing, Llc | Data race detection with per-thread memory protection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5860070A (en) * | 1996-05-31 | 1999-01-12 | Oracle Corporation | Method and apparatus of enforcing uniqueness of a key value for a row in a data table |
| CN1545033A (en) * | 2003-11-14 | 2004-11-10 | 清华大学 | LUN CACHE method for FC-SAN memory system |
| CN1808389A (en) * | 2006-02-20 | 2006-07-26 | 南京联创科技股份有限公司 | Autonomous locking method based on shared memory for account background memory database |
| CN101252603A (en) * | 2008-04-11 | 2008-08-27 | 清华大学 | Cluster distributed type lock management method based on storage area network SAN |
-
2012
- 2012-05-15 CN CN201210150721.1A patent/CN102681892B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5860070A (en) * | 1996-05-31 | 1999-01-12 | Oracle Corporation | Method and apparatus of enforcing uniqueness of a key value for a row in a data table |
| CN1545033A (en) * | 2003-11-14 | 2004-11-10 | 清华大学 | LUN CACHE method for FC-SAN memory system |
| CN1808389A (en) * | 2006-02-20 | 2006-07-26 | 南京联创科技股份有限公司 | Autonomous locking method based on shared memory for account background memory database |
| CN101252603A (en) * | 2008-04-11 | 2008-08-27 | 清华大学 | Cluster distributed type lock management method based on storage area network SAN |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572568A (en) * | 2013-10-15 | 2015-04-29 | 阿里巴巴集团控股有限公司 | Read lock operation method, write lock operation method and system |
| CN104572568B (en) * | 2013-10-15 | 2021-07-23 | 蚂蚁金服（杭州）网络技术有限公司 | Read lock operation method, write lock operation method and system |
| CN105243101A (en) * | 2015-09-17 | 2016-01-13 | 浪潮(北京)电子信息产业有限公司 | Method and system for multiple controllers to perform concurrent access to same data compressed file |
| CN105243101B (en) * | 2015-09-17 | 2019-05-10 | 浪潮(北京)电子信息产业有限公司 | The method and system that multi-controller concurrently accesses same compressed data file |
| CN108140009B (en) * | 2015-10-13 | 2022-06-24 | 微软技术许可有限责任公司 | Distributed autonomous RDMA-based B-tree key value manager |
| CN108140009A (en) * | 2015-10-13 | 2018-06-08 | 微软技术许可有限责任公司 | B-tree key assignments manager of the distributed freedom formula based on RDMA |
| CN105511969B (en) * | 2015-11-25 | 2020-05-19 | 中国船舶工业系统工程研究院 | Method for mutual exclusion between cross-process threads |
| CN105511969A (en) * | 2015-11-25 | 2016-04-20 | 中国船舶工业系统工程研究院 | Inter-process thread mutex method |
| CN108762940B (en) * | 2018-04-12 | 2020-09-04 | 武汉斗鱼网络科技有限公司 | Multithreading access method and device |
| CN108762940A (en) * | 2018-04-12 | 2018-11-06 | 武汉斗鱼网络科技有限公司 | Multi-threaded Access Methods and device |
| US11556395B2 (en) * | 2020-01-24 | 2023-01-17 | Microsoft Technology Licensing, Llc | Data race detection with per-thread memory protection |
| CN112631742A (en) * | 2020-12-30 | 2021-04-09 | 上海金卓科技有限公司 | Resource access authority management device, method and system |
| CN112631742B (en) * | 2020-12-30 | 2023-10-31 | 上海金卓科技有限公司 | Resource access authority management device, method and system |
| CN112929278A (en) * | 2021-02-24 | 2021-06-08 | 深圳市吉祥腾达科技有限公司 | Method and system for multi-core parallel fast forwarding of data packets by network communication equipment |
| CN113220458A (en) * | 2021-05-26 | 2021-08-06 | 西安热工研究院有限公司 | Real-time scheduling method and device for realizing load balance of database connection pool |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102681892B (en) | 2014-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102681892B (en) | Key-Value type write-once read-many lock pool software module and running method thereof | |
| US9935975B2 (en) | Hard object: lightweight hardware enforcement of encapsulation, unforgeability, and transactionality | |
| US6421690B1 (en) | Computer memory management system | |
| CN102880518B (en) | Fair scalable reader-writer mutual exclusion | |
| Alistarh et al. | Threadscan: Automatic and scalable memory reclamation | |
| US6826757B2 (en) | Lock-free implementation of concurrent shared object with dynamic node allocation and distinguishing pointer value | |
| Hoeflinger | Extending OpenMP to clusters | |
| Atkinson et al. | Data types and persistence | |
| WO2000000906A2 (en) | Method and apparatus for locking | |
| Brown et al. | Pragmatic primitives for non-blocking data structures | |
| US11698893B2 (en) | System and method for use of lock-less techniques with a multidimensional database | |
| Cypher | The communication requirements of mutual exclusion | |
| Gordon et al. | Static lock capabilities for deadlock freedom | |
| Prokopec | Cache-tries: concurrent lock-free hash tries with constant-time operations | |
| CN107103253B (en) | Disk sensitive information scanning system based on MPI AC (alternating Current) serial matching parallel algorithm | |
| Blanusa et al. | Scalable fine-grained parallel cycle enumeration algorithms | |
| CN102047222A (en) | Blocking and bounding wrapper for thread-safe data collections | |
| CN112000670B (en) | Multithreading program data unified management method and system and electronic equipment | |
| US20060184759A1 (en) | Permanent pool memory management method and system | |
| Blelloch et al. | Concurrent reference counting and resource management in wait-free constant time | |
| Nguyen et al. | Understanding and combating memory bloat in managed data-intensive systems | |
| Sheffi et al. | The era theorem for safe memory reclamation | |
| Wang et al. | DHash: dynamic hash tables with non-blocking regular operations | |
| Adamanskiy et al. | EJDB-Embedded JSON database engine | |
| Rivera et al. | Heapless: Dynamic data structures without dynamic heap allocator for rust |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: XI'AN TPRI POWER STATION INFORMATION TECHNOLOGY CO Free format text: FORMER OWNER: XI'AN THERMAL POWER RESEARCH INSTITUTE CO., LTD. Effective date: 20150423 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 710032 XI'AN, SHAANXI PROVINCE TO: 710075 XI'AN, SHAANXI PROVINCE |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20150423 Address after: 710075 Xi'an City, Shaanxi province high tech Zone Torch Hotel, block B, 11 Patentee after: XI'AN TPRI POWER STATION INFORMATION TECHNOLOGY CO., LTD. Address before: 710032 Xingqing Road, Shaanxi, China, No. 136, No. Patentee before: Xi'an Thermal Power Research Institute Co., Ltd. |
|
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20140820 Termination date: 20180515 |