CN110134439A - The method of method for constructing data structure and write-in data, reading data without lockization - Google Patents
The method of method for constructing data structure and write-in data, reading data without lockization Download PDFInfo
- Publication number
- CN110134439A CN110134439A CN201910253786.0A CN201910253786A CN110134439A CN 110134439 A CN110134439 A CN 110134439A CN 201910253786 A CN201910253786 A CN 201910253786A CN 110134439 A CN110134439 A CN 110134439A
- Authority
- CN
- China
- Prior art keywords
- queue
- data
- vernier
- serial number
- producer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 91
- 238000004519 manufacturing process Methods 0.000 claims abstract description 57
- 238000012545 processing Methods 0.000 abstract description 19
- 238000010276 construction Methods 0.000 abstract description 3
- 238000012163 sequencing technique Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 18
- 238000003860 storage Methods 0.000 description 12
- 230000007246 mechanism Effects 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000005304 joining Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000033764 rhythmic process Effects 0.000 description 3
- 229910002056 binary alloy Inorganic materials 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010924 continuous production Methods 0.000 description 2
- 230000036961 partial effect Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000686 essence Substances 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000005389 magnetism Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
- G06F9/3854—Instruction completion, e.g. retiring, committing or graduating
- G06F9/3856—Reordering of instructions, e.g. using queues or age tags
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides the method for a kind of method for constructing data structure without lockization and write-in data, reading data, which includes: setting data queue, and the data queue includes queue body；The queue body is divided into sequentially continuously arranged multiple queue element (QE)s, every team's column element is successively recorded using digital number according to the time sequencing that queue element (QE) enters queue body；The manufacturing schedule vernier of from the producer to queue body write-in data is generated according to the digital number for the queue element (QE) for being located at queue body head；The consumption progress vernier that the consumer of data is read from the queue body is generated according to the digital number for the queue element (QE) for being located at queue body tail portion.The data queue of construction method building, data processing speed is fast, is conducive to promote computer performance.
Description
Technical field
This application involves data processing field more particularly to a kind of method for constructing data structure without lockization and write-in number
According to, read data method.
Background technique
Computer needs to handle a large amount of data, how effectively to realize that storage and group organization data are one and important ask
Topic.
To improve the processing speed to data, multithreading can be used, concurrent operations are carried out (for example, write-in number to data
According to read data etc.), the safety and validity of data when to guarantee concurrent operations, it usually needs dependent on various locks behaviour
Make, for example, guaranteeing the consistency of data using mutual exclusion lock when existing largely to the concurrent reading and writing access request of data.
But a large amount of lock operation is used, it may lead the stopping of hair line journey or suspend mode, therefore make the delay of paired data access request,
Reduce data processing speed.
Therefore, how to improve there are when multiple concurrent access requests, the processing speed to data is problem to be solved.
Summary of the invention
In view of this, the application provides a kind of method for constructing data structure without lockization and write-in data, reads data
Method, to improve the processing speed of data.
In a first aspect, the present invention provides a kind of method for constructing data structure without lockization, which comprises
Data queue is set, and the data queue includes queue body；
The queue body is divided into sequentially continuously arranged multiple queue element (QE)s, enters queue body according to queue element (QE)
Time sequencing successively records every team's column element using digital number；
It is generated according to the digital number for the queue element (QE) for being located at queue body head to queue body write-in data
The manufacturing schedule vernier of the producer；
It is generated according to the digital number for the queue element (QE) for being located at queue body tail portion from the queue body and reads data
The consumption progress vernier of consumer.
The method according to claim 1, wherein the manufacturing schedule vernier includes producer's vernier sequence
Number, the digital number that the basis is located at the queue element (QE) on queue body head generates the life to queue body write-in data
The manufacturing schedule vernier of production person, comprising:
Using the digital number for the queue element (QE) positioned at queue body head determined as producer's vernier serial number；
The consumption progress vernier includes consumer's vernier serial number, and the basis is located at the queue element (QE) of queue body tail portion
The digital number of element generates the consumption progress vernier that the consumer of data is read from the queue body, comprising:
Root is using the digital number for the queue element (QE) positioned at queue body tail portion determined as consumer's vernier serial number.
Optionally, this method further include:
Create queuing memory file, the queuing memory file includes the queue body of the data queue and each described
Queue element (QE)；
Vernier memory file is created, the vernier memory file includes manufacturing schedule vernier and consumption progress vernier.
Optionally, the digital number is indicated using the binary system signless integer numerical value of setting position.
Optionally, the data queue further includes queue heads, and the queue element (QE) includes queue element (QE) head and queue element (QE)
Body；
The queue heads include following field: for identifying the version number of data queue's format, each queue element (QE)
The length and the queue body of the length of plain head and each queue element (QE) ferritic include the total quantity of queue element (QE)；
The queue element (QE) head includes following field: the physical length of the data queue indicates each queue element (QE)
Generate the timestamp of time and the digital number of each queue element (QE).
Second aspect, the present invention provide a kind of method to data queue's write-in data, and the data queue is according to upper
State the data queue of the method building of the building of data structure described in either method；The method packet of data is written to data queue for this
It includes:
The data queue is initialized, the initial memory buffer address of the data queue, the data are obtained
Producer's vernier serial number in the manufacturing schedule vernier of the physical length of queue and the data queue；
The queue element (QE) of data to be written is obtained according to the initial memory buffer address and producer's vernier serial number
Address；
Data are written into be written in the corresponding queue element (QE) in the queue element (QE) address；
The sum of the length of the data of write-in and the physical length of the data queue are used as, data are written next time
The physical length of data queue before；
Updated producer's vernier serial number is used as after producer's vernier serial number is increased setting value.
Optionally, the team of data to be written is obtained according to the initial memory buffer address and producer's vernier serial number
Column element address, comprising:
The offset address of the queue element (QE) of data to be written is determined according to producer's vernier serial number；
Queue element (QE) address by the sum of the initial memory buffer address and the offset address as data to be written.
Optionally, the offset of the queue element (QE) that data to be written are determined according to current producer's vernier serial number
Location, comprising:
It include the sum of queue element (QE) according to the queue body in current producer's vernier serial number and the data queue
Amount determines the index of the queue element (QE) of data to be written；
According to the product of the length of queue body in the data queue and the index of the queue element (QE) of the data to be written
With the offset address of the queue element (QE) of the length computation data to be written of queue heads in the data queue.
Optionally, the queue body according in producer's vernier serial number and the data queue includes queue element (QE)
Total quantity determine data to be written queue element (QE) index, comprising:
Total quantity of the producer's vernier serial number to the queue element (QE) that the queue body includes is carried out modulo operation to obtain
Index of the result arrived as the queue element (QE) of the data to be written.
The third aspect, the present invention provide a kind of method for reading data from data queue, appoint according to the data queue
The data queue of the method building of the building of data structure described in one method；This includes: from the method that data queue reads data
The data queue is initialized, obtain the data queue initial memory buffer address and the number
According to consumer's vernier serial number in the producer's vernier serial number and consumption progress vernier in the manufacturing schedule vernier of queue；
Judge whether producer's vernier serial number and the difference of consumer's vernier serial number are more than or equal to safe distance threshold
Value；
If so, adjusting consumer's vernier serial number, make producer's vernier serial number and the consumer adjusted
The difference of vernier serial number is equal to the safety distance threshold；
If it is not, local save and update producer's vernier serial number；
Judge whether consumer's vernier serial number is less than or equal to updated producer's vernier serial number；
If being equal to, data are not read from the data queue, executes local described above and save and update the production
The step of person's vernier serial number.
Optionally, this method further include:
When judging that consumer's vernier serial number is less than updated producer's vernier serial number, according to described first
Beginning memory buffer address and consumer's vernier serial number obtain the queue element (QE) address of data to be read；
Data are read from the corresponding queue element (QE) in queue element (QE) address of the data to be read；
Updated consumer's vernier serial number is used as after consumer's vernier serial number is increased setting value.
The above-mentioned construction method it can be seen from above description is recorded in data queue by the serial number of queue element (QE)
Queue element (QE), avoidable index unroll problem, without lock mechanism can be realized to data queue go out to and enqueue operations,
Realize the data structure without lockization, data processing speed is fast, can promote computer performance.Also, due to not needing lock mechanism, disappear
Expense person's thread and producer thread can be the different threads in the same process, the thread being also possible in different processes.It can
It realizes multithreading or multi-process parallel processing, is applicable under the higher scene of data processing speed performance to computer.
Detailed description of the invention
Fig. 1 is a kind of process signal of method for constructing data structure without lockization shown in one exemplary embodiment of the application
Figure；
Fig. 2 is a kind of data structure schematic diagram of data queue shown in one exemplary embodiment of the application
Fig. 3 is a kind of process signal of method to data queue's write-in data shown in one exemplary embodiment of the application
Figure；
Fig. 4 is a kind of process signal of method that data are read from data queue shown in one exemplary embodiment of the application
Figure；
Fig. 5 is a kind of hardware structure schematic diagram of computer equipment shown in one exemplary embodiment of the application.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all embodiments consistent with the application.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the application.
It is only to be not intended to be limiting the application merely for for the purpose of describing particular embodiments in term used in this application.
It is also intended in the application and the "an" of singular used in the attached claims, " described " and "the" including majority
Form, unless the context clearly indicates other meaning.It is also understood that term "and/or" used herein refers to and wraps
It may be combined containing one or more associated any or all of project listed.
Data structure is that a kind of pair of data used in computer carry out storage and organizational form, data queue (Queue)
It is a kind of common data structure.
Data queue is that one kind is operated according to the principle of " first in first out ", and queue is written from one end in data, can be described as
" joining the team " reads data from the other end, and data are left queue, be can be described as " team out ", i.e., the element of most advanced enqueue is also most
The element of first dequeue.Caused thread stops when can improving concurrent operations to a certain extent or suspend mode is asked for data queue
Topic improves data processing speed.
In data structure level, there are two types of implementations for data queue:
(1) based on the data queue of chained list
Data queue based on chained list, the queue are indicated with chained list, are recorded by a head pointer and a tail pointer
The head node and tail node of chained list, the producer write data into queue (joining the team), are equivalent to chained list head and add new node, disappear
Data (team out) is read in Fei Zhecong queue, is equivalent to from chained list tail portion and takes out (deletion) node.The above operation can also in turn,
The producer writes data into queue from chained list tail portion, and consumer reads data from chained list head.
Refer to/tail pointer since enqueue/dequeue needs to modify head, when there are multiple concurrently access read-write requests, needs to lead to
It crosses lock mechanism to protect data queue, the atom lock mechanism of lightweight generally can not be used, the critical of heavyweight need to be used
The lock mechanisms such as area or semaphore, the heavier lock mechanism of use is unfavorable for the raising of data processing speed, in the data to computer
Under the higher scene of processing speed performance, it is often used without the data queue based on chained list.
(2) based on the data queue of array.
It is realized based on the data queue of array by array, specifically, being drawn point by head index with a pygochord
The head element and the corresponding array indexing of tail element of other record queue.The producer writes data into queue (joining the team), be equivalent to
Data are written in the array element that head index is directed toward；Consumer reads data (team out) from queue, is equivalent to from pygochord and draws direction
Array element takes out data.
The opposite data queue based on chained list, the data queue based on array only needs to update head index and pygochord draws i.e.
Can, the atomic lock of lightweight can be used, therefore, help to improve data processing speed, promote computer performance.
But this based on the data queue of array, there are following main problems:
(1) it needs to consider the problems of that index unrolls
When the producer writes data into queue, need to compare the size that head index and pygochord draw, whether to judge queue
Expire.Normal condition following index draws more than or equal to pygochord, illustrates that queue is less than, can write data into queue, when indexing back
Around when, head index can draw less than pygochord, at this time, it may be necessary to could judge whether queue has expired using more complicated condition, therefore, shadow
Ring data-handling efficiency.
(2) data element in queue is operated for being drawn by head index and pygochord, is based on atom variable
Address is operated, thus can only multiple cross-threads in individual process use, can not be used between different processes, therefore,
It is not suitable for multi-process.
Above-mentioned data queue there are aiming at the problem that, the present invention provides a kind of data queue's construction method without lockization, can
To realize can also fitting the problem of data processing speed not only can be improved, and unroll there is no index without lockization for data queue
For multi-process.
Fig. 1 is a kind of process signal of method for constructing data structure without lockization shown in one exemplary embodiment of the application
Figure, it is shown referring to Fig.1, this method comprises:
Step 100, setting data queue, data queue includes queue body.
In this step, data queue is also that one kind is operated based on the principle of " first in first out ", and data are written from one end
Queue reads data from the other end.The data structure of the data queue includes queue body, can also include queue heads.
Queue body is divided into sequentially continuously arranged multiple queue element (QE)s by step 110, enters queue according to queue element (QE)
The time sequencing of body successively records every team's column element using digital number, and queue element (QE) includes queue element (QE) head and queue element (QE)
Body.
In this step, queue body includes multiple queue element (QE)s, and the time order and function for entering queue body herein with queue element (QE) is suitable
Digital number (following digital number abbreviation serial number, indicated with SEQ) is set for queue element (QE) on the basis of sequence.In computer technology,
Machine language generallys use binary counting, and therefore, the digital number of queue element (QE) is, for example, to use binary representation, and the sequence
Number SEQ is signless integer numerical value, the index times that the digit of the binary system serial number is 2, such as 64 or 128 etc., when
Right digital number can also indicate that the present embodiment does not limit this with the numerical value of other forms.
For the serial number SEQ of record queue element, the serial number SEQ for first entering queue element (QE) is, for example, 0, is entered later
The serial number of the queue element (QE) of team successively adds 1.According to 64 binary representation serial number SEQ, then the maximum value of serial number SEQ is
18446744073709551615 (more than 184,400,000,000 hundred million), by taking the maximum frequency 4GHz of the CPU of current computer is calculated as an example, CPU
Each clock cycle of operation consumes a serial number (CPU executes every instruction and at least consumes one or more clock cycle), then
The serial number of queue element (QE) is used continuously 4,200,000,000 seconds enough in above-mentioned data queue, and about 136 years, in the computer of operation this method
It may be considered unlimited serial number in the life cycle of program.
Because there is queue in data queue when the producer writes data into data queue using " first in first out " principle
It (joins the team) when element enters queue body, queue element (QE) enters queue body from the head of queue body, when consumer reads from data queue
Access according to when, that is, when having queue element (QE) to leave queue body (out to), queue element (QE) leaves queue body from the tail portion of queue body.With
Constantly go out team or join the team, positioned at the serial number of the queue element (QE) on queue body head and the sequence of the queue element (QE) positioned at queue body tail portion
It is number incremented by successively, such as incremented by successively 1.
Queue element (QE) positioned at queue body head enters the time of queue body earlier than the queue element (QE) for being located at queue body tail portion
Into the time of queue body, therefore the serial number for being located at the queue element (QE) on queue body head can be greater than the queue positioned at queue body tail portion
The serial number of element, and the serial number of queue element (QE) has enough growth spaces, it is therefore, avoidable to index problem of unrolling.
Step 120 is generated according to the digital number for the queue element (QE) for being located at queue body head to data queue's write-in data
The producer manufacturing schedule vernier；
Step 130 generates from data queue according to the digital number for the queue element (QE) for being located at queue body tail portion and reads data
Consumer consumption progress vernier.
In above-mentioned two step, the concept of vernier (being denoted as Cursor), including manufacturing schedule vernier and consumption progress are introduced
Vernier, the producer be to the logic executor of queue body write-in data (being put into queue element (QE) to queue body), in logic can be
It is executed in one thread, referred to as producer thread.Consumer is to read data from queue body (to take out queue element (QE) from queue body
Element) logic executor, can be executed in a thread in logic, referred to as consumer thread.
Manufacturing schedule vernier (is denoted as Producer Corsur), for recording the manufacturing schedule of the producer.The trip of consumption progress
It marks (being denoted as Consumer Corsur), for recording the consumption progress of consumer.
Manufacturing schedule vernier may include but be not limited to following element: producer's vernier serial number, the serial number enter next time
The serial number of the queue element (QE) of queue body, producer's vernier serial number can be for positioned at the serial numbers of the queue element (QE) of queue head；It is raw
Production person's vernier timestamp, for describing the renewal time of producer's vernier serial number.
Consumption progress vernier may include but be not limited to following element: consumer's vernier serial number, the serial number are left next time
The serial number of the queue element (QE) of queue body, the serial number can be for positioned at the serial numbers of the queue element (QE) of queue tail；When consumer's vernier
Between stab, for describing the renewal time of consumer's vernier serial number.
Vernier (including manufacturing schedule vernier and consumption progress vernier) is really one piece of memory buffer (buffer), if number
According to queue for same process internal thread communicate, then memory buffer buffer can by basis Memory Allocation API come
It obtains, for example by calling the malloc function in the library C or calling the new function etc. in C++, can also be based on passing through
The system of mmap (method of Memory Mapping File) a kind of, which is called, to be obtained.
For being located at the serial number of queue element (QE) of queue head, queue element (QE) is to enter queue from queue body head
Body, for first entering the serial number of queue element (QE) of queue body, for example, 0, the serial number of the queue element (QE) joined the team later according to
It is secondary to add 1, therefore, as queue element (QE) enters queue body, the serial number of the queue element (QE) positioned at queue head can be obtained.
For being located at the serial number of queue element (QE) of queue tail, since the length of queue body is fixed, that is,
Positioned at queue body tail portion queue element (QE) be located at queue body head the distance between queue element (QE) be it is confirmable, therefore,
It can be calculated according to the difference between the serial number of queue element (QE) and the length of queue body for being located at queue body head positioned at queue
The serial number of the queue element (QE) of body tail portion, for example, being located at the serial number M of the queue element (QE) on queue body head, the length example of queue body
For example N is then located at the serial number M-N of the queue element (QE) of queue body tail portion.
The manufacturing schedule of the producer can be known by manufacturing schedule vernier and consumption progress vernier, i.e. enter queue next time
The position of the queue element (QE) of body and leave next time queue body queue element (QE) position.
According to existing above-mentioned data queue based on array or based on the data queue of chained list, when the producer produces
Data, that is, when thering are data to join the team, it is necessary first to judge whether data queue has expired, if data queue has expired, need to wait for disappearing
The person's of expense consumption data has dequeuing data, otherwise data join the team and can cover the data joined the team earliest at this time.
Work as consumption data of consumers, that is, when having dequeuing data, it is necessary first to judge whether data queue is sky, if data
Queue is sky, then needs to wait producer's creation data, that is, have data to join the team, otherwise consumer can get invalid data at this time.
The reason of causing the above problem is: the speed of production of the producer is greater than the consumption rate of consumer, and the producer is necessary
The consumption progress of consumer is paid close attention to, consumer must also pay close attention to the manufacturing schedule of the producer, must pass through lock mechanism management at this time
Head index or pygochord draw, and get invalid data when covering old data or dequeuing data when joining the team to avoid data.
Theoretically, the scene that the speed of production of the producer is greater than the consumption rate of consumer necessarily occurs.For this purpose, tradition is thought
Road is to solve the problems, such as this by inhibiting the speed of production (rhythm) of the producer.When speed of production is greater than consumption rate, data
Queue can gradually completely, the producer can wait consumer spending at this time for filling, until data queue is non-full state, producer's ability
Continue that data are written to data queue.But if the data source of the producer constantly generates data simultaneously, the producer can only be
A part of data are actively abandoned before joining the team.
If we change a kind of thinking: when (or close to completely) has been expired by data queue, consumer being allowed actively to abandon number
According to then the producer is also just not necessarily to pay close attention to the consumption progress of consumer, so the producer is not without being concerned about whether data queue has expired
It needs to lock data queue.And the manufacturing schedule of the producer can be read directly in consumer, it is only necessary to guarantee the consumption of consumer
Progress is less than or equal to the manufacturing schedule of the producer (not more than) read.The producer does not need concern consumption in this way
Person, consumer only need to read the manufacturing schedule of the producer, and concern becomes unidirectional from two-way, and producers and consumers do not need
Lock queue.
This is also the thinking of the present invention program, according to foregoing description it is found that being provided with manufacturing schedule vernier in this programme
With consumption progress vernier, consumer only need to may know that the manufacturing schedule of the producer, the producer by reading manufacturing schedule vernier
It only needs data speed of production (rhythm) the continuous production data according to oneself and data queue is written, without paying close attention to consumption
Talking about without lock for data queue may be implemented that is, without reading any consumer's vernier in the consumption progress of person.
Seen from the above description, the scheme of this example records the queue element (QE) in data queue by the serial number of queue element (QE)
Element, avoidable index unroll problem, without lock mechanism can be realized to data queue go out to and enqueue operations, realize nothing
The data structure of lockization, data processing speed is fast, can promote computer performance.Also, due to not needing lock mechanism, consumer's line
Journey and producer thread can be the different threads in the same process, the thread being also possible in different processes.It can be achieved more
Thread parallel processing, is applicable under the higher scene of data processing speed performance to computer.
In some instances, above-mentioned data queue can also include queue heads, queue element (QE) can in addition to including queue body
To include queue element (QE) head and queue element (QE) ferritic；
Queue heads include following field: for the version number of mark data queue format, the length of every team's column element head and
The length and queue body of every team's column element body include the total quantity of queue element (QE)；
Queue element (QE) head includes following field: the physical length of data queue, indicate the individual queue Element generation time when
Between stamp and every team's column element digital number.
Referring to the data structure of specific descriptions data queue provided in this embodiment shown in Fig. 2, referring to shown in Fig. 2,
Data queue includes queue heads and queue body, and queue body includes N number of queue element (QE), for example, queue element (QE) 1, queue element (QE) 2, team
Column element 3, queue element (QE) 4 ..., queue element (QE) N；Each queue element (QE) includes queue element (QE) head and queue element (QE) ferritic.
Queue heads may include but be not limited to following field:
(1) version number is used for mark data queue format；
(2) timestamp, for describing the generation time of queue body；
(3) length of the length of queue element (QE) head, usual every team's column element head is all the same, therefore, the queue element (QE) head
Length can indicate the length of each queue element (QE) head, be represented by Element Header Size；
(4) length of the length of queue element (QE) ferritic, usual every team's column element is all the same, therefore, the queue element (QE) ferritic it is big
The small length that can indicate each queue element (QE) ferritic, which is the maximum value of queue element (QE) ferritic, is represented by Size
Element Body Max；
(5) queue body includes the total quantity of queue element (QE), the alternatively referred to as length of queue body, is represented by Element
Count。
The data structure of queue heads is described by taking C language as an example are as follows:
Queue element (QE) head may include but be not limited to following field:
(1) physical length of queue body, for describing in queue body actually comprising the quantity of queue element (QE)；
(2) timestamp, for describing the generation time of queue element (QE)；
(3) version number；For identifying the format of queue element (QE) head；
(4) serial number of queue element (QE), for describing the digital number of queue element (QE).
Queue element (QE) head data structure is described by taking C language as an example are as follows:
The data structure of queue element (QE) is described by taking C language as an example are as follows:
The data structure of entire data queue is described by taking C language as an example are as follows:
The length of above-mentioned queue element (QE) head refers to the size of the memory headroom of computer occupied by queue element (QE) head, can use word
According with number or byte number indicates, for example, the length of queue element (QE) head is 5 characters, refers to that the occupied memory of queue element (QE) head is empty
Between be 5 characters.
Similar, the length of queue element (QE) ferritic refers to the occupied memory headroom size of queue element (QE) ferritic, the length of queue body
Refer to that the occupied memory headroom size of queue body, the length of queue heads refer to the occupied memory headroom size of queue heads, data team
The length index of column is according to the occupied memory headroom size of queue.
In an optional embodiment, the method for constructing data structure without lockization, can with the following steps are included:
Step S140, queuing memory file is created, queuing memory file includes the queue heads of the data queue, queue body
With every team's column element and its serial number；
Step S150, vernier memory file is created, vernier queuing memory file includes manufacturing schedule vernier and consumption progress
Vernier.
In order to can still read after above-mentioned data queue is written data or reads the program restarting of data
Therefore the data queue at secondary (i.e. the newest moment of last time degree operation) and the relevant information of vernier create queuing memory file
With creation vernier memory file.It include the relevant information of queue body and every team's column element in queuing memory file, in vernier queue
Deposit includes manufacturing schedule vernier and the relevant information for consuming progress vernier in file.
The embodiment of the invention also provides a kind of data read-write method based on data queue, according to the data queue
The data queue of the method building of the data structure building of any of the above-described embodiment；As shown in figure 3, the data read-write method packet
It includes:
Step 200 initializes data queue, obtains initial memory buffer address, the data queue of data queue
Physical length and data queue manufacturing schedule vernier in producer's vernier serial number.
Before being written and read processing for data queue, the algorithm of data queue's initialization can be first carried out to data team
Column are initialized, can be by independent queue initialization program Lai the initialization of complete paired data queue, it may include following procedure:
Firstly, queuing memory initialization is carried out, specifically includes the following steps:
Determine that queuing memory file whether there is；
If it does not exist, then according to the relevant parameter of data queue creation queuing memory file, in the queuing memory file
Relevant information including data queue, for example, data queue initial memory buffer address and data queue physical length, when
It so can also include the other information of data queue described above, such as queue heads, queue body and queue element (QE) (queue element (QE)
Plain head and queue element (QE) ferritic) etc..
If queuing memory file has existed, required relevant information is directly obtained from queuing memory file i.e.
Can, such as initial memory buffer address and the physical length of data queue etc. of data queue.
Secondly, vernier internal memory initialization is carried out, specifically includes the following steps:
Determine that vernier backup file whether there is；
If it is present with vernier backup file initialization vernier memory file (for example, the vernier after copy is backed up text
Part is as vernier memory file)；
If it does not, further checking whether there are vernier memory file, if it does not exist, then creation vernier memory text
Producer's vernier serial number of manufacturing schedule vernier is initialized as 0, otherwise directly reads the producer in vernier memory file by part
Vernier serial number can also further read consumer's vernier serial number.
Specifically, vernier memory file may include producer's vernier memory file and consumer's vernier memory file,
And there is usually one corresponding producer's vernier memory file can be denoted as Producer Memory to manufacturing schedule vernier
File.Consumption progress vernier can be denoted as Consumer for one or more, corresponding consumer's vernier memory file
Which consumer Memory File-X, X expression is.
Queuing memory file and producer's vernier memory text are opened with read-write mode for example, can call by mmap system
Part obtains queuing memory buffering buffer (i.e. Address of Queue) and producer vernier memory buffer buffer, in turn
Get initial memory buffer address and the producer's vernier serial number of data queue.
Step S210, the queue element (QE) of data to be written is obtained according to initial memory buffer address and producer's vernier serial number
Address；
Step S220, it is written into the corresponding queue element (QE) in data write-in queue element (QE) address.
In above-mentioned two step, determined first according to initial memory buffer address and current producer's vernier serial number to be written
The queue element (QE) address of data is written into data later and is written in the corresponding queue element (QE) in queue element (QE) address, complete in this way
At the write-in of data, specifically, data to be written are that the producer obtains from data source, data to be written are specifically writable
In the queue element (QE) ferritic of queue element (QE).
Step S230, the sum of the length of data to be written and the physical length of data queue are used as and data are written next time
The physical length of data queue before.
The physical length of data queue is the total length for having been written into this partition queue element element of data, whenever there is new number
After write-in data queue, the physical length of data queue can change, and therefore, be written after data to data queue every time
Physical length is updated, using the physical length of updated data queue as data queue before write-in data next time
Physical length, the physical length of the data queue are, for example, the physical length of queue body, update herein and are specifically as follows update team
The physical length of queue body in column element head.
Step S240, updated producer's vernier serial number is used as after producer's vernier serial number being increased setting value.
It is further to update producer's vernier serial number in this step, when being written into data write-in data queue each time,
There is new queue element (QE) to enter queue body head, corresponding producer's vernier serial number can be incremented by successively, such as is incremented by 1, accordingly
Updated producer's vernier serial number can be obtained.Certainly, increased setting value of data producer vernier serial number of every write-in
It can be other quantity, however it is not limited to setting value 1.
In the present embodiment, the process of data write-in data queue, the i.e. implementation procedure of producer's algorithm are described, is recycled
Above-mentioned steps are executed, successively to data queue's recurrent wrIting data.
In an optional embodiment, according to initial memory buffer address and the producer described in above-mentioned steps S210
Vernier serial number obtains the queue element (QE) address of data to be written, may comprise steps of:
Step S211, the offset address of the queue element (QE) of data to be written is determined according to producer's vernier serial number；
Step S212, the queue element (QE) by the sum of initial memory buffer address and offset address as data to be written
Location.
The offset of the queue element (QE) of data to be written is determined described in above-mentioned steps S211 according to producer's vernier serial number
Address can specifically include, comprising:
S2110, it is determined according to the queue body in producer's vernier serial number and data queue comprising the total quantity of queue element (QE)
The index of the queue element (QE) of data to be written；
Assuming that the length that the total quantity of queue element (QE) is queue body is N, the serial number of queue element (QE) is denoted as SEQ, can basis
Following formula one calculate the index INDEX of queue element (QE):
Formula one: INDEX=SEQ%N.
In above-mentioned formula one, % indicates modulo operation, then the index of queue element (QE) N modulus can be obtained by SEQ
INDEX。
By the description of previous embodiment it is found that producer's vernier serial number is located at the sequence of the queue element (QE) on queue body head
Number, the index of the queue element (QE) of data to be written can be calculated according to above-mentioned formula one, i.e., by current producer's vernier serial number
Modulo operation is carried out to the total quantity for the queue element (QE) that queue body includes, obtained result can be used as the queue of data to be written
The index of element.
S2111, according to the sum of products of the length of queue body in data queue and the index of the queue element (QE) of data to be written
The offset address of the queue element (QE) of the length computation of queue heads data to be written in data queue.
The step is specifically, the offset address of the queue element (QE) of data to be written can be queue element (QE) relative to queue
The offset address on body head illustrates the offset address for the queue element (QE) for determining above-mentioned data to be written below with a specific example
Process, referring to the data structure of data queue shown in Fig. 2, specifically:
Data queue includes: queue heads (being denoted as Queue Header) and queue body (being denoted as Queue Body).Queue heads
For fixed length, size is denoted as Size of Queue Header；Queue body is fixed length, is made of N number of continuous queue element (QE), can be with
It is interpreted as queue element (QE) array, N is also the total data for the queue element (QE) that queue body includes here, can be described as of queue element (QE)
Number, or the length of referred to as queue body.
Queue element (QE) is made of queue element (QE) head and queue element (QE) ferritic.Queue element (QE) head is fixed length, and size is denoted as Size of
Element Header.Queue element (QE) ferritic be it is elongated, maximum value is denoted as Size of Element Body.Queue element (QE) is most
Big value is denoted as Size of Element, then the length of queue element (QE) can be calculated by following formula two:
Two: Size of Element=Size of Element Header+Size of Element Body of formula；
Then the length Size of Queue of data queue can be calculated according to following formula three:
Three: Size of Queue=Size of Queue Header+Size of Element*N of formula；
Queue element (QE) is denoted as Off set of Element relative to the deflected length on queue body head, is based on following formula
Four can determine above-mentioned deflected length by the serial number of queue element (QE) are as follows:
Four: Offset of Element=Size of Queue Header+Size of Element* of formula
INDEX。
Assuming that the address of the initial memory buffering buffer of data queue is Address of Queue, then queue element (QE)
Location (being denoted as Address of Element) can be calculated according to following formula five:
Five: Address of Element=Address of Queue+Offset of Element of formula；
Based on above-mentioned formula it is found that the queue element (QE) for determining data to be written serial number (i.e. producer's vernier serial number)
Behind initial memory buffer address, the queue element (QE) address of data to be written can be determined.
The embodiment of the present invention also provides a kind of method for reading data from data queue, as shown in figure 4, this method is specifically wrapped
Include step:
Step S300, data queue is initialized, obtains initial memory buffer address and the data of data queue
Consumer's vernier serial number in producer's vernier serial number and consumption progress vernier in the manufacturing schedule vernier of queue；
Before being read out data for data queue, the algorithm of data queue's initialization is first carried out to data queue
It is initialized, guarantees that queuing memory file, producer's vernier memory file and consumer's vernier memory file exist.
Specifically, vernier memory file may include producer's vernier memory file and consumer's vernier memory file,
And there is usually one corresponding producer's shared memory file can be denoted as Producer Memory to manufacturing schedule vernier
File.Consumption progress vernier can be denoted as Consumer for one or more, corresponding consumer's vernier memory file
Which consumer Memory File-X, X expression is.
Queuing memory file and producer's vernier memory are opened with read-only mode specifically, can call by mmap system
File obtains the memory buffer buffer (i.e. Address of Queue) and producer's vernier memory buffer of data queue
Buffer, and then get initial memory buffer address and the producer's vernier serial number of data queue.
It can be called by mmap system and consumer's vernier memory file is opened with read-write mode, obtain consumer's vernier memory
Buffer is buffered, and then gets consumer person's vernier serial number.
Step S301, judge whether producer's vernier serial number and the difference of consumer's vernier serial number are more than or equal to safe distance threshold
Value；
Step S302, if so, adjustment consumer's vernier serial number, makes producer's vernier serial number and consumer's vernier adjusted
The difference of serial number is equal to safety distance threshold.
In above-mentioned two step, first determine whether producer's vernier serial number and the difference of consumer's vernier serial number are more than or equal to
Safety distance threshold, reason are: consumer's vernier serial number can theoretically be less than or equal to producer's vernier serial number, but when production
When the formation speed of person is much larger than the consumption rate of consumer, data queue, which can write, to be expired or approaches completely, in order to avoid such feelings
Condition in the present embodiment, is actively abandoned partial data using the method for adjustment consumer's vernier serial number, that is, consumer, will be worked as
Partition queue element element in the queue body of preceding data queue close to queue body tail portion is abandoned, in this way, consumer's vernier serial number is actively
Close to queue body head, consumer's vernier serial number will increase, consumer's vernier sequence after making producer's vernier serial number and increasing
Number difference be equal to safety distance threshold, guarantee that queue will not expire in such a way that consumer actively abandons partial data.
Here safety distance threshold can be a certain value of the suitably less than length of queue body, for example, for queue body
The 3/4 of length.
Step S303, if it is not, local save and update producer's vernier serial number；
Step S304, judge whether consumer's vernier serial number is less than or equal to producer's vernier serial number；
If step S305, being equal to, data are not read from data queue, execute above-mentioned steps 303；
If step S306, being less than, data to be read are obtained according to initial memory buffer address and consumer's vernier serial number
Queue element (QE) address；
According to current consumer's vernier serial number, corresponding queue element (QE) address is obtained according to formula five
AddressofElement is converted to queue element (QE) data structure.
Step S307, data are read from the corresponding queue element (QE) in queue element (QE) address of data to be read.
When the difference of producer's vernier serial number and consumer's vernier serial number is within safety distance threshold, consumer can be from
Data are read in data queue, subsequent that data are written to data queue with the producer, producer's vernier serial number will increase, this
When, consumer thread can locally save and update producer's vernier serial number, i.e. consumer can further locally create one
The copy of part producer's vernier serial number is subsequent directly to read updated producer's vernier sequence from local as local variable
Number.
Further compare consumer's vernier serial number and producer's vernier serial number later, when consumer's vernier serial number is equal to life
When production person's vernier serial number, illustrate no new data, consumer does not read data, executes above-mentioned steps 303, subsequent with the producer
Data are written to data queue, producer's vernier serial number will increase, when consumer's vernier serial number is less than producer's vernier serial number,
Illustrate have new data that can read, in turn, reads data from data queue.
It is true according to initial memory buffer address and current consumer's vernier serial number first when reading data from data queue
The queue element (QE) address of fixed data to be read, and then data are read from the corresponding queue element (QE) in queue element (QE) address, it completes
The reading of data, and then required processing can be carried out to data are read.
Step S308, by consumer's vernier serial number after consumer's vernier serial number increase setting value as update.
It is further to update consumer's vernier serial number in this step, each time after data queue's reading data, that is, it is located at
The queue element (QE) of queue body tail portion can leave queue body, and corresponding consumer's vernier serial number can be incremented by successively, such as be incremented by 1, according to
Updated consumer's vernier serial number can be obtained in this.Certainly, data of every reading, the increased setting value of consumer's vernier serial number
It may be other quantity, however it is not limited to setting value 1.
The method for reading data of the present embodiment describes the process that data are read from data queue, i.e. consumer algorithm
Implementation procedure is recycled and executes above-mentioned steps, successively recycles from data queue and reads data.If there are multiple consumers, each disappear
The person of expense is identical to the process of reading data, and details are not described herein.
By read method to above-mentioned data queue and wiring method description it is found that the embodiment of the present invention
Method has the advantages that
(1) there is the serial number concept of the queue element (QE) of " infinite space ", producer's vernier serial number and consumer by introducing
Vernier serial number monotonic increase does not need using lock mechanism without carrying out operation of unrolling according to the length of queue body, is a kind of
Really without Suo Hua data queue, the operation complexity to data can be reduced.
(2) data queue without lockization can striding course use, it is more convenient for being split as according to computer function to be performed
A process executes, and carries out logic decoupling, and producer's process or consumer process's collapse do not influence another party.
(3) producer according to oneself data speed of production/rhythm continuous production data and queue is written, without close
It infuses consumer spending progress (without reading any consumer's vernier information).It here is the key that really without lock.
(4) consumer need to only pay close attention to manufacturing schedule (need to read producer's vernier serial number), when the consumption progress of consumer is big
When lagging behind the manufacturing schedule (difference of the two vernier serial number is greater than safety distance threshold) of the producer greatly, then consumer actively abandons
A part of data realize the key really without Suo Hua data queue here.
(5) if we regard data queue as a lacking beginning and end data flow, consumer is exactly to exist forever
The producer is pursued, the producer is slow, and consumer will produce the equalizer the producer (but not being more than)；The producer is quicker, consumer
When falling behind, consumer actively abandons a part of data, makes to keep a balance therebetween.
(6) by introducing the concept of producer's vernier and the backup of consumer's vernier, producers or consumers' journey can be supported
After sequence is collapsed or restarted, it is still able to maintain the manufacturing schedule of data queue and the consistency of consumption progress.
In an optional embodiment, further manufacturing schedule vernier and consumption progress vernier can also be carried out
After backup, the program for executing above-mentioned producer's algorithm if necessary and the program for executing consumer algorithm are restarted, remain in reading
Secondary manufacturing schedule vernier and consumption progress vernier, then can be used shared drive and swim as manufacturing schedule vernier and consumption progress
Target memory buffer buffer.
Specifically, including following procedure:
Before the operating system of computer is shut down/restarted, by the memory buffer of manufacturing schedule vernier and consumption progress vernier
Buffer is saved in backup file, in short, can be realized by copying to vernier memory file, is covered with vernier memory file
Lid vernier backup file.Aforesaid operations can be completed by shutdown self-starting script.
After starting operating system, pass through vernier backup file generated vernier memory file.This operation can pass through booting
Self-starting script is completed.
By introducing the concept to manufacturing schedule vernier and consumption progress vernier backup, producers or consumers can be supported
The production of program crashing or the data queue after restarting or the consistency for consuming progress.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, the journey
The step of any of the above-described embodiment the method is realized when sequence is executed by processor.
The method for constructing data structure without lockization of above-described embodiment, to the method for data queue's write-in data and from data
The method that data are read in queue can be realized by computer equipment.
Fig. 5 is the hardware structure schematic diagram of computer equipment 500 provided in an embodiment of the present invention.It is shown in Figure 5, it should
Computer equipment includes: machine readable storage medium 501 and processor 502, can also include non-volatile media 503, communication
Interface 504 and bus 505；Wherein, machine readable storage medium 501, processor 502, non-volatile media 503 and communication connect
Mouth 504 completes mutual communication by bus 505.Processor 502 is by reading and executing in machine readable storage medium 501
With the method for constructing data structure without lockization, the method to data queue's write-in data or the method from data queue's reading data
The corresponding machine-executable instruction of control logic, can be performed foregoing embodiments description method.
Machine readable storage medium referred to herein can be any electronics, magnetism, optics or other physical stores dress
It sets, may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium may is that RAM
(Radom Access Memory, random access memory), volatile memory, nonvolatile memory, flash memory, storage driving
Device (such as hard disk drive), any kind of storage dish (such as CD, dvd) perhaps similar storage medium or them
Combination.
Non-volatile media can be nonvolatile memory, flash memory, memory driver (such as hard disk drive), any class
The storage dish (such as CD, dvd) of type perhaps similar non-volatile memory medium or their combination.
In addition, computer equipment can be various electronic equipments, such as server, mobile phone, personal digital assistant
(PDA) etc., it names just a few.
The foregoing is merely the preferred embodiments of the application, not to limit the application, all essences in the application
Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the application protection.
Claims (11)
1. a kind of method for constructing data structure without lockization, which is characterized in that the described method includes:
Data queue is set, and the data queue includes queue body；
The queue body is divided into sequentially continuously arranged multiple queue element (QE)s, the time of queue body is entered according to queue element (QE)
Sequence successively records every team's column element using digital number；
The production to queue body write-in data is generated according to the digital number for the queue element (QE) for being located at queue body head
The manufacturing schedule vernier of person；
The consumption that data are read from the queue body is generated according to the digital number for the queue element (QE) for being located at queue body tail portion
The consumption progress vernier of person.
2. the method according to claim 1, wherein the manufacturing schedule vernier includes producer's vernier serial number,
The digital number that the basis is located at the queue element (QE) on queue body head generates the production to queue body write-in data
The manufacturing schedule vernier of person, comprising:
Using the digital number for the queue element (QE) positioned at queue body head determined as producer's vernier serial number；
The consumption progress vernier includes consumer's vernier serial number, and the basis is located at the queue element (QE) of queue body tail portion
Digital number generates the consumption progress vernier that the consumer of data is read from the queue body, comprising:
Root is using the digital number for the queue element (QE) positioned at queue body tail portion determined as consumer's vernier serial number.
3. the method according to claim 1, wherein further include:
Create queuing memory file, the queuing memory file include the data queue the queue body and each queue
Element；
Vernier memory file is created, the vernier memory file includes manufacturing schedule vernier and consumption progress vernier.
4. method according to claim 1-3, which is characterized in that the digital number using setting position two into
Signless integer numerical value processed indicates.
5. method according to claim 1 to 3, which is characterized in that the data queue further includes queue heads and queue
Body, the queue body are queue element (QE) sequence, and the queue element (QE) includes queue element (QE) head and queue element (QE) ferritic；
The queue heads include following field: for identifying the version number of data queue's format, each queue element (QE) head
Length and each queue element (QE) ferritic length and the queue body include queue element (QE) total quantity；
The queue element (QE) head includes following field: the physical length of the data queue indicates that each queue element (QE) generates
The digital number of the timestamp of time and each queue element (QE).
6. a kind of method to data queue's write-in data, which is characterized in that the data queue is according to the claims
The data queue of the method building of the described in any item data structure buildings of 1-5；The method packet of data is written to data queue for this
It includes:
The data queue is initialized, the initial memory buffer address of the data queue, the data queue are obtained
Physical length and the data queue manufacturing schedule vernier in producer's vernier serial number；
The queue element (QE) address of data to be written is obtained according to the initial memory buffer address and producer's vernier serial number；
Data are written into be written in the corresponding queue element (QE) in the queue element (QE) address；
The sum of the length of the data of write-in and the physical length of the data queue are used as and are written before data next time
The physical length of data queue；
Updated producer's vernier serial number is used as after producer's vernier serial number is increased setting value.
7. according to the method described in claim 6, it is characterized in that, according to the initial memory buffer address and the producer
Vernier serial number obtains the queue element (QE) address of data to be written, comprising:
The offset address of the queue element (QE) of data to be written is determined according to producer's vernier serial number；
Queue element (QE) address by the sum of the initial memory buffer address and the offset address as data to be written.
8. the method according to the description of claim 7 is characterized in that it is described according to current producer's vernier serial number determine to
The offset address of the queue element (QE) of data is written, comprising:
The total quantity for including queue element (QE) according to the queue body in current producer's vernier serial number and the data queue is true
The index of the queue element (QE) of fixed data to be written；
According to the sum of products institute of the length of queue body in the data queue and the index of the queue element (QE) of the data to be written
State the offset address of the queue element (QE) of the length computation data to be written of queue heads in data queue.
9. according to the method described in claim 8, it is characterized in that, described according to producer's vernier serial number and the data
Queue body in queue includes the index that the total quantity of queue element (QE) determines the queue element (QE) of data to be written, comprising:
Total quantity of the producer's vernier serial number to the queue element (QE) that the queue body includes is carried out modulo operation to obtain
As a result the index as the queue element (QE) of the data to be written.
10. a kind of method for reading data from data queue, which is characterized in that the data queue is according to the claims
The data queue of the method building of the described in any item data structure buildings of 1-5；This reads the method packet of data from data queue
It includes characterized by comprising
The data queue is initialized, obtain the data queue initial memory buffer address and the data team
Consumer's vernier serial number in producer's vernier serial number and consumption progress vernier in the manufacturing schedule vernier of column；
Judge whether producer's vernier serial number and the difference of consumer's vernier serial number are more than or equal to safety distance threshold；
If so, adjusting consumer's vernier serial number, make producer's vernier serial number and consumer's vernier adjusted
The difference of serial number is equal to the safety distance threshold；
If it is not, local save and update producer's vernier serial number；
Judge whether consumer's vernier serial number is less than or equal to updated producer's vernier serial number；
If being equal to, data are not read from the data queue, executes local described above and save and update producer's trip
The step of marking serial number.
11. according to the method described in claim 10, it is characterized by further comprising:
When judging that consumer's vernier serial number is less than updated producer's vernier serial number, according to described initial interior
It deposits buffer address and consumer's vernier serial number obtains the queue element (QE) address of data to be read；
Data are read from the corresponding queue element (QE) in queue element (QE) address of the data to be read；
Updated consumer's vernier serial number is used as after consumer's vernier serial number is increased setting value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910253786.0A CN110134439B (en) | 2019-03-30 | 2019-03-30 | Lock-free data structure construction method and data writing and reading methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910253786.0A CN110134439B (en) | 2019-03-30 | 2019-03-30 | Lock-free data structure construction method and data writing and reading methods |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110134439A true CN110134439A (en) | 2019-08-16 |
| CN110134439B CN110134439B (en) | 2021-09-28 |
Family
ID=67568999
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910253786.0A Active CN110134439B (en) | 2019-03-30 | 2019-03-30 | Lock-free data structure construction method and data writing and reading methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110134439B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110888727A (en) * | 2019-11-26 | 2020-03-17 | 北京达佳互联信息技术有限公司 | Method, device and storage medium for realizing concurrent lock-free queue |
| CN111143236A (en) * | 2019-12-07 | 2020-05-12 | 杭州安恒信息技术股份有限公司 | Memory mapping implementation queue and data reading and writing method thereof |
| CN111552575A (en) * | 2019-12-31 | 2020-08-18 | 远景智能国际私人投资有限公司 | Message queue-based message consumption method, device and equipment |
| CN112148488A (en) * | 2020-09-22 | 2020-12-29 | 杭州电魂网络科技股份有限公司 | Message processing method and system based on multi-cycle cache |
| CN112597249A (en) * | 2020-12-26 | 2021-04-02 | 湖南快乐阳光互动娱乐传媒有限公司 | Synchronous distribution and storage method and system for service data |
| CN113176896A (en) * | 2021-03-19 | 2021-07-27 | 中盈优创资讯科技有限公司 | Method for randomly taking out object based on single-in single-out lock-free queue |
| CN113377509A (en) * | 2021-06-08 | 2021-09-10 | 上海哔哩哔哩科技有限公司 | Data processing method and system |
| CN113377295A (en) * | 2021-08-13 | 2021-09-10 | 北京华云安信息技术有限公司 | Data storage and reading method, device and equipment for multi-producer single-consumer |
| WO2023087975A1 (en) * | 2021-11-18 | 2023-05-25 | International Business Machines Corporation | Lock-free data aggregation on distributed systems |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110145835A1 (en) * | 2009-12-14 | 2011-06-16 | Verisign, Inc. | Lockless Queues |
| CN103391256A (en) * | 2013-07-25 | 2013-11-13 | 武汉邮电科学研究院 | Base station user plane data processing and optimizing method based on Linux system |
| CN104639596A (en) * | 2013-11-08 | 2015-05-20 | 塔塔咨询服务有限公司 | System and method for multiple sender support in low latency fifo messaging using rdma |
| CN104954351A (en) * | 2014-05-23 | 2015-09-30 | 腾讯科技（深圳）有限公司 | Data detection method and apparatus |
| CN105868031A (en) * | 2016-03-24 | 2016-08-17 | 车智互联（北京）科技有限公司 | A data transmission device and method |
-
2019
- 2019-03-30 CN CN201910253786.0A patent/CN110134439B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110145835A1 (en) * | 2009-12-14 | 2011-06-16 | Verisign, Inc. | Lockless Queues |
| CN103391256A (en) * | 2013-07-25 | 2013-11-13 | 武汉邮电科学研究院 | Base station user plane data processing and optimizing method based on Linux system |
| CN104639596A (en) * | 2013-11-08 | 2015-05-20 | 塔塔咨询服务有限公司 | System and method for multiple sender support in low latency fifo messaging using rdma |
| CN104954351A (en) * | 2014-05-23 | 2015-09-30 | 腾讯科技（深圳）有限公司 | Data detection method and apparatus |
| CN105868031A (en) * | 2016-03-24 | 2016-08-17 | 车智互联（北京）科技有限公司 | A data transmission device and method |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110888727A (en) * | 2019-11-26 | 2020-03-17 | 北京达佳互联信息技术有限公司 | Method, device and storage medium for realizing concurrent lock-free queue |
| CN110888727B (en) * | 2019-11-26 | 2022-07-22 | 北京达佳互联信息技术有限公司 | Method, device and storage medium for realizing concurrent lock-free queue |
| CN111143236A (en) * | 2019-12-07 | 2020-05-12 | 杭州安恒信息技术股份有限公司 | Memory mapping implementation queue and data reading and writing method thereof |
| CN111552575A (en) * | 2019-12-31 | 2020-08-18 | 远景智能国际私人投资有限公司 | Message queue-based message consumption method, device and equipment |
| CN111552575B (en) * | 2019-12-31 | 2023-09-12 | 远景智能国际私人投资有限公司 | Message consumption method, device and equipment based on message queue |
| CN112148488A (en) * | 2020-09-22 | 2020-12-29 | 杭州电魂网络科技股份有限公司 | Message processing method and system based on multi-cycle cache |
| CN112597249A (en) * | 2020-12-26 | 2021-04-02 | 湖南快乐阳光互动娱乐传媒有限公司 | Synchronous distribution and storage method and system for service data |
| CN113176896A (en) * | 2021-03-19 | 2021-07-27 | 中盈优创资讯科技有限公司 | Method for randomly taking out object based on single-in single-out lock-free queue |
| CN113377509A (en) * | 2021-06-08 | 2021-09-10 | 上海哔哩哔哩科技有限公司 | Data processing method and system |
| CN113377295A (en) * | 2021-08-13 | 2021-09-10 | 北京华云安信息技术有限公司 | Data storage and reading method, device and equipment for multi-producer single-consumer |
| CN113377295B (en) * | 2021-08-13 | 2021-12-10 | 北京华云安信息技术有限公司 | Data storage and reading method, device and equipment for multi-producer single-consumer |
| WO2023087975A1 (en) * | 2021-11-18 | 2023-05-25 | International Business Machines Corporation | Lock-free data aggregation on distributed systems |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110134439B (en) | 2021-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110134439A (en) | The method of method for constructing data structure and write-in data, reading data without lockization | |
| EP2240859B1 (en) | A multi-reader, multi-writer lock-free ring buffer | |
| Attiya et al. | Sequential consistency versus linearizability | |
| US8607249B2 (en) | System and method for efficient concurrent queue implementation | |
| EP1271314B1 (en) | Sleep queue management for operating systems | |
| US20170212781A1 (en) | Parallel execution of blockchain transactions | |
| Henzinger et al. | Quantitative relaxation of concurrent data structures | |
| JP5488697B2 (en) | Multi-core processor system, synchronization control method, and synchronization control program | |
| US8806168B2 (en) | Producer-consumer data transfer using piecewise circular queue | |
| CN103218176A (en) | Data processing method and device | |
| WO2021114665A1 (en) | Quantum chip controller, quantum computation processing system, and electronic device | |
| US20210303551A1 (en) | State-based queue protocol | |
| CN110286856A (en) | Roll up cloning process, device, electronic equipment and machine readable storage medium | |
| JP6944107B2 (en) | Information processing equipment, information processing systems and programs | |
| CN109033365A (en) | A kind of data processing method and relevant device | |
| US20110145515A1 (en) | Method for modifying a shared data queue and processor configured to implement same | |
| Anderson et al. | Wait-free object-sharing schemes for real-time uniprocessors and multiprocessors | |
| CN104572483B (en) | Dynamic memory management device and method | |
| Aboulenein et al. | Hardware support for synchronization in the Scalable Coherent Interface (SCI) | |
| JP2006503361A (en) | Data processing apparatus and method for synchronizing at least two processing means in data processing apparatus | |
| KR20080025652A (en) | Demand-based processing resource allocation | |
| JP6891596B2 (en) | Processor | |
| CN105677249B (en) | The division methods of data block, apparatus and system | |
| JPWO2019008715A1 (en) | Data load program, data load method and data load device | |
| CN110337637B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |