CN105138410A - Message queue achievement method and device based on disk buffer - Google Patents
Message queue achievement method and device based on disk buffer Download PDFInfo
- Publication number
- CN105138410A CN105138410A CN201510547996.2A CN201510547996A CN105138410A CN 105138410 A CN105138410 A CN 105138410A CN 201510547996 A CN201510547996 A CN 201510547996A CN 105138410 A CN105138410 A CN 105138410A
- Authority
- CN
- China
- Prior art keywords
- message
- disk file
- read
- write
- size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a message queue achievement method and device based on a disk buffer. The method includes the steps that a monitor thread is established so as to monitor the amount of messages in a message queue periodically according to a preset periodic value; when the amount of messages reaches the preset upper limit value of messages, messages to be received are converted to binary sequence strings, the binary sequence strings are sequentially written into a predetermined magnetic disk file, and the amount of messages written into the magnetic disk file and the deviation position of the written messages are recorded; when the amount of messages reaches the preset lower limit value of the messages, the messages are read from the predetermined magnetic disk file in sequence, the messages are written into message queue, and the amount of read messages and the deviation position of the read messages are recorded. By means of the method, the problem of integrity of a distributed system when multiple mainframes transmit messages quickly is solved, and the effect that the message transmitting speed is guaranteed while message integrity is guaranteed as well is achieved.
Description
Technical field
The embodiment of the present invention relates to data transmission technology, particularly relates to a kind of implementation method and device of the message queue based on disk buffering.
Background technology
Along with the fast development of internet, the scale of distributed system is increasing, and range of application is also more and more wider.During the larger distributed system formed using multiple main frame (can be terminal), the reliable and stable transmission carrying out message between multiple main frame realizes often by message queue.
At present, the message in the distributed system of multiple main frame composition between each main frame is transmitted by message queue.Such as: host A, host B and host C composition distributed system, the message that host A will be sent to host B is written in the message queue a created in host A, and host B obtains the message that host A sends from message queue a.If host B needs to be sent to host C after processing described message, then in host B, create message queue b, by the message queue b of the message write host B after process, host C obtains the message that host B sends from message queue b.
Only use the storage space of the message queue of internal memory to be limited, when a large amount of message is passed to queue continuously, likely occur that full situation is write in queue.When message queue is write full time, may occur abandoning new data, abandon the situation of old data or obstruction new data, and the appearance of above-mentioned situation must affect the integrality of message.If message queue is not in internal memory, then the speed of Message Transmission then can be comparatively slow, can not ensure the ageing of Message Transmission.
Summary of the invention
The invention provides a kind of implementation method and device of the message queue based on disk buffering, to realize ensureing message integrity while guarantee Message Transmission speed.
First aspect, embodiments provides a kind of implementation method of the message queue based on disk buffering, comprising:
Create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message;
When described size of message reaches default message higher limit, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message;
When described size of message reaches default message lower limit, from described predetermined disk file, order reads message, and described message is write described message queue, the deviation post of the size of message that record reads and reading message.
Second aspect, the embodiment of the present invention additionally provides a kind of implement device of the message queue based on disk buffering, and this device comprises:
Message monitoring unit, for create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message;
Disk writing unit, during for reaching default message higher limit at described size of message, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message;
Message writing unit, during for reaching default message lower limit at described size of message, from described predetermined disk file, order reads message, and described message is write described message queue, the deviation post of the size of message that record reads and reading message.
The present invention by create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message, the message conversion received after described size of message being reached default message higher limit is binary sequence string, described binary sequence string is sequentially written in predetermined disk file, when described size of message reaches default message lower limit, from described predetermined disk file, order reads message, described message is write described message queue, ensure that the message buffering received after described message queue is write completely is in disk file, when size of message in described message queue is less than default lower limit, the order message read in above-mentioned predetermined disk file writes described message queue, and there will not be and abandon old data, abandon the situation of new data or obstruction new data, solve the integrity issue between multiple main frame during quick pass-along message in distributed system, realize the effect ensureing message integrity while ensureing Message Transmission speed.
Accompanying drawing explanation
Fig. 1 a is the process flow diagram of the implementation method of the message queue based on disk buffering in the embodiment of the present invention one;
Fig. 1 b is the implementation procedure schematic diagram of the message queue based on disk buffering in the embodiment of the present invention one;
Fig. 2 is the process flow diagram writing the method for disk file in the implementation method of the message queue based on disk buffering in the embodiment of the present invention two;
Fig. 3 is the process flow diagram writing the method for message queue in the implementation method of the message queue based on disk buffering in the embodiment of the present invention three;
Fig. 4 is the structural representation of the implement device of the message queue based on disk buffering in the embodiment of the present invention four.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in further detail.Be understandable that, specific embodiment described herein is only for explaining the present invention, but not limitation of the invention.It also should be noted that, for convenience of description, illustrate only part related to the present invention in accompanying drawing but not entire infrastructure.
Embodiment one
The process flow diagram of the implementation method of a kind of message queue based on disk buffering that Fig. 1 a provides for the embodiment of the present invention one, the present embodiment is applicable between many main frames and carries out reliably, the situation of stable Message Transmission, the method can be performed by the implement device of the message queue based on disk buffering, this device is configured in the storage medium that each main frame above-mentioned can read, described storage medium can fixed configurations among described main frame, described storage medium can also be mobile memory medium, such as, described storage medium can comprise ROM/RAM, disk or CD etc.Described method specifically comprises the steps:
Step 110, create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message.
Create a monitoring thread periodically to scan described message queue according to the periodic quantity preset, obtain the size of message that described message queue is current.When receiving the message writing described message queue, before described message is write described message queue, inquire about the size of message that described message queue is current.Shown in Fig. 1 b, if described current size of message does not reach the higher limit of default described message queue, then described message is write described message queue.If described current size of message reaches default higher limit, then described message is write in predetermined disk file.Wherein, described higher limit is the maximum amount of message that described message queue can store, and accordingly, lower limit is the minimum message amount that described message queue stores.For the message queue that Linux system carries, message manages according to the pattern of first in first out, and the maximum support size of message of message queue is 600, and single message is maximum is no more than 8KB.According to the needs of practical application, higher limit and the lower limit of described message queue can be pre-set.
Step 120, when described size of message reaches default message higher limit, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message.
What terminal obtained that user pre-sets writes disk file index, described in write disk file index and comprise write pointer corresponding to the current position writing disk file name, the size of message of current write and current write message.After program is restarted, message can be write fast from the assigned address of the disk file of specifying by described written document index, and count from the size of message of the current write in disk file.Terminal according to described in write in disk file index current writes disk file name and open and write disk file accordingly.The described size of message writing the current write of disk index is compared with the maximum amount of message of the disk file preset, if described in write the size of message write in disk file and reach maximum amount of message, disk file is write described in then closing, create a new disk file and write disk file as new, described disk file of writing newly is automatically named according to the rule that user is arranged or points out user-in file name.Such as, can be the name of described disk file according to system timestamp when creating described disk file, be specifically as follows the decimal system timestamp being accurate to Millisecond when creating file, can called after 1438242860243.mdt.Terminal writes disk file described in opening, and the described current disk file name of writing write in disk file index is updated to the new filename writing disk file, the size of message of current write is set to 0, current write pointer is set to 0.
After described size of message reaches default message higher limit, if terminal receives message again, then described message sequence is turned to binary sequence string.According to the length of described binary sequence string according to being written in disk file from small to large, more described binary sequence string is write disk file in order.Such design eliminates the step of resolving message, improves the execution efficiency of write message.At every turn to writing disk file write a piece of news, the size of message of current write is added 1, and the write pointer that the position of the current write message of corresponding change is corresponding.
Step 130, when described size of message reaches default message lower limit, from described predetermined disk file, order reads message, and described message is write described message queue, the size of message that record reads and read the deviation post of message.
What terminal obtained that user pre-sets reads disk file index, described in read disk file index and comprise current disk file name to be read, the size of message of the read pointer that the position of current reading message is corresponding and current reading.After program is restarted, can carry out message reading fast from the assigned address of the disk file of specifying by described file index of reading, the message count through reading can count from the size of message of the current reading of reading to record in disk file index.
Describedly read disk file index or disk file may not exist, disk file to be read can be determined according to following method.Such as:
When reading disk file index and disk file to be read all exists, open disk file to be read according to the described disk file name to be read reading disk file index.
When reading disk file index and do not exist and disk file to be read existing, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero.
When disk file to be read does not exist, described monitoring thread is slept the time of presetting.
If disk file to be read exists, the described disk file to be read of described monitoring thread inquiry is to determine whether there is the buffered message do not read in described disk file.Concrete grammar is as follows:
Judge the filename of disk file to be read and whether mate according to writing the filename writing disk file that disk file index determines;
If the filename of described disk file to be read mates with the described filename writing disk file, then the size of message of more current reading and the described size of message writing the current write of disk file;
When writing the size of message of current write of disk file described in the size of message of current reading is less than, determine in described disk file, to there is the buffered message do not read;
When writing the size of message of current write of disk file described in the size of message of current reading is more than or equal to, determine in described disk file, to there is not the buffered message do not read.
If the filename of described disk file to be read does not mate with the described filename writing disk file, then judge whether the size of message in described disk file to be read reaches default maximum amount of message;
When size of message in described disk file to be read reaches described maximum amount of message, illustrate that the message that this disk file stores reads all, close and delete described disk file to be read, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero.
When size of message in described disk file to be read does not reach described maximum amount of message, determine to there is the buffered message do not read in described disk file to be read.
If there is not the buffered message do not read in described disk file, then described monitoring thread is made to sleep the time of presetting.
As shown in Figure 1 b, can should be specifically based on the implementation procedure of the message queue of disk buffering:
The r/w cell of terminal receives the next message of outside transmission, when the size of message of described message queue does not reach higher limit, by described message write message queue.When the size of message of described message queue reaches higher limit, by described message write disk file.Read the read operation that unit realizes message queue, described in before reading, read whether have message in message queue described in unit judges.If have message in described message queue, then after reading message success, from described message queue, delete this message that success is read.If there is no message in described message queue, then read unit and return a null pointer.Described message queue manages according to the pattern of first in first out, from the head of described message queue, read message, inserts new message from the afterbody of described message queue.Message monitoring unit is according to the described message queue of monitoring of the cycle preset, when the detailed amount monitoring described message queue is lower than the lower limit preset, the buffered message whether having in disk file and do not read is checked by monitoring thread, if had, then read buffered message in message queue by message writing unit, if proceed monitoring after the time span no, then making the sleep of described monitoring thread preset.
The technical scheme of the present embodiment, by create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message, the message conversion received after described size of message being reached default message higher limit is binary sequence string, described binary sequence string is sequentially written in predetermined disk file, when described size of message reaches default message lower limit, from described predetermined disk file, order reads message, described message is write described message queue, ensure that the message buffering received after described message queue is write completely is in disk file, when size of message in described message queue is less than default lower limit, the order message read in above-mentioned predetermined disk file writes described message queue, and there will not be and abandon old data, abandon the situation of new data or obstruction new data, solve the integrity issue between multiple main frame during quick pass-along message in distributed system, reach the effect ensureing message integrity while ensureing Message Transmission speed.
Embodiment two
Fig. 2 is the process flow diagram writing the method for disk file in the implementation method of the message queue based on disk buffering in the embodiment of the present invention two.The technical scheme of the present embodiment, on the basis of technical scheme disclosed in above-described embodiment, enters the method writing disk file be described in detail to writing message further.
The implementation method of the described message queue based on disk buffering, comprising:
Step 201, terminal receives message.
Whether step 202, the message judged in message queue by r/w cell reach higher limit.If reach higher limit, then perform step 204, if do not reach higher limit, then perform step 203.
Step 203, the message write message queue will received by described r/w cell.
Step 204, judge whether disk file to be written is opened by described r/w cell, if disk file to be written is opened, then perform step 205, if disk file to be written is not opened, then perform step 206.
Step 205, judge whether the size of message write in disk file reaches default maximum amount of message by described r/w cell, if the size of message write in disk file reaches default maximum amount of message, then perform step 209, if the size of message write in disk file does not reach default maximum amount of message, then perform step 210.
Whether step 206, terminal judges are write disk index and are existed, if exist, then perform step 207, if do not exist, perform step 208.
If user does not pre-set write disk index file, when opening disk file first, described in write disk file index and may not exist, need to perform step 208 and generate one and write disk index.
Step 207, terminal read the disk file information to be written write in disk file index and written information amount, open disk file to be written accordingly.
Step 208, terminal create and open a new disk file, and arranging and having write size of message is zero.
Terminal creates a new disk file as writing disk file, described in write disk file the filename decimal system timestamp that is accurate to Millisecond when creating file according to user determine.Manual or the terminal of user automatically generates according to the disk file created and writes disk file index, current disk file name of writing is updated to the new filename writing disk file, the size of message of current write is set to 0, current write pointer is set to 0.
Step 209, terminal close disk file, and create and open a new disk file, upgrading written information number is zero.
If described in write the size of message write in disk file and reach maximum amount of message, disk file is write described in then closing, create a new disk file by terminal and write disk file as new, described in write disk file the filename decimal system timestamp that is accurate to Millisecond when creating file according to user determine.Manual or the terminal of user upgrades automatically writes disk file index, replaces and originally currently writes disk file name, the size of message of current write is replaced with 0, current write pointer is replaced with 0 with the new filename writing disk file.
Step 210, by r/w cell, the message of reception is write in new disk file, written information number is added 1.
By r/w cell, the message received being write in new disk file, at every turn to writing disk file write a piece of news, the size of message of current write being added 1, and the write pointer that the position of the current write message of corresponding change is corresponding.
Step 211, terminal upgrade writes disk file index.
Upgrade according to write pointer corresponding to the size of message of current write and the position of current write message and write disk file index.
The technical scheme of the present embodiment, the message write received after the size of message of message queue being reached default message higher limit by r/w cell is predetermined to be write in disk file, avoid writing completely the situation occurring abandoning old data, abandon new data or obstruction new data at described message queue, ensured the integrality that data are transmitted.
Embodiment three
Fig. 3 is the process flow diagram writing the method for message queue in the implementation method of the message queue based on disk buffering in the embodiment of the present invention three.The technical scheme of the present embodiment, on the basis of technical scheme disclosed in above-described embodiment, is described in detail to the method reading message write message queue from disk file further.
The implementation method of the described message queue based on disk buffering, comprising:
Step 301, by the size of message in the queue of monitoring thread monitoring message.
Whether step 302, the size of message judged in message queue by monitoring thread reach lower limit, if described size of message reaches lower limit, then perform step 303, if described size of message does not reach lower limit, then perform step 313.
Whether step 303, terminal judges disk file to be read is opened, if disk file to be read is opened, then performs step 308, if disk file to be read is not opened, then performs step 304.
Whether step 304, terminal judges are read disk file index and are existed, if described in read disk index and exist, then perform step 306, if described in read disk index and do not exist, then perform step 305.
Whether step 305, terminal judges disk file to be read exists, if described disk file to be read exists, then performs step 307, if described disk file to be read does not exist, then performs step 313.
Step 306, terminal read disk file index, obtain disk file name to be read, read pointer, the quantity of information that reads, open disk file to be read.
Terminal obtains disk file name to be read according to reading disk file index, opens disk file to be read according to described disk file name.
Step 307, terminal are chosen message write disk file formerly and are opened, and upgrading read pointer is 0, and the quantity of information of reading is 0.
If read disk index not exist but disk file to be read existence, then choose message write disk file formerly.Because when message write disk file, the write of described message performs in order, and after writing a full disk, just another disk newly-built carries out message write, and so, first writing the message that the disk of message stores is the message being first passed to terminal.Open the disk file chosen, read pointer corresponding for the position of current reading message is set to 0, the size of message of current reading is set to 0.
Whether step 308, terminal judges disk file to be read is identical with the filename writing disk file, if filename is identical, then performs step 310, if filename is not identical, then performs step 309.
Step 309, message writing unit judge whether the size of message read reaches default maximum amount of message, then perform step 311, if the size of message read does not reach default maximum amount of message, then perform step 312.
If the size of message read reaches default maximum amount of message, then illustrate that the message stored in this disk file has all read, then perform step 311, if the size of message read does not reach default maximum amount of message, the buffered message do not read in addition in this disk file is then described, then performs step 312.
Step 310, message writing unit judge whether the size of message of current reading is less than the size of message of the current write writing disk file, if the size of message of current reading is less than the size of message of the current write writing disk file, then perform step 312, if the size of message of current reading is not less than the size of message of the current write writing disk file, then perform step 313.
If the size of message of current reading is less than the size of message of the current write writing disk file, the buffered message do not read in addition in this disk file is then described, then perform step 312, if the size of message of current reading equals the size of message of the current write writing disk file, illustrate that the message write in this disk reads all, then perform step 313.
Step 311, terminal are closed and are deleted current disk file to be read, and choose message write disk file formerly and open, upgrading read pointer is 0, and the Information Number of reading is 0.
When the size of message read in current disk file to be read reaches default maximum amount of message, illustrate that the message in current disk file to be read has read, close current disk file to be read, buttoned-up current disk file to be read is deleted.Choose message write disk file formerly to open, read pointer corresponding for the position of current reading message is set to 0, the size of message of current reading is set to 0.
Step 312, message writing unit read an information write message queue, the size of message upgrading read pointer and read.
Read a piece of news from the position of the read pointer disk file to be read, through described message write message queue, the described read pointer of corresponding amendment points to the next memory location of current location, and the size of message read is added 1.
The time that the sleep of step 313, monitoring thread is preset.
The technical scheme of the present embodiment, when size of message in described message queue is less than default lower limit, described message queue is write by the message writing unit order message read in disk file to be read, the speed of Message Transmission between the main frame that can be ensured multiple terminals by message queue pass-along message, realizes the effect ensureing message integrity while ensureing Message Transmission speed.
Embodiment four
Fig. 4 is the structural representation of the implement device of the message queue based on disk buffering in the embodiment of the present invention four.Described device comprises:
Message monitoring unit 410, for create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message;
Disk writing unit 420, during for reaching default message higher limit at described size of message, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message;
Message writing unit 430, during for reaching default message lower limit at described size of message, from described predetermined disk file, order reads message, and described message is write described message queue, the deviation post of the size of message that record reads and reading message.
The present invention by message monitoring unit 410 create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message, the message conversion received after described size of message being reached default message higher limit by disk writing unit 420 is binary sequence string, described binary sequence string is sequentially written in predetermined disk file, when described size of message reaches default message lower limit, message is read by message writing unit 430 order from described predetermined disk file, described message is write described message queue, ensure that the message buffering received after described message queue is write completely is in disk file, when size of message in described message queue is less than default lower limit, the order message read in above-mentioned predetermined disk file writes described message queue, and there will not be and abandon old data, abandon the situation of new data or obstruction new data, solve the integrity issue between multiple main frame during quick pass-along message in distributed system, realize the effect ensureing message integrity while ensureing Message Transmission speed.
Further, described device also comprises:
Write disk file index acquiring unit, for before described binary sequence string is sequentially written in predetermined disk file, what acquisition pre-set writes disk file index, wherein, write disk file index described in and comprise write pointer corresponding to the current position writing disk file name, the size of message of current write and current write message;
Disk file opens unit, opens write disk file for writing disk file index described in basis;
Disk file creating unit, if for described in write the maximum amount of message that the size of message write in disk file reaches default, write disk file described in then closing, create and open a disk file and write disk file as new, described in renewal, write disk file index.
Further, described device also comprises:
Buffered message query unit, during for reaching default message lower limit at described size of message, before from described predetermined disk file, order reads message, described monitoring thread inquires about predetermined disk file to determine whether there is the buffered message do not read in described disk file;
If there is not the buffered message do not read in described disk file, then described monitoring thread is made to sleep the time of presetting.
Further, described device also comprises:
Disk file determining unit to be read, predetermined disk file is inquired about with before determining whether to exist in described disk file the buffered message that do not read for described monitoring thread, when reading disk file index and disk file to be read all exists, disk file to be read is determined according to described disk file index of reading, wherein, described disk file index of reading comprises current disk file name to be read, the size of message of the read pointer that the position of current reading message is corresponding and current reading;
When reading disk file index and do not exist and disk file to be read existing, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When disk file to be read does not exist, described monitoring thread is slept the time of presetting.
Further, described buffered message query unit specifically for:
Judge the filename of disk file to be read and whether mate according to writing the filename writing disk file that disk file index determines;
If the filename of described disk file to be read mates with the described filename writing disk file, then the size of message of more current reading and the described size of message writing the current write of disk file;
When writing the size of message of current write of disk file described in the size of message of current reading is less than, determine in described disk file, to there is the buffered message do not read;
When writing the size of message of current write of disk file described in the size of message of current reading is more than or equal to, determine in described disk file, to there is not the buffered message do not read, described monitoring thread is slept the time of presetting;
If the filename of described disk file to be read does not mate with the described filename writing disk file, then judge whether the size of message in described disk file to be read reaches default maximum amount of message;
When size of message in described disk file to be read reaches described maximum amount of message, close and delete described disk file to be read, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When size of message in described disk file to be read does not reach described maximum amount of message, determine to there is the buffered message do not read in described disk file to be read.
The implement device of the above-mentioned message queue based on disk buffering can perform the implementation method of the message queue based on disk buffering that any embodiment of the present invention provides, and possesses the corresponding functional module of manner of execution and beneficial effect.
Note, above are only preferred embodiment of the present invention and institute's application technology principle.Skilled person in the art will appreciate that and the invention is not restricted to specific embodiment described here, various obvious change can be carried out for a person skilled in the art, readjust and substitute and can not protection scope of the present invention be departed from.Therefore, although be described in further detail invention has been by above embodiment, the present invention is not limited only to above embodiment, when not departing from the present invention's design, can also comprise other Equivalent embodiments more, and scope of the present invention is determined by appended right.
Claims (10)
1. based on an implementation method for the message queue of disk buffering, it is characterized in that, comprising:
Create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message;
When described size of message reaches default message higher limit, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message;
When described size of message reaches default message lower limit, from described predetermined disk file, order reads message, and described message is write described message queue, the deviation post of the size of message that record reads and reading message.
2. method according to claim 1, is characterized in that, before described binary sequence string is sequentially written in predetermined disk file, also comprises:
Obtain pre-set write disk file index, wherein, described in write disk file index and comprise write pointer corresponding to the current position writing disk file name, the size of message of current write and current write message;
Open write disk file according to described disk file index of writing;
If described in write the maximum amount of message that the size of message write in disk file reaches default, then write disk file described in closing, create and open a disk file and write disk file as new, described in renewal, write disk file index.
3. method according to claim 1, is characterized in that, when described size of message reaches default message lower limit, before from described predetermined disk file, order reads message, also comprises:
Described monitoring thread inquires about predetermined disk file to determine whether there is the buffered message do not read in described disk file;
If there is not the buffered message do not read in described disk file, then described monitoring thread is made to sleep the time of presetting.
4. method according to claim 3, is characterized in that, described monitoring thread inquires about predetermined disk file with before determining whether to exist in described disk file the buffered message that do not read, also comprises:
When reading disk file index and disk file to be read all exists, disk file to be read is determined according to described disk file index of reading, wherein, described disk file index of reading comprises current disk file name to be read, the size of message of the read pointer that the position of current reading message is corresponding and current reading;
When reading disk file index and do not exist and disk file to be read existing, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When disk file to be read does not exist, described monitoring thread is slept the time of presetting.
5. method according to claim 3, is characterized in that, described monitoring thread inquires about predetermined disk file to determine whether there is the buffered message do not read in described disk file, comprising:
Judge the filename of disk file to be read and whether mate according to writing the filename writing disk file that disk file index determines;
If the filename of described disk file to be read mates with the described filename writing disk file, then the size of message of more current reading and the described size of message writing the current write of disk file;
When writing the size of message of current write of disk file described in the size of message of current reading is less than, determine in described disk file, to there is the buffered message do not read;
When writing the size of message of current write of disk file described in the size of message of current reading is more than or equal to, determine in described disk file, to there is not the buffered message do not read, described monitoring thread is slept the time of presetting;
If the filename of described disk file to be read does not mate with the described filename writing disk file, then judge whether the size of message in described disk file to be read reaches default maximum amount of message;
When size of message in described disk file to be read reaches described maximum amount of message, close and delete described disk file to be read, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When size of message in described disk file to be read does not reach described maximum amount of message, determine to there is the buffered message do not read in described disk file to be read.
6. based on an implement device for the message queue of disk buffering, it is characterized in that, comprising:
Message monitoring unit, for create monitoring thread with according to preset the periodic monitoring message queue of periodic quantity in size of message;
Disk writing unit, during for reaching default message higher limit at described size of message, be binary sequence string by the message conversion received again, described binary sequence string is sequentially written in predetermined disk file, records the size of message of the write of described disk file and the deviation post of write message;
Message writing unit, during for reaching default message lower limit at described size of message, from described predetermined disk file, order reads message, and described message is write described message queue, the deviation post of the size of message that record reads and reading message.
7. device according to claim 6, is characterized in that, described device also comprises:
Write disk file index acquiring unit, for before described binary sequence string is sequentially written in predetermined disk file, what acquisition pre-set writes disk file index, wherein, write disk file index described in and comprise write pointer corresponding to the current position writing disk file name, the size of message of current write and current write message;
Disk file opens unit, opens write disk file for writing disk file index described in basis;
Disk file creating unit, if for described in write the maximum amount of message that the size of message write in disk file reaches default, write disk file described in then closing, create and open a disk file and write disk file as new, described in renewal, write disk file index.
8. device according to claim 6, is characterized in that, described device also comprises:
Buffered message query unit, during for reaching default message lower limit at described size of message, before from described predetermined disk file, order reads message, described monitoring thread inquires about predetermined disk file to determine whether there is the buffered message do not read in described disk file;
If there is not the buffered message do not read in described disk file, then described monitoring thread is made to sleep the time of presetting.
9. device according to claim 8, is characterized in that, described device also comprises:
Disk file determining unit to be read, predetermined disk file is inquired about with before determining whether to exist in described disk file the buffered message that do not read for described monitoring thread, when reading disk file index and disk file to be read all exists, disk file to be read is determined according to described disk file index of reading, wherein, described disk file index of reading comprises current disk file name to be read, the size of message of the read pointer that the position of current reading message is corresponding and current reading;
When reading disk file index and do not exist and disk file to be read existing, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When disk file to be read does not exist, described monitoring thread is slept the time of presetting.
10. device according to claim 8, is characterized in that, described buffered message query unit specifically for:
Judge the filename of disk file to be read and whether mate according to writing the filename writing disk file that disk file index determines;
If the filename of described disk file to be read mates with the described filename writing disk file, then the size of message of more current reading and the described size of message writing the current write of disk file;
When writing the size of message of current write of disk file described in the size of message of current reading is less than, determine in described disk file, to there is the buffered message do not read;
When writing the size of message of current write of disk file described in the size of message of current reading is more than or equal to, determine in described disk file, to there is not the buffered message do not read, described monitoring thread is slept the time of presetting;
If the filename of described disk file to be read does not mate with the described filename writing disk file, then judge whether the size of message in described disk file to be read reaches default maximum amount of message;
When size of message in described disk file to be read reaches described maximum amount of message, close and delete described disk file to be read, from remaining disk file, message write disk file is formerly determined according to the order of message write disk file, using message write disk file formerly as disk file to be read, the read pointer that the position of the current reading message of disk file described in initialization is corresponding is zero, and the size of message of current reading is zero;
When size of message in described disk file to be read does not reach described maximum amount of message, determine to there is the buffered message do not read in described disk file to be read.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510547996.2A CN105138410A (en) | 2015-08-31 | 2015-08-31 | Message queue achievement method and device based on disk buffer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510547996.2A CN105138410A (en) | 2015-08-31 | 2015-08-31 | Message queue achievement method and device based on disk buffer |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105138410A true CN105138410A (en) | 2015-12-09 |
Family
ID=54723766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510547996.2A Pending CN105138410A (en) | 2015-08-31 | 2015-08-31 | Message queue achievement method and device based on disk buffer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105138410A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528306A (en) * | 2016-11-08 | 2017-03-22 | 天津海量信息技术股份有限公司 | Message queue storage method |
| CN107590005A (en) * | 2017-08-31 | 2018-01-16 | 武汉光谷信息技术股份有限公司 | The acquisition methods and system of data in a kind of Redis memory queues |
| CN108667725A (en) * | 2018-04-25 | 2018-10-16 | 哈尔滨电气股份有限公司 | A kind of industrial AnyRouter and implementation method based on a variety of accesses and edge calculations |
| CN110460534A (en) * | 2019-07-26 | 2019-11-15 | 腾讯云计算（北京）有限责任公司 | A kind of request message report method, device, equipment and storage medium |
| CN110908798A (en) * | 2019-11-08 | 2020-03-24 | 丁剑明 | Multi-process cooperative network traffic analysis method and device |
| CN111400056A (en) * | 2019-12-31 | 2020-07-10 | 远景智能国际私人投资有限公司 | Message queue-based message transmission method, device and equipment |
| CN111538600A (en) * | 2020-02-25 | 2020-08-14 | 远景智能国际私人投资有限公司 | Message processing method and device, computer equipment and storage medium |
| CN111552575A (en) * | 2019-12-31 | 2020-08-18 | 远景智能国际私人投资有限公司 | Message queue-based message consumption method, device and equipment |
| CN111858399A (en) * | 2019-04-30 | 2020-10-30 | 贵州白山云科技股份有限公司 | Data transmission method and device, computer readable storage medium and computer equipment |
| CN112148441A (en) * | 2020-07-28 | 2020-12-29 | 易视飞科技成都有限公司 | Embedded message queue realizing method of dynamic storage mode |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050060442A1 (en) * | 2003-09-15 | 2005-03-17 | Intel Corporation | Method, system, and program for managing data transmission through a network |
| CN101145968A (en) * | 2007-08-02 | 2008-03-19 | 中兴通讯股份有限公司 | Data transmission and receiving method between network management system and transmission device |
| CN101754169A (en) * | 2008-12-02 | 2010-06-23 | 中兴通讯股份有限公司 | Service management method and system for received instructions of home location register |
| CN102223681A (en) * | 2010-04-19 | 2011-10-19 | 中兴通讯股份有限公司 | M2M system and cache control method therein |
| CN102231164A (en) * | 2011-07-06 | 2011-11-02 | 华中科技大学 | Extensible makeup language (XML) incremental transmission and interaction method for multidisciplinary virtual experiment platform |
| CN104102552A (en) * | 2013-04-15 | 2014-10-15 | 深圳中兴网信科技有限公司 | Message processing method and device |
-
2015
- 2015-08-31 CN CN201510547996.2A patent/CN105138410A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050060442A1 (en) * | 2003-09-15 | 2005-03-17 | Intel Corporation | Method, system, and program for managing data transmission through a network |
| CN101145968A (en) * | 2007-08-02 | 2008-03-19 | 中兴通讯股份有限公司 | Data transmission and receiving method between network management system and transmission device |
| CN101754169A (en) * | 2008-12-02 | 2010-06-23 | 中兴通讯股份有限公司 | Service management method and system for received instructions of home location register |
| CN102223681A (en) * | 2010-04-19 | 2011-10-19 | 中兴通讯股份有限公司 | M2M system and cache control method therein |
| CN102231164A (en) * | 2011-07-06 | 2011-11-02 | 华中科技大学 | Extensible makeup language (XML) incremental transmission and interaction method for multidisciplinary virtual experiment platform |
| CN104102552A (en) * | 2013-04-15 | 2014-10-15 | 深圳中兴网信科技有限公司 | Message processing method and device |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106528306A (en) * | 2016-11-08 | 2017-03-22 | 天津海量信息技术股份有限公司 | Message queue storage method |
| CN107590005A (en) * | 2017-08-31 | 2018-01-16 | 武汉光谷信息技术股份有限公司 | The acquisition methods and system of data in a kind of Redis memory queues |
| CN108667725A (en) * | 2018-04-25 | 2018-10-16 | 哈尔滨电气股份有限公司 | A kind of industrial AnyRouter and implementation method based on a variety of accesses and edge calculations |
| CN111858399B (en) * | 2019-04-30 | 2023-11-24 | 贵州白山云科技股份有限公司 | Data transmission method, device, computer readable storage medium and computer equipment |
| CN111858399A (en) * | 2019-04-30 | 2020-10-30 | 贵州白山云科技股份有限公司 | Data transmission method and device, computer readable storage medium and computer equipment |
| CN110460534A (en) * | 2019-07-26 | 2019-11-15 | 腾讯云计算（北京）有限责任公司 | A kind of request message report method, device, equipment and storage medium |
| CN110460534B (en) * | 2019-07-26 | 2024-05-14 | 腾讯云计算（北京）有限责任公司 | Method, device, equipment and storage medium for reporting request message |
| CN110908798A (en) * | 2019-11-08 | 2020-03-24 | 丁剑明 | Multi-process cooperative network traffic analysis method and device |
| CN111400056A (en) * | 2019-12-31 | 2020-07-10 | 远景智能国际私人投资有限公司 | Message queue-based message transmission method, device and equipment |
| CN111552575A (en) * | 2019-12-31 | 2020-08-18 | 远景智能国际私人投资有限公司 | Message queue-based message consumption method, device and equipment |
| CN111552575B (en) * | 2019-12-31 | 2023-09-12 | 远景智能国际私人投资有限公司 | Message consumption method, device and equipment based on message queue |
| CN111538600A (en) * | 2020-02-25 | 2020-08-14 | 远景智能国际私人投资有限公司 | Message processing method and device, computer equipment and storage medium |
| CN111538600B (en) * | 2020-02-25 | 2023-09-12 | 远景智能国际私人投资有限公司 | Message processing method, device, computer equipment and storage medium |
| CN112148441A (en) * | 2020-07-28 | 2020-12-29 | 易视飞科技成都有限公司 | Embedded message queue realizing method of dynamic storage mode |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105138410A (en) | Message queue achievement method and device based on disk buffer | |
| US11307769B2 (en) | Data storage method, apparatus and storage medium | |
| EP2898430B1 (en) | Mail indexing and searching using hierarchical caches | |
| US20090113130A1 (en) | System and method for updating dirty data of designated raw device | |
| CN107851122B (en) | Large scale storage and retrieval of data with well-bounded life | |
| CN106951375B (en) | Method and device for deleting snapshot volume in storage system | |
| CN106021445A (en) | Cached data loading method and apparatus | |
| US11507277B2 (en) | Key value store using progress verification | |
| US10120791B2 (en) | Data read apparatus, data read method, and storage medium storing data read program | |
| CN109448777A (en) | A kind of test method and device of NVME solid state hard disk | |
| CN113312008B (en) | Processing method, system, equipment and medium for file read-write service | |
| US9588884B2 (en) | Systems and methods for in-place reorganization of device storage | |
| CN108280215B (en) | Hybrid updating method of E-commerce index file based on Solr | |
| CN108829345B (en) | Data processing method of log file and terminal equipment | |
| CN104615459A (en) | MoCA equipment parameter configuration method and device | |
| CN107430546A (en) | A kind of file updating method and storage device | |
| KR101676175B1 (en) | Apparatus and method for memory storage to protect data-loss after power loss | |
| CN104252423A (en) | Consistency processing method and device based on multi-core processor | |
| CN105446652A (en) | Method and device for storing camera shooting data | |
| CN110866068A (en) | Announcement data storage method and device based on HDFS | |
| CN110941595A (en) | File system access method and device | |
| CN110502258A (en) | A kind of method, equipment and readable medium upgrading network interface card | |
| JP6097054B2 (en) | Control system and control method | |
| CN109582685B (en) | Method and device for recording context request execution process | |
| CN108776578B (en) | Method and system for rapidly merging objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151209 |