CN101686209B - Method and device for storing message in message retransmission system - Google Patents
Method and device for storing message in message retransmission system Download PDFInfo
- Publication number
- CN101686209B CN101686209B CN 200810166811 CN200810166811A CN101686209B CN 101686209 B CN101686209 B CN 101686209B CN 200810166811 CN200810166811 CN 200810166811 CN 200810166811 A CN200810166811 A CN 200810166811A CN 101686209 B CN101686209 B CN 101686209B
- Authority
- CN
- China
- Prior art keywords
- message
- data file
- current data
- daily record
- index daily
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method and a device for storing a message in a message retransmission system. The method comprises the following steps: sequentially adding the message to a current data file; generating a first index journal of the message; and storing the first index journal into a memory. The invention has the difference from the prior art that in the embodiment of the invention, the index of the message is stored into the memory so that the content of the message can be acquired from the data file of a hard disk and directly from the index of the memory according to message ID when the message is read, thereby the random reading and writing frequency of the hard disk and the random reading and writing frequency of files of the hard disk are reduced, and the file performance is enhanced; and in addition, the invention can further recover the index of the memory according to the stored journal files when a system is restarted.
Description
Technical field
The present invention relates to the network data exchange field, particularly relate to the method and apparatus of storing message in the message repeater system.
Background technology
Along with popularizing of the Internet, Email (Email), Java messenger service (Java MessagingService, JMS) etc. the asynchronous communication instrument also begins to popularize, the asynchronous communication instrument generally adopts the message repeater system storing message, therefore, also more and more higher to the requirement of asynchronous communication message in the message repeater system.Generally speaking, the process of asynchronous communication message is in the message repeater system: at first transmit leg sends a message to message repeater system, message repeater system is stored message, and by scheduling, message is sent to the recipient, if the recipient takes defeat, message repeater system starts retransmission process, after message sent successfully, message repeater system was deleted the message of described storage.As can be seen, message repeater system important link when how storing message is asynchronous communication.
In the prior art, message repeater system has the method for following several storing messages:
(1) realization of JMS, JMS is the technical specification of relevant Message Oriented Middleware on the Java platform, is a kind of java application interface, the service that establishment can be provided, send, receive, read message.The implementation method of JMS adopts database as the medium of storing message.Concrete storing message flow process is: receiving thread receives the message that transmit leg sends, and described message is saved in the database, wakes the transmission thread simultaneously up, successfully responds to return to the sender; Then, the transmission thread obtains the message that does not also send from database and sends to message recipient, sends successfully back this message of deletion from database.The shortcoming of this method is, if size of message is bigger, message repeater system just needs the message of storage larger data amount, cause performance of database seriously to reduce, and message is stored or therefrom obtained to message repeater system by socket (socket) to database, also can produce the performance loss in unnecessary data storehouse, make that the hardware and software cost of database and maintenance cost are all higher relatively.
(2) adopt the mode in message stores to a file.The Internet email transmits agency (Mail Transfer Agent, MTA), be can be with from mail user agent (Mail UserAgent, MUA) mail is transmitted to the program of designated user, most of MTA adopts file as the storage medium of message, and an email message is stored in the file.Its storing process is: Simple Mail Transfer protocol (Simple Mail Transfer Protocol, SMTP) process receives email message, and with in file of email message preservation, simultaneously this document is joined transmit queue, return to the Email Sender then and successfully respond; MTA gets access to file to be sent again from transmit queue, namely be email message, and sends this document to mail reception person, sends successfully back deletion this document.The method of an email message of a file storage of this employing, can avoid the high development cost and the maintenance cost that adopt database to store, but also there is shortcoming in this method, adopt single file storage email message, under the bigger situation of mail data amount, too much file can reach the bottleneck (for example the inode number reaches the upper limit) of file system.
(3) part JMS, for example, ActiveMQ has the plug-in unit of Kaha, and employing is with the storage means of all message stores in same file.This method need be set up index file at hard disk, the side-play amount of recording messages in this document in this index file, then message repeater system according to the message stores process of described index file is: receiving thread receives the message that transmit leg sends, it is saved in the current file, upgrade index file according to described message content, wake the transmission thread then up, successfully respond to return to the sender; When the needs message repeater system sends this message, send thread and from index file, obtain the side-play amount of message to be sent in current file, obtain message content according to described side-play amount, and send this message to message recipient, send successfully back this message of deletion from file, the final updating index file.This method can not make that file system reaches bottleneck, but also there is shortcoming, the wherein introducing of index file, exist when reading and writing data in making, need the index file in the repetitive read-write hard disk and preserve the file of message, just caused the physics magnetic head to the too high result of the random read-write number of times of hard disk, make the readwrite performance of file descend, moreover, the content of index file can't be deleted, behind system's long-play, also may cause the excessive phenomenon of index file, take unnecessary hard drive space.
In a word, need the urgent technical problem that solves of those skilled in the art to be exactly at present: the method for storing message in a kind of message repeater system of proposition how can innovate, with under the situation of big data quantity, solve the file read-write performance decline problem that the random read-write number of times to file on the hard disk too much causes.
Summary of the invention
Technical problem to be solved by this invention provides the storage means in a kind of message repeater system, with under the situation of big data quantity, reduces the random read-write operation a large amount of to file on the hard disk, and promotes the performance of file random read-write.
The present invention also provides the device of storing message in a kind of message repeater system, in order to guarantee said method realization and application in practice.
In order to address the above problem, the invention discloses the method for storing message in a kind of message repeater system, comprising:
Message is added into current data file in the hard disk in order;
Generate the first index daily record of described message;
Internal memory is put in the described first index daily record.
Preferably, after the first index daily record of the described message of described generation, also comprise:
The described first index daily record is added into journal file in the hard disk in proper order, and described journal file is used for preserving the index daily record of message.
Preferably, described message is added into before the current data file in the hard disk in order, also comprises:
In internal memory, attempt obtaining corresponding index daily record according to message id, when obtaining not then, search the current data file for storing message.
Preferably, after the described current data file of searching for storing message, also comprise:
Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file.
Preferably, described described index daily record is put into after the internal memory, also comprises:
System is carried out initialization, with the content according to index daily record in the described journal file, in internal memory, recover the index daily record.
Preferably, described message is added into after the current data file in order, also comprises:
Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record.
Preferably, describedly internal memory put in the described first index daily record specifically comprised:
Message id and the described first index daily record are put into internal memory with the form of key and value, and described kev is message id, and described value is the described first index daily record.
Preferably, the content of the described first index journalizing comprises: storage operation, message id, file sequence, side-play amount and/or message-length.
Preferably, the described specific implementation that described message sequence is write described current data file is:
Be byte stream with described message conversion;
Order is added described byte stream in current data file.
The embodiment of the invention also discloses the device of storing message in a kind of message repeater system, comprising:
First adds module, is used for message is added in order the current data file of hard disk;
First generation module is used for generating the first index daily record of described message;
Preserve module, be used for internal memory is put in the described first index daily record.
Preferably, described device also comprises:
Second adds module, is used for the described first index daily record is added in proper order the journal file of hard disk, and described journal file is used for preserving the index daily record of message;
Preferably, described device also comprises:
First acquisition module is used for attempting obtaining corresponding index daily record at internal memory according to message id;
Search module, be used for when described acquisition module obtains less than the index daily record, searching the current data file that is used for storing message in the hard disk.
Preferably, described device also comprises:
Whether first judge module, the size that is used for judging described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file.
Preferably, described device also comprises:
Initialization module is used for system is carried out initialization, with the content according to index daily record in the described journal file, recovers the index daily record in internal memory.
Preferably, described device also comprises:
Increase module, for increasing the reference count of described current data file, described reference count is used for the number of the described current data file message of record.
Preferably, the described first interpolation module comprises:
The first conversion submodule, being used for described message conversion is byte stream;
Add submodule, be used for adding described byte stream in the current data file order.
The present invention also provides the method for obtaining message in a kind of message repeater system, comprising:
From internal memory, obtain the first index daily record of message according to message id;
From current data file, obtain the content of described message according to the described first index daily record.
Preferably, the content of the described first index daily record comprises: storage operation, message id, file sequence, side-play amount and/or message-length.
Preferably, describedly from current data file, obtain the content of described message according to the described first index daily record, specifically comprise:
Obtain file sequence, side-play amount and message-length in the described first index daily record;
Locate current data file according to described file sequence, and from described current data file, obtain byte stream according to described side-play amount and message-length;
Described byte stream is converted to message.
The present invention also provides the device that obtains message in a kind of message repeater system, comprising:
Second acquisition module is used for the first index daily record of obtaining message from internal memory according to message id;
The 3rd acquisition module is for the content of obtaining described message according to the described first index daily record from current data file.
Preferably, the content of the described first index daily record: comprise storage operation, message id, file sequence, side-play amount and/or message-length.
Preferably, described the 3rd acquisition module comprises:
First obtains submodule, for file sequence, side-play amount and the message-length of the current data file of obtaining the first index daily record;
The locator module is used for the data file according to this message of described file sequence location storage;
Second obtains submodule, is used for obtaining byte stream according to described side-play amount and message-length from described data file;
The second conversion submodule is used for described byte stream is converted to message.
The present invention also provides the method for deletion message in a kind of message repeater system, comprising:
The first index daily record according to message generates the second index daily record, and the described first index daily record is to generate gained when message is added into current data file;
The first index daily record of this message correspondence of deletion from internal memory.
Preferably, described from internal memory the deletion this message correspondence the first index daily record before, also comprise:
The described second index daily record is added in the journal file in proper order, and described journal file is used for preserving the index daily record of message.
Preferably, the described first index daily record according to message generates before the second index daily record, also comprises:
From internal memory, obtain the first index daily record of message according to message id.
Preferably, described method also comprises:
Reduce the reference count of the current data file at message place, described reference count is used for the number of the described current data file message of record.
Preferably, also comprise:
Whether the reference count of judging described current data file is zero, if then delete described current data file and current journal file.
Preferably, comprise in the described second index daily record: deletion action, message id, file sequence, side-play amount and/or message-length.
The present invention also provides the device of deletion message in a kind of message repeater system, comprising:
Second generation module is used for generating the second index daily record according to the first index daily record of message, and the described first index daily record is to generate gained when message is added into current data file;
Removing module is used for from the first index daily record of this message correspondence of internal memory deletion.
Preferably, described device also comprises:
The 3rd adds module, is used for the described second index daily record is added into journal file in proper order, and described journal file is used for preserving the index daily record of message;
Preferably, described device also comprises:
The 4th acquisition module is used for the first index daily record of obtaining message according to message id at the index of internal memory.
Preferably, described device also comprises:
Reduce module, be used for the reference count of the current data file at minimizing message place, described reference count is used for the number of the described current data file message of record.
Preferably, described device also comprises:
Second judge module is used for judging whether the reference count of described current data file is zero, if then delete described current data file and current journal file.
Compared with prior art, the present invention has the following advantages:
In embodiments of the present invention, at first message is added in order the current data file in the hard disk, generates the first index daily record of described message then, at last internal memory is put in the first index daily record.The present invention is by increasing the index daily record in internal memory, when obtaining data, can directly from data file, get access to message content by the index daily record in the internal memory, compared with prior art, reduced the random read-write operation to index file on the hard disk, thereby can promote the random read-write performance of file, further, message is kept in the different data files, and be that data file increases a reference count, under the situation that system moves always, old data file and journal file finally all can be deleted fall, thereby have reduced taking disk space.
Description of drawings
Fig. 1 is the flow chart of the method embodiment 1 of storing message in a kind of message repeater system of the present invention;
Fig. 2 is the flow chart of the method embodiment 2 of storing message in a kind of message repeater system of the present invention;
Fig. 3 is the structured flowchart of the device embodiment 1 of storing message in a kind of message repeater system of the present invention;
Fig. 4 is the structured flowchart of the device embodiment 2 of storing message in a kind of message repeater system of the present invention;
Fig. 5 is the flow chart that obtains the method embodiment 1 of message in a kind of message repeater system of the present invention;
Fig. 6 is the flow chart that obtains the method embodiment 2 of message in a kind of message repeater system of the present invention;
Fig. 7 is the structured flowchart that obtains the device preferred embodiment of message in a kind of message repeater system of the present invention;
Fig. 8 uses the flow chart that preferred embodiment shown in Figure 7 obtains message;
Fig. 9 is the flow chart of the method embodiment of deletion message in a kind of message repeater system of the present invention;
Figure 10 is the structured flowchart of the device preferred embodiment of deletion message in a kind of message repeater system of the present invention.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
The present invention can be used in numerous general or special purpose calculation element environment or the configuration.For example: personal computer, server computer, handheld device or portable set, plate equipment, multiprocessor device, comprise distributed computing environment (DCE) of above any device or equipment etc.
The present invention can describe in the general context of the computer executable instructions of being carried out by computer, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract data type, program, object, assembly, data structure etc.Also can in distributed computing environment (DCE), put into practice the present invention, in these distributed computing environment (DCE), be executed the task by the teleprocessing equipment that is connected by communication network.In distributed computing environment (DCE), program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
One of core idea of the embodiment of the invention is, in embodiments of the present invention, can at first be added into message in the current data file in order, then generate the first index daily record of described message, and the described first index daily record is added into journal file in proper order, described journal file is used for preserving the journalizing of message, at last internal memory is put in the described first index daily record.The present invention is by storing message in current data file, and generate the first index daily record it is kept in the journal file, also the described first index daily record is kept in the internal memory simultaneously, can be according to the first index daily record of internal memory when obtaining data, directly from data file, get access to message content, thereby under the situation of big data quantity, reduced the random read-write number of times of magnetic head to index file on the hard disk, promoted the random read-write performance of file.
With reference to figure 1, show the flow chart of the method embodiment 1 of storing message in a kind of message repeater system of the present invention, can may further comprise the steps:
Step 101: message is added into current data file in order;
Step 102: generate the first index daily record of described message;
Step 103: internal memory is put in the described first index daily record.
In the present embodiment, described data file is current file for storing message, and in a file, and in follow-up arrival message, order is added the byte stream of this message in the file back, can reduce the number of times that file is write at random with message stores; In the present embodiment, to the storage that is operating as of message, therefore, the described first index daily record can comprise storage operation, message id, file sequence, side-play amount and/or message-length; Wherein, the index daily record is mapping relations of setting up for the position of preserving according to message, according to described index daily record, just can obtain the content of message from data file, and in actual applications, the implementation of index daily record can have multiple.
In the present embodiment, when in internal memory, putting into described index daily record, be to preserve with the form of a mapped file, message id and the described first index daily record are put into internal memory with the form of key and value, described key is message id, and described value is the described first index daily record; In practice, can be according to the operation of storing message in data file, obtain message in the side-play amount of data file, form the first index daily record according to the sequence of side-play amount, data file, message-length etc., then the first index daily record is put in the index mapping of internal memory; So that when obtaining message, can from internal memory, obtain information such as side-play amount according to message id, just can from data file, read message.
As can be seen, because adopted the method for in internal memory, preserving the index daily record of message, and set up data file to preserve message content at hard disk, set up journal file to record the details of this message operation, so when reading message, directly from internal memory, just can obtain the corresponding first index daily record according to message id, can directly from hard disk, find the data file of this message of storage according to the content of the first index daily record, and read message, thereby reduced the random read-write number of times to file on the hard disk, promoted the random read-write performance of file.
With reference to figure 2, show the flow chart of the method embodiment 2 of storing message in a kind of message repeater system of the present invention, can may further comprise the steps:
Step 201: from internal memory, attempt obtaining the corresponding first index daily record according to message id, when obtaining not then, search the current data file for storing message;
In actual applications, for fear of the repetitive operation to message stores, can be before storing message, at first in internal memory, search the first index daily record that whether has this message according to message id, if exist, illustrate that this message has been stored in corresponding data file and has suffered, then no longer carry out subsequent operation, if obtain the first index daily record less than this message, search the current data file for storing message again;
Step 202: whether judge the size of described current data file greater than a threshold value, if, then enter step 203, if not, then enter step 204;
Whether judge current data file for storing message greater than a threshold value, this threshold value can independently be arranged by the user, and this is the measure of being avoided the excessive situation of capacity of a data file to take;
Step 203: create the new data file, and described new data file is set to current data file;
If current data file is greater than a certain threshold value, a newly-built data file then, and newly-built data file is set to current data file;
Step 204: message is added into described current data file in order;
Preferably, described step 204 can comprise following substep:
Substep 2041: be byte stream with described message conversion;
In practice, message is that form with byte stream is kept in the data file;
Substep 2042: order is added described byte stream in current data file;
Step 205: increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record;
After message successfully is stored to data file, increase the reference count of current data file, wherein said reference count is used for the number of the described current data file message of record, when data file increases a message, the reference count of this data file adds one, be understandable that when message of this data file of deletion, the reference count of this data file subtracts one; Under the situation that system moves always, the message that old data file comprises finally all can be deleted be fallen, and this moment, the reference count meeting became zero, just this data file can be deleted, and has guaranteed that the file on the hard disk is can data volume not excessive;
Step 206: generate the first index daily record of described message;
After the storing message, can generate the storing daily record of this message in data file, the content of described interpolation journalizing comprises storage operation, message id, file sequence, side-play amount and/or message-length;
Step 207: the described first index daily record is added into journal file in proper order, and described journal file is used for preserving the journalizing of message;
Described journal file is for the index daily record of preserving message, the details that message is operated can be generated an index daily record of this message, and this index daily record is kept in the journal file; Need to prove that in actual applications, data file and journal file are one to one, when carrying out the message preservation, can in data file, add the particular content of message, and in corresponding journal file, add the index daily record of this message; In the present embodiment, setting up the purpose of journal file recording messages operation, mainly is for when the system initialization, can recover the index daily record in internal memory according to the index daily record in the journal file;
Step 208: internal memory is put in the first index daily record.
Wherein, in step 207, when internal memory is put in the described first index daily record, message id and the first index daily record can be put into internal memory with the form of key and value, described key can be message id, and described value can be the described first index daily record, as can be seen, index in internal memory can be understood as a mapping in fact, can find the value of correspondence according to the value of key;
Preferably, present embodiment can also comprise:
Step 209: system is carried out initialization, with the content according to the index daily record of recording in the described journal file, in internal memory, recover the index daily record.
Recover index daily record in the internal memory by journal file during initialization, the system that refers to is when restarting, can the index daily record of message in internal memory be rebulid out according to the content in the journal file, concrete process is: read every index daily record in the journal file when restarting, if read the first index daily record, then adding the first index daily record of this message in the index mapping of internal memory, namely is the storage operation that message is done; Read after all journal files, the index in the internal memory has just recovered, and can carry out normal message read-write operation.
Need to prove, described system is carried out initialized step, not only can after storing message, carry out, also message can obtained, after perhaps deleting message system is carried out initialization, system is carried out initialized step when carry out, do not influence realization of the present invention, the present invention need not this is made restriction.
For aforesaid each method embodiment, for simple description, so it all is expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not subjected to the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in the specification all belongs to preferred embodiment, and related action and module might not be that the present invention is necessary.
For making those skilled in the art understand the present invention better, the present invention is further described below in conjunction with object lesson that the present invention uses in practice.
Steps A 1: in internal memory, attempt obtaining corresponding index daily record according to message id, when obtaining not then, search the current data file for storing message;
Suppose that in practice message id is abcd, in internal memory, attempt obtaining according to message id so and the index daily record of this message id correspondence, when obtaining not then, then search the current data file for storing message;
Steps A 2: whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file;
In the present example, described threshold value is set to 64M, if then the size of current data file is greater than 64M, need to create the new data file, described new data file is set to current data file shoulder to shoulder, wherein, the setting of threshold value can be according to the message number of hard disk in the reality, also can be according to user's actual demand, the size of threshold value does not influence realization of the present invention, and the present invention does not make restriction to this;
Steps A 3: be byte stream with described message conversion;
In practice, message is that form with byte stream is kept in the data file;
Steps A 4: order is added described byte stream in current data file;
Steps A 5: increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record;
In the present example, each data file has a reference count in internal memory, and the corresponding relation of it and data file can realize that key is the data file sequence number with the mode of key and value, and value is reference count; Need to prove, because journal file and data file are one to one, so 1 corresponding journal file 1 of data file, so the existence of reference count also can be so that old journal file also can be deleted;
Steps A 6: generate the first index daily record of described message;
Side-play amount in the time of can calculating store byte stream in steps A 4 in data file in this step, can generate the first index daily record of message, and in the present example, the first concrete index daily record can be as shown in the table:
| Operation: 1byte | Message id: 16byte | File sequence: 4byte | Side-play amount: 8byte | Message-length: 4byte |
| 0 | Abcd | 4 | 100 | 20K |
Wherein, the value of operation is 0, can represent storage operation that this message is carried out, and is then corresponding, is set to deletion action with 1, wherein, also numerical value can be set arbitrarily as required when arranging, and the present invention does not make restriction to this;
Steps A 7: the described first index daily record is added into journal file in proper order, and described journal file is used for preserving the index daily record of message;
Because file sequence classifies 4 as, then the address of message stores is data file 4, therefore the described first index daily record is also needed to be added into journal file 4;
Steps A 8: internal memory is put in the described first index daily record.
According to the first above-mentioned index daily record, when representing to put into the process of index with the java language, script is:
Log?log＝new?Log()；
Log.operation=0; // suppose that 0 refers to add message
Log.dataFileSequence=" 4 "; // data file 4
Log.dataFileOffset=100; // 100 side-play amount
Log.messageLength=20; // message-length is 20
IndexMap.put (" abcd ", log); // be the daily record of abcd with message id, put into the internal memory index
//indexMap just generates well when system starts; Such as: Map indexMap=new HashMap ()
In sum, in embodiments of the present invention, by said method, the index daily record is kept in the internal memory, and in system initialization, can recover index daily record in the internal memory by journal file, thereby can solve the readwrite performance of file, and introduce the concept of reference count, make that old data file and journal file can be deleted, even if can guarantee under the situation of big data quantity, also can be less relatively to taking of disk.
Corresponding with the method that the method embodiment 1 of storing message in a kind of message repeater system of the invention described above provides, referring to Fig. 3, the present invention also provides the device embodiment 1 of storing message in a kind of message repeater system, and in the present embodiment, this device comprises:
First adds module 301, is used for message is added into current data file in order;
In the present embodiment, form module and can in internal memory, preserve the index daily record of message, first adds module can the data file on hard disk preserve message content, therefore, when reading described message, directly from internal memory, just can obtain the corresponding first index daily record according to message id, can directly find the data file of this message of storage according to the content of the first index daily record, thereby reduced the random read-write number of times to file on the hard disk, promoted the readwrite performance of file.
Corresponding with the method that the method embodiment 2 of storing message in a kind of message repeater system of the invention described above provides, referring to Fig. 4, the present invention also provides the preferred embodiment 2 of the device of storing message in a kind of message repeater system, and in the present embodiment, this device comprises:
Whether first judge module 403, the size that is used for judging described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file;
First adds module 404, is used for message is added into current data file in order;
Preferably, the described first interpolation module can comprise following submodule:
The first conversion submodule 4041, being used for described message conversion is byte stream;
Add submodule 4042, be used for adding described byte stream in the current data file order.
Second adds module 407, is used for the described first index daily record is added into journal file in proper order, and described journal file is used for preserving the journalizing of message;
Described second adds module can record this message operation in journal file details;
Preferably, present embodiment can also comprise:
With reference to figure 5, show the flow chart that obtains the method embodiment 1 of message in a kind of message repeater system of the present invention, this method comprises:
Step 501: the first index daily record of from internal memory, obtaining message according to message id;
Step 502: the content of from current data file, obtaining described message according to the first index daily record.
Need to prove, the content of the described first index daily record comprises storage operation, message id, file sequence, side-play amount and/or message-length, when obtaining message, can directly from internal memory, obtain the corresponding first index daily record according to message id, according to getting access to the first index daily record, can directly from the data file of storing message, get access to the content of this message.
As can be seen, the user directly obtains the index daily record in internal memory, compare with the method that in the prior art index file is kept on the hard disk, has saved the read-write operation to index file on the hard disk, thereby reduced the random read-write operation to file on the hard disk, promoted the random read-write performance of file.
With reference to figure 6, show the flow chart that obtains the method embodiment 2 of message in a kind of message repeater system of the present invention, this method comprises:
Step 601: the first index daily record of from internal memory, obtaining message according to message id;
In the present embodiment, suppose that message id is abcd, be understandable that the first index daily record that this step gets access to is:
| Operation: 1byte | Message id: 16byte | File sequence: 4byte | Side-play amount: 8byte | Message-length: 4byte |
| 0 | Abcd | 4 | 100 | 20 |
Step 602: obtain file sequence, side-play amount and message-length in the described first index daily record;
Namely be the information of obtaining data file 4,100 side-play amount, and length is 20 message;
Step 603: locate described current data file according to described file sequence, and from described current data file, obtain byte stream according to described side-play amount and message-length;
Step 604: described byte stream is converted to message.
When obtaining message, realize that with the java language script of acquisition process is as follows:
Log log=indexMap.get (" abcd "); // get access to the first index daily record according to message id
If (log!=null) if // exist
Message msg=read Message (log.dataFileSequence, log.dataFileOffset, log.messageLength); // according to the file sequence sequence number of data file, side-play amount and message-length read message.
}
With reference to figure 7, show the structured flowchart that obtains the device preferred embodiment of message in a kind of message repeater system of the present invention, this method comprises:
The 3rd acquisition module 702 is for the content of obtaining described message according to the first index daily record from current data file;
Preferably, described second acquisition module can comprise following submodule:
First obtains submodule 7021, for file sequence, side-play amount and the message-length of the current data file of obtaining the first index daily record;
Second obtains submodule 7023, is used for obtaining byte stream according to described side-play amount and message-length from described data file;
The second conversion submodule 7024 is used for described byte stream is converted to message.
With reference to figure 8, show and use the flow chart that preferred embodiment shown in Figure 7 obtains message, can may further comprise the steps:
Step 801: first acquisition module obtains the first index daily record of message from internal memory according to message id;
Step 802: first obtains file sequence, side-play amount and the message-length that submodule obtains the current data file in the first index daily record;
Step 803: the locator module is according to the data file of this message of described file sequence location storage;
Step 804: second obtains submodule obtains byte stream according to described side-play amount and message-length from described data file;
Step 805: the second conversion submodule is converted to message with described byte stream.
With reference to figure 9, show the flow chart of the method embodiment of deletion message in a kind of message repeater system of the present invention, this method comprises:
Step 901: the first index daily record according to message generates the second index daily record, and the described first index daily record is to generate gained when message is added into current data file;
In this step, main purpose is to generate the second index daily record, and wherein corresponding with the first index daily record of preceding method, the described second index daily record is specifically as follows in practice:
| Operation: 1byte | Message id: 16byte | File sequence: 4byte | Side-play amount: 8byte | Message-length: 4byte |
| 1 | Abcd | 4 | 100 | 20 |
As can be seen, comprise deletion action, message id, file sequence, side-play amount and message-length in the described second index daily record, wherein, numeral 1 representative in the table is operating as deletion;
Step 902: the described second index daily record is added in the journal file in proper order, and described journal file is used for preserving the index daily record of message;
Step 904: the first index daily record of this message correspondence of deletion from internal memory.
Need to prove that in the present embodiment, described method can also comprise:
Step 905: reduce the reference count of the current data file at message place, described reference count is used for the number of the described current data file message of record.
After deletion message, need the reference count of the current data file at minimizing message place, so just can guarantee that the reference count of data file is in a kind of state of renewal;
Step 906: whether the reference count of judging described current data file is zero, if then delete described current data file and current journal file.
When the reference count of described data file is zero, current data file and current journal file are deleted in the lump, saved a large amount of hard drive spaces.
With reference to Figure 10, show the structured flowchart of the device preferred embodiment of deletion message in a kind of message repeater system of the present invention, this device comprises:
The 3rd adds module 1002, is used for the described second index daily record is added into journal file in proper order, and described journal file is used for preserving the journalizing of message;
Removing module 1003 is used for from the index of this message correspondence of internal memory deletion;
The 4th acquisition module 1004 is used for the first index daily record of obtaining message according to message id at the index of internal memory;
Need to prove that each embodiment in this specification all adopts the mode of going forward one by one to describe, what each embodiment stressed is and the difference of other embodiment that identical similar part is mutually referring to getting final product between each embodiment.For device class embodiment, because it is similar substantially to method embodiment, so description is fairly simple, relevant part gets final product referring to the part explanation of method embodiment.
Also need to prove, in this article, relational terms such as first and second grades only is used for an entity or operation are made a distinction with another entity or operation, and not necessarily requires or hint and have the relation of any this reality or in proper order between these entities or the operation.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby make and comprise that process, method, article or the equipment of a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or also be included as the intrinsic key element of this process, method, article or equipment.Do not having under the situation of more restrictions, the key element that is limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
More than the method and apparatus of storing message in a kind of message repeater system provided by the present invention is described in detail, used specific case herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (18)
1. the method for storing message in the message repeater system is characterized in that this method comprises:
In internal memory, attempt obtaining corresponding index daily record according to message id, when obtaining not then, search the current data file for storing message;
Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file;
Message is added into described current data file in order;
Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record;
Generate the first index daily record of described message; The content of the described first index journalizing comprises: the sequence of storage operation, message id, data file, side-play amount and/or message-length;
Internal memory is put in the described first index daily record;
When deletion message, reduce the reference count of described current data file, and when described reference count is zero, delete described current data file and current journal file.
2. the method for claim 1 is characterized in that, after the first index daily record of the described message of described generation, also comprises:
The described first index daily record is added into journal file in the hard disk in proper order, and described journal file is used for preserving the index daily record of message.
3. method as claimed in claim 2 is characterized in that, described described index daily record is put into after the internal memory, also comprises:
System is carried out initialization, with the content according to index daily record in the described journal file, in internal memory, recover the index daily record.
4. the method for claim 1 is characterized in that, describedly internal memory is put in the described first index daily record is specifically comprised:
Message id and the described first index daily record are put into internal memory with the form of key and value, and described key is message id, and described value is the described first index daily record.
5. the method for claim 1 is characterized in that, the described specific implementation that described message sequence is write described current data file is:
Be byte stream with described message conversion;
Order is added described byte stream in current data file.
6. the storage device in the message repeater system is characterized in that, comprising:
First acquisition module is used for attempting obtaining corresponding index daily record at internal memory according to message id;
Search module, be used for when described acquisition module obtains less than the index daily record, searching the current data file that is used for storing message in the hard disk;
Whether first judge module, the size that is used for judging described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file;
First adds module, is used for message is added into described current data file in order;
Increase module, for increasing the reference count of described current data file, described reference count is used for the number of the described current data file message of record;
First generation module is used for generating the first index daily record of described message;
Preserve module, be used for internal memory is put in the described first index daily record;
Storage device in the described message repeater system also is used for: reduce the reference count of the current data file at message place when deletion message, and delete described current data file and current journal file when described reference count is zero.
7. device as claimed in claim 6 is characterized in that, also comprises:
Second adds module, is used for the described first index daily record is added in proper order the journal file of hard disk, and described journal file is used for preserving the index daily record of message.
8. device as claimed in claim 7 is characterized in that, also comprises:
Initialization module is used for system is carried out initialization, with the content according to index daily record in the described journal file, recovers the index daily record in internal memory.
9. device as claimed in claim 6 is characterized in that, described first adds module comprises:
The first conversion submodule, being used for described message conversion is byte stream;
Add submodule, be used for adding described byte stream in the current data file order.
10. obtain the method for message in the message repeater system, it is characterized in that, comprising:
From internal memory, obtain the first index daily record of message according to message id; The content of the described first index journalizing comprises: the sequence of storage operation, message id, data file, side-play amount and/or message-length; The described first index daily record generates in the following manner: attempt obtaining corresponding index daily record according to message id in internal memory, when obtaining not then, search the current data file for storing message; Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file; Message is added into described current data file in order; Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record; Generate the first index daily record of described message;
From current data file, obtain the content of described message according to the described first index daily record;
When deletion message, reduce the reference count of the current data file at message place;
Whether the reference count of judging described current data file is zero, if then delete described current data file and current journal file.
11. method as claimed in claim 10 is characterized in that, describedly obtains the content of described message according to the described first index daily record from current data file, specifically comprises:
Obtain sequence, side-play amount and the message-length of the data file in the described first index daily record;
Sequence according to described data file is located current data file, and obtains byte stream according to described side-play amount and message-length from described current data file;
Described byte stream is converted to message.
12. obtain the device of message in the message repeater system, it is characterized in that, comprising:
Second acquisition module is used for the first index daily record of obtaining message from internal memory according to message id; The content of the described first index journalizing comprises: the sequence of storage operation, message id, data file, side-play amount and/or message-length; The described first index daily record generates in the following manner: attempt obtaining corresponding index daily record according to message id in internal memory, when obtaining not then, search the current data file for storing message; Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file; Message is added into described current data file in order; Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record; Generate the first index daily record of described message;
The 3rd acquisition module is for the content of obtaining described message according to the described first index daily record from current data file;
The device that obtains message in the described message repeater system also is used for: reduce the reference count of the current data file at message place when deletion message, and whether the reference count of judging described current data file is zero, if then delete described current data file and current journal file.
13. device as claimed in claim 12 is characterized in that, described the 3rd acquisition module comprises:
First obtains submodule, is used for obtaining sequence, side-play amount and the message-length of data file of the current data file of the first index daily record;
The locator module is used for the data file according to this message of sequence location storage of described data file;
Second obtains submodule, is used for obtaining byte stream according to described side-play amount and message-length from described data file;
The second conversion submodule is used for described byte stream is converted to message.
14. the method for deletion message in the message repeater system is characterized in that this method comprises:
From internal memory, obtain the first index daily record of message according to message id;
The first index daily record according to message generates the second index daily record, and the described first index daily record is to generate gained when message is added into current data file; The content of the described first index journalizing comprises: the sequence of storage operation, message id, data file, side-play amount and/or message-length; The generating mode of the described first index daily record is as follows: attempt obtaining corresponding index daily record according to message id in internal memory, when obtaining not then, search the current data file for storing message; Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file; Message is added into described current data file in order; Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record; Generate the first index daily record of described message;
The first index daily record of this message correspondence of deletion from internal memory;
When deletion message, reduce the reference count of the current data file at message place;
Whether the reference count of judging described current data file is zero, if then delete described current data file and current journal file.
15. method as claimed in claim 14 is characterized in that, described from internal memory the deletion this message correspondence the first index daily record before, also comprise:
The described second index daily record is added in the journal file in proper order, and described journal file is used for preserving the index daily record of message.
16. method as claimed in claim 14 is characterized in that, comprises in the described second index daily record: the sequence of deletion action, message id, data file, side-play amount and/or message-length.
17. the device of deletion message is characterized in that in the message repeater system, comprising:
The 4th acquisition module is used for the first index daily record of obtaining message according to message id at the index of internal memory;
Second generation module is used for generating the second index daily record according to the first index daily record of message, and the described first index daily record is to generate gained when message is added into current data file; The content of the described first index journalizing comprises: the sequence of storage operation, message id, data file, side-play amount and/or message-length; The described first index daily record generates in the following manner: attempt obtaining corresponding index daily record according to message id in internal memory, when obtaining not then, search the current data file for storing message; Whether judge the size of described current data file greater than a threshold value, if, then create the new data file, and described new data file is set to current data file, if not, it is constant then to keep described current data file; Message is added into described current data file in order; Increase the reference count of described current data file, described reference count is used for the number of the described current data file message of record; Generate the first index daily record of described message;
Removing module is used for from the first index daily record of this message correspondence of internal memory deletion;
Reduce module, be used for the reference count of the current data file at minimizing message place;
Second judge module is used for judging whether the reference count of described current data file is zero, if then delete described current data file and current journal file.
18. device as claimed in claim 17 is characterized in that, also comprises:
The 3rd adds module, is used for the described second index daily record is added into journal file in proper order, and described journal file is used for preserving the index daily record of message.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200810166811 CN101686209B (en) | 2008-09-24 | 2008-09-24 | Method and device for storing message in message retransmission system |
| HK10107945.9A HK1141645A1 (en) | 2008-09-24 | 2010-08-20 | Method and apparatus for storing message in message retransmisson system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200810166811 CN101686209B (en) | 2008-09-24 | 2008-09-24 | Method and device for storing message in message retransmission system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101686209A CN101686209A (en) | 2010-03-31 |
| CN101686209B true CN101686209B (en) | 2013-10-09 |
Family
ID=42049185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200810166811 Active CN101686209B (en) | 2008-09-24 | 2008-09-24 | Method and device for storing message in message retransmission system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN101686209B (en) |
| HK (1) | HK1141645A1 (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101883181A (en) * | 2010-06-18 | 2010-11-10 | 中兴通讯股份有限公司 | Method and terminal for processing short message |
| CN102694828B (en) * | 2011-03-23 | 2016-03-30 | 中兴通讯股份有限公司 | A kind of method of distributed cache system data access and device |
| CN102843396B (en) * | 2011-06-22 | 2018-03-13 | 中兴通讯股份有限公司 | Data write-in and read method and device in a kind of distributed cache system |
| CN102890675B (en) * | 2011-07-18 | 2015-05-13 | 阿里巴巴集团控股有限公司 | Method and device for storing and finding data |
| CN102521112A (en) * | 2011-11-18 | 2012-06-27 | 深圳中兴网信科技有限公司 | Memory-based reading and writing method of log information |
| CN103136215A (en) * | 2011-11-24 | 2013-06-05 | 腾讯科技（深圳）有限公司 | Data read-write method and device of storage system |
| CN103259828B (en) * | 2012-02-21 | 2016-04-06 | 阿里巴巴集团控股有限公司 | A kind of message storage management method of message-oriented middleware and system |
| CN103391257A (en) * | 2012-05-08 | 2013-11-13 | 阿里巴巴集团控股有限公司 | Message storing and forwarding method, device and system |
| CN102868564A (en) * | 2012-10-10 | 2013-01-09 | 山东中创软件商用中间件股份有限公司 | Information persistence method and system |
| WO2014075273A1 (en) * | 2012-11-15 | 2014-05-22 | 华为技术有限公司 | Data read/write method, device and system |
| CN104102750B (en) * | 2013-04-11 | 2018-04-27 | 深圳市腾讯计算机系统有限公司 | Operation data recording method, device and server |
| CN103559119B (en) * | 2013-10-25 | 2016-03-02 | 华为技术有限公司 | File operation requests disposal route and device |
| CN103678570B (en) * | 2013-12-10 | 2016-06-01 | 中国人民解放军理工大学 | The multi-level storage of journal file in cloud environment and restoration methods and system |
| CN104077363B (en) * | 2014-06-10 | 2017-11-03 | 新浪网技术(中国)有限公司 | Mail server and its method for carrying out mail full-text search |
| CN104123104B (en) * | 2014-08-13 | 2018-05-15 | 中国银行股份有限公司 | Daily record control system and method |
| CN108132845A (en) * | 2016-12-01 | 2018-06-08 | 阿里巴巴集团控股有限公司 | Message storage, delivering method and device and electronic equipment |
| CN109714228B (en) * | 2018-12-24 | 2020-10-16 | 工联数据技术(杭州)有限公司 | Global monitoring system for equipment and workers |
| CN109739822B (en) * | 2018-12-26 | 2021-04-13 | 中国移动通信集团江苏有限公司 | Message storage method, device, equipment and storage medium |
| CN111797063A (en) * | 2020-06-28 | 2020-10-20 | 中孚信息股份有限公司 | Streaming data processing method and system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5129074A (en) * | 1988-09-22 | 1992-07-07 | Hitachi Vlsi Engineering Corporation | Data string storage device and method of storing and retrieving data strings |
| CN1687932A (en) * | 2005-05-30 | 2005-10-26 | 北大方正集团有限公司 | Index structuring method for fast searching mass picture based on content |
| CN101217385A (en) * | 2008-01-15 | 2008-07-09 | 中兴通讯股份有限公司 | A method and system for the temporary storage and treatment of charging bill |
-
2008
- 2008-09-24 CN CN 200810166811 patent/CN101686209B/en active Active
-
2010
- 2010-08-20 HK HK10107945.9A patent/HK1141645A1/en unknown
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5129074A (en) * | 1988-09-22 | 1992-07-07 | Hitachi Vlsi Engineering Corporation | Data string storage device and method of storing and retrieving data strings |
| CN1687932A (en) * | 2005-05-30 | 2005-10-26 | 北大方正集团有限公司 | Index structuring method for fast searching mass picture based on content |
| CN101217385A (en) * | 2008-01-15 | 2008-07-09 | 中兴通讯股份有限公司 | A method and system for the temporary storage and treatment of charging bill |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101686209A (en) | 2010-03-31 |
| HK1141645A1 (en) | 2010-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101686209B (en) | Method and device for storing message in message retransmission system | |
| US9507821B2 (en) | Mail indexing and searching using hierarchical caches | |
| US11567681B2 (en) | Method and system for synchronizing requests related to key-value storage having different portions | |
| US9537970B2 (en) | Publisher-based message data caching in a publish-subscription environment | |
| US9246859B2 (en) | Peer-to-peer collaboration of publishers in a publish-subscription environment | |
| KR100747466B1 (en) | A device management client and device management method using nodes having additional properties | |
| CN110968431A (en) | Message processing method, device and equipment | |
| CN109710185A (en) | Data processing method and device | |
| US11748357B2 (en) | Method and system for searching a key-value storage | |
| CN110750372B (en) | Log system and log management method based on shared memory | |
| US9185181B2 (en) | Shared cache for potentially repetitive message data in a publish-subscription environment | |
| CN111309266A (en) | Distributed storage metadata system log optimization system and method based on ceph | |
| US8874666B2 (en) | Publisher-assisted, broker-based caching in a publish-subscription environment | |
| WO2020233461A1 (en) | Message queue-based communication method and apparatus | |
| CN104579920A (en) | Mail sending method and device | |
| KR102247247B1 (en) | Method to manage database | |
| CN114896215A (en) | Metadata storage method and device | |
| CN112488708B (en) | Block chain account relevance query method and false transaction screening method | |
| CN113626457A (en) | Method and system for realizing database and cache consistency by cache deletion retry mechanism | |
| CN115080625A (en) | Caching method, device and equipment based on Spring Cache framework and storage medium | |
| CN113961566A (en) | Transaction flow processing method and system | |
| CN101453481A (en) | Method for implementing notification through cache | |
| CN110830361B (en) | Mail data storage method and device | |
| CN114063931A (en) | Data storage method based on big data | |
| CN110515993B (en) | Tax data conversion method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1141645 Country of ref document: HK |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1141645 Country of ref document: HK |