CN102053923A - Storage method and storage device for logbook data - Google Patents
Storage method and storage device for logbook data Download PDFInfo
- Publication number
- CN102053923A CN102053923A CN2009102372031A CN200910237203A CN102053923A CN 102053923 A CN102053923 A CN 102053923A CN 2009102372031 A CN2009102372031 A CN 2009102372031A CN 200910237203 A CN200910237203 A CN 200910237203A CN 102053923 A CN102053923 A CN 102053923A
- Authority
- CN
- China
- Prior art keywords
- daily record
- record data
- sub
- buffer area
- buffer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a storage method and storage device for logbook data. The storage device comprises a receiver, a buffer area supervisor, a cache and a storage unit, wherein the receiver is used for receiving the logbook data; the storage unit is used for requesting different types of data in turn from the buffer area supervisor, and storing the logbook data in a database after receiving the logbook data; the buffer area supervisor is used for saving the received logbook data in the chache; and when the request of the storage unit is received, the unextracted logbook data of the requested type in the cache is extracted and is sent to the storage unit. According to the invention, the efficiency of logbook data batch processing can be improved.
Description
Technical field
The present invention relates to the daily record data field of storage, be specifically related to a kind of storage means and memory storage of daily record data.
Background technology
At present, carrying out daily record data when storage, mainly adopt single buffer area technology, promptly the take over party completely stores the data of receiving into a shared buffer area, and processing side takes out data more one by one and handles from buffer area then.
This traditional processing mode is no problem in the forms data processing logic, but performance is very low in the batch processing of polytype data.
Summary of the invention
The technical problem to be solved in the present invention provides a kind of storage means and memory storage of daily record data, can improve the efficient of daily record data batch processing.
In order to address the above problem, the invention provides a kind of memory storage of daily record data, comprise the receiver that is used to receive daily record data; Also comprise:
The buffer queue management device;
Buffer memory;
Storer is used for asking dissimilar daily record datas in turn to described buffer queue management device, is stored in the database after receiving daily record data;
The daily record data that described buffer queue management device is used for receiving is saved in described buffer memory; After the request of receiving described storer, extract the daily record data of the institute's request type that did not extract in the described buffer memory, send to described storer.
Further, described storer is received to be stored in the database behind the daily record data and is meant: after described storer is received daily record data at every turn, store in the database the daily record data of being received is disposable.
Further, described buffer memory comprises:
With a plurality of one to one sub-buffer areas of all types of daily record datas;
Described buffer queue management device is saved in the daily record data that receives in the described buffer memory and is meant: described buffer queue management device is saved in the daily record data that receives in the pairing sub-buffer area of daily record data of the type;
The daily record data that described buffer queue management device extracts the institute's request type that did not extract in the described buffer memory is meant: described buffer queue management device in the pairing sub-buffer area of the daily record data of institute's request type, the daily record data that did not extract.
Further, described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
Described buffer queue management device is saved in the daily record data that is received in the described writing position according to the buffer memory indication information of this sub-buffer area when daily record data being saved in the sub-buffer area, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
Further, described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Described buffer memory indication information comprises write pointer and read pointer; Described write pointer is used for indicating the writing position of corresponding sub-buffer area daily record data; Described read pointer is used for indicating the reading location of corresponding sub-buffer area daily record data;
Described buffer queue management device received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one when daily record data being saved in the sub-buffer area; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
Further, described buffer queue management device adopts a buffer pointers array to preserve the reading and writing pointer that corresponds respectively to each sub-buffer area.
The present invention also provides a kind of storage means of daily record data, comprising:
Receive daily record data, the daily record data that is received is saved in the buffer memory;
Ask dissimilar daily record datas in turn;
Extract the daily record data of the institute's request type that did not extract in the described buffer memory, store in the database.
Further, the daily record data that is extracted being stored in the step in the database, is disposable the storing in the database of daily record data of extracting at every turn.
Further, before the step that receives daily record data, also comprise:
Buffer memory is divided into and a plurality of one to one sub-buffer areas of all types of daily record datas;
The daily record data that is received is saved in the buffer memory is meant: the daily record data that is received is saved in the pairing sub-buffer area of daily record data of the type;
The daily record data that extracts the institute's request type that did not extract in the described buffer memory is meant: in the pairing sub-buffer area of the daily record data of institute's request type, and the daily record data that did not extract.
Further, described method also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
When daily record data being saved in the sub-buffer area, according to the buffer memory indication information of this sub-buffer area the daily record data that is received is saved in the described writing position, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
Further, described method also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information comprises write pointer and read pointer respectively, and described write pointer is used for indicating the writing position of corresponding sub-buffer area daily record data, and described read pointer is used for indicating the reading location of corresponding sub-buffer area daily record data;
When daily record data being saved in the sub-buffer area, received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
Further, the described reading and writing pointer that corresponds respectively to each sub-buffer area is kept in the buffer pointers array.
Dependence between technical scheme separation daily record data of the present invention receives and stores is independently moving it separately in the thread, avoid relying on because of waiting for the performance loss that causes mutually and moving, and can improve the efficient of daily record data batch processing; A prioritization scheme of the present invention carries out cache management by type with daily record data, when database storing, can once obtain many data of specified type daily record, so that when carrying out database storing, can once store many daily record datas, make up many records in the SQL statement such as inserting at one, processing mode than an Insert statement of a record, can under the batch disposition, improve the system data handling property greatly, stable, processing service efficiently is provided in the height processing environment of concurrent (several thousand record/second); Another prioritization scheme of the present invention can provide the good monitoring support to system running state, provides convenient support for the tracking and the monitoring of data mode.
Description of drawings
Fig. 1 is the structural representation of memory storage of the daily record data of embodiment two;
Fig. 2 is the schematic flow sheet of storage means of the daily record data of embodiment 11.
Embodiment
Below in conjunction with drawings and Examples technical scheme of the present invention is described in detail.
Embodiment one, and a kind of memory storage of daily record data comprises: buffer queue management device KlogCache; Buffer memory;
Receiver KReceiver is used for receiving extraneous---the daily record data that sends as server;
Storer KDBCarriage is used for asking dissimilar daily record datas in turn to described buffer queue management device, asks one type daily record data at every turn; After also being used to receive the daily record data that described buffer queue management device sent, be stored in the database;
The daily record data that described buffer queue management device is used for that described receiver is received is saved in described buffer memory; And after the request of receiving described storer, extract the daily record data of the institute's request type that did not extract in the described buffer memory, send to described storer.
Because after receiving not is to write database immediately, but carry out buffer memory according to type earlier, the reception of daily record data and storage can independently be carried out separately, thereby avoided because of waiting for that mutually the performance loss and the operation that cause rely on.
In the present embodiment, the type of described daily record data can be divided in advance according to actual conditions, such as the difference according to operation, behavior, daily record data incorporated into is dissimilar daily record datas.
Described storer can but be not limited in storage circulation, ask; Do not get rid of first request during practical application, the scheme of preserving database again in circulating to storage yet.
In the present embodiment, described storer is asked dissimilar daily record datas to be meant in turn and is asked all types of daily record datas successively, when three types of A, B, C are arranged, the daily record data of cycle request category-A type, category-B type, C type then, this moment, all types of requested number of times of daily record data was a basis equalization; Also can be meant according to certain order request, such as the daily record data of cycle request category-A type, category-B type, category-A type, C type, so when the daily record data of some type more for a long time, can ask several times more; This order can give tacit consent to, pre-if artificial specify etc.In the practical application, how to ask dissimilar daily record datas can also select other implementation according to demand in turn.
The described daily record data that did not extract is the daily record data of new the type that receives after the request of last stored device, may be one or more; Described storer is stored in the database and can be meant after receiving daily record data: after described storer is received daily record data at every turn, store in the database the daily record data of being received is disposable.
Owing to can once ask many, database is advanced in the disposable storage of many daily record datas of asking, therefore can significantly improve memory property, the lifting that allows the processing speed of daily record data produce matter.
Embodiment two, and a kind of memory storage of daily record data as shown in Figure 1, comprising: buffer queue management device KlogCache, receiver KReceiver, buffer memory and storer KDBCarriage.
In the present embodiment, described buffer memory comprises: correspond respectively to a plurality of sub-buffer area of dissimilar daily record datas, just with a plurality of one to one sub-buffer areas of all types of daily record datas; Other realizes that details can be with embodiment one.
In the present embodiment, described sub-buffer area can but be not limited to an annular data pool; Can be array or chained list; In the practical application, do not get rid of other form of employing yet and realize sub-buffer area.
In the present embodiment, described buffer queue management device is saved in the daily record data that receives in the described buffer memory and is meant: described buffer queue management device is saved in the daily record data that receives in the pairing sub-buffer area of daily record data of the type;
The daily record data that described buffer queue management device extracts the institute's request type that did not extract in the described buffer memory is meant: described buffer queue management device in the pairing sub-buffer area of the daily record data of institute's request type, the daily record data that did not extract.
When practical application, if buffer memory unallocated be sub-buffer area, described buffer queue management device also can be preserved the sign that is used to indicate its type simultaneously when preserving daily record data; Like this when extracting the daily record data of institute's request type, can according to described sign find the daily record data of request type to some extent.
The memory storage of embodiment three, a kind of daily record data comprises: buffer queue management device KlogCache, receiver KReceiver, buffer memory and storer KDBCarriage; Described buffer memory comprises and a plurality of one to one sub-buffer areas of all types of daily record datas.
In the present embodiment, described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
Described buffer queue management device is saved in the daily record data that is received in the described writing position according to the buffer memory indication information of this sub-buffer area when daily record data being saved in the sub-buffer area, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
From above as seen, described buffer memory indication information is corresponding one by one with described sub-buffer area, and in fact just the daily record data with all types of is corresponding one by one.
When practical application, do not get rid of the situation of not preserving the buffer memory indication information yet, realize sub-buffer area such as forms such as adopting FIFO (first-in first-out register), storehouse, emptying sub-buffer area behind each extraction daily record data, needs just all extract the daily record data in the sub-buffer area when extracting and get final product; Or set up one for every daily record data and extract zone bit, whether extracted to indicate this daily record data.
Other realizes that details can be with embodiment one or two.
The memory storage of embodiment four, a kind of daily record data comprises: buffer queue management device KlogCache, receiver KReceiver, buffer memory and storer KDBCarriage; Described buffer memory comprises and a plurality of one to one sub-buffer areas of all types of daily record datas.
In the present embodiment, described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Described buffer memory indication information comprises write pointer p_write and read pointer p_read; Described write pointer p_writ is used for indicating the writing position of corresponding sub-buffer area daily record data; Described read pointer p_read is used for indicating the reading location of corresponding sub-buffer area daily record data.
Described buffer queue management device received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one when daily record data being saved in the sub-buffer area; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
Such as the sub-buffer area of hypothesis is the array form; During extraction, write pointer is 6, points to the 6th array, and read pointer is 3, points to the 3rd array, then extracts daily record data since the 3rd array, extracts to stop behind the 3rd, 4,5 daily record data in the array extracting; Read pointer is increased by 3, become 6, this also just means will be since the 6th array when extracting next time.
During practical application, do not get rid of the buffer memory indication information that adopts other form yet, such as the writing position of preserving and the address of reading location with register or other memory device, or the writing position of preserving and the sign of reading location etc.The buffer memory indication information can save as the two parts that are respectively applied for indication reading and writing position, also can be an integral body, and wherein a part is used to indicate writing position, and another part is used to indicate reading location.
During practical application, also can be that daily record data of every extraction just adds one with described read pointer, judge then whether read pointer and write pointer equate, are then to stop to extract; Otherwise the daily record data in the position that continuation extraction read pointer points to.The scheme of extracting the disposable increase read pointer in back is can efficient higher, but need count the bar number of the daily record data that extracted.
In the present embodiment, described buffer queue management device can but be not limited to adopt a buffer pointers CachePointer array to preserve the reading and writing pointer that corresponds respectively to each sub-buffer area; During practical application, do not get rid of the independent reading and writing pointer of preserving the daily record data of each sub-buffer area correspondence yet, or adopt other form to concentrate and preserve, such as the mapping table of sub-buffer area sign and reading and writing pointer etc.
Other realizes that details can be with embodiment one or two.
The memory storage of embodiment five, a kind of daily record data comprises: buffer queue management device KlogCache, receiver KReceiver, storer KDBCarriage, buffer memory and a watch-dog KUiStatus.
Described watch-dog is used for the monitoring when monitoring that each described buffer memory indication information moves the read-write situation of various types of daily record datas, with instant understanding, tracker ruuning situation;
Other realizes that details can be with above arbitrary embodiment.
Embodiment six, and a kind of storage means of daily record data comprises:
Receiving extraneous---the daily record data as server sends is saved in the daily record data that is received in the buffer memory;
Ask dissimilar daily record datas in turn, ask one type daily record data at every turn;
Extract the daily record data of the institute's request type that did not extract in the described buffer memory, store in the database.
Because after receiving not is to write database immediately, but carry out buffer memory according to type earlier, the reception of daily record data and storage can independently be carried out separately, thereby avoided because of waiting for that mutually the performance loss and the operation that cause rely on.
In the present embodiment, the type of described daily record data can be divided in advance according to actual conditions, such as the difference according to operation, behavior, daily record data incorporated into is dissimilar daily record datas.
Can but be not limited in storage circulation, ask; Do not get rid of first request during practical application, the scheme of preserving database again in circulating to storage yet.
The described daily record data that did not extract is the daily record data of new the type that receives after the request of last stored device, may be one or more; The daily record data that is extracted is stored in the step in the database, can be disposable the storing in the database of daily record data of extracting at every turn.
Owing to can once ask many, database is advanced in many disposable storages of asking, therefore can significantly improve memory property, the lifting that allows the processing speed of daily record data produce matter.
The storage means of embodiment seven, a kind of daily record data except the content of embodiment six, also comprises before the step that receives daily record data: buffer memory is divided into and a plurality of one to one sub-buffer areas of all types of daily record datas; Other realizes that details can be with embodiment six.
In the present embodiment, each described sub-buffer area can but be not limited to an annular data pool; Can be array or chained list; In the practical application, do not get rid of other form of employing yet and realize sub-buffer area.
In the present embodiment, the daily record data that is received is saved in the buffer memory is meant: the daily record data that is received is saved in the pairing sub-buffer area of daily record data of the type;
The daily record data that extracts the institute's request type that did not extract in the described buffer memory is meant: in the pairing sub-buffer area of the daily record data of institute's request type, and the daily record data that did not extract.
When practical application, if buffer memory unallocated be sub-buffer area, also can when preserving daily record data, preserve the sign that is used to indicate its type simultaneously; Like this when extracting the daily record data of institute's request type, can according to described sign find the daily record data of request type to some extent.
Embodiment eight, and a kind of storage means of daily record data except the content of embodiment seven, also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
When daily record data being saved in the sub-buffer area, according to the buffer memory indication information of this sub-buffer area the daily record data that is received is saved in the described writing position, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
From above as seen, described buffer memory indication information is corresponding one by one with described sub-buffer area, and in fact just the daily record data with all types of is corresponding one by one.
When practical application, do not get rid of the situation of not preserving the buffer memory indication information yet, realize sub-buffer area such as forms such as adopting FIFO (first-in first-out register), storehouse, emptying sub-buffer area behind each extraction daily record data, needs just all extract the daily record data in the sub-buffer area when extracting and get final product; Or set up one for every daily record data and extract zone bit, whether extracted to indicate this daily record data.
Other realizes that details can be with embodiment six or seven.
The storage means of embodiment nine, a kind of daily record data except the content of embodiment seven, also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information comprises write pointer p_write and read pointer p_read respectively, described write pointer p_writ is used for indicating the writing position of corresponding sub-buffer area daily record data, and described read pointer p_read is used for indicating the reading location of corresponding sub-buffer area daily record data.
When daily record data being saved in the sub-buffer area, received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
Such as the sub-buffer area of hypothesis is the array form; During extraction, write pointer is 6, points to the 6th array, and read pointer is 3, points to the 3rd array, then extracts daily record data since the 3rd array, extracts to stop behind the 3rd, 4,5 daily record data in the array extracting; Read pointer is increased by 3, become 6, this also just means will be since the 6th array when extracting next time.
During practical application, do not get rid of the buffer memory indication information that adopts other form yet, such as the writing position of preserving and the address of reading location with register or other memory device, or the writing position of preserving and the sign of reading location etc.The buffer memory indication information can save as the two parts that are respectively applied for indication reading and writing position, also can be an integral body, and wherein a part is used to indicate writing position, and another part is used to indicate reading location.
During practical application, also can be that daily record data of every extraction just adds one with described read pointer, judge then whether read pointer and write pointer equate, are then to stop to extract; Otherwise the daily record data in the position that continuation extraction read pointer points to.The scheme of extracting the disposable increase read pointer in back is can efficient higher, but need count the bar number of the daily record data that extracted.
In the present embodiment, can but be not limited to adopt a buffer pointers CachePointer array to preserve the reading and writing pointer that corresponds respectively to each sub-buffer area; During practical application, do not get rid of the independent reading and writing pointer of preserving the daily record data of each sub-buffer area correspondence yet, or adopt other form to concentrate and preserve, such as the mapping table of sub-buffer area sign and reading and writing pointer etc.
Other realizes that details can be with embodiment six or seven.
The storage means of embodiment ten, a kind of daily record data except the content of above arbitrary embodiment, also comprises:
Monitoring when monitoring that each described buffer memory indication information moves the read-write situation of various types of daily record datas is with instant understanding, tracker ruuning situation;
Other realizes that details can be with above arbitrary embodiment.
Embodiment 11, a kind of storage means of daily record data; In the present embodiment, described buffer queue management device KlogCache, receiver KReceiver, storer KDBCarriage all can the thread form realize; In the practical application, also can adopt hardware to realize.
The sequential of concrete treatment scheme as shown in Figure 2.
201, receiver KReceive receives daily record data from server;
202, described receiver calls AddLogCache (KLogData*logData, ENUM_DATE_CATEGORY enFrom), and daily record data is saved in sub-buffer area corresponding with the type of this daily record data in the described buffer queue management device;
203, described receiver calls Increase WritePointer (ENUM_LOG_TYPE enLogType, ENUM_DATE_CATEGORY enFrom), the write pointer reach that sub-buffer area that will be corresponding with the type of this daily record data is corresponding;
204, storer KDBCarriage periodically calls BuildeSqlValue (MY SQL*conn, ENUM_LOG_TYPE enLogType, int nIdx, char*pChar), from described buffer queue management device in the sub-buffer area corresponding with the daily record data of a certain type, extract one or many daily record datas be configured among the character string pChar;
205, described memory calls SaveLogItems function carries out database storing;
206, described memory calls SetReadPointer (ENUM_LOG_TYPE enLogType, int nDescPos), read pointer that will be corresponding with this sub-buffer area moves to the relevant position according to the record number that reads.
During practical application, be not limited to use above cited function, as long as can finish function of the same race.
Certainly; the present invention also can have other various embodiments; under the situation that does not deviate from spirit of the present invention and essence thereof; those of ordinary skill in the art work as can make various corresponding changes and distortion according to the present invention, but these corresponding changes and distortion all should belong to the protection domain of claim of the present invention.
Claims (12)
1. the memory storage of a daily record data comprises the receiver that is used to receive daily record data; It is characterized in that, also comprise:
The buffer queue management device;
Buffer memory;
Storer is used for asking dissimilar daily record datas in turn to described buffer queue management device, is stored in the database after receiving daily record data;
The daily record data that described buffer queue management device is used for receiving is saved in described buffer memory; After the request of receiving described storer, extract the daily record data of the institute's request type that did not extract in the described buffer memory, send to described storer.
2. memory storage as claimed in claim 1 is characterized in that:
Described storer is received to be stored in the database behind the daily record data and is meant: after described storer is received daily record data at every turn, store in the database the daily record data of being received is disposable.
3. memory storage as claimed in claim 1 is characterized in that, described buffer memory comprises:
With a plurality of one to one sub-buffer areas of all types of daily record datas;
Described buffer queue management device is saved in the daily record data that receives in the described buffer memory and is meant: described buffer queue management device is saved in the daily record data that receives in the pairing sub-buffer area of daily record data of the type;
The daily record data that described buffer queue management device extracts the institute's request type that did not extract in the described buffer memory is meant: described buffer queue management device in the pairing sub-buffer area of the daily record data of institute's request type, the daily record data that did not extract.
4. memory storage as claimed in claim 3 is characterized in that:
Described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
Described buffer queue management device is saved in the daily record data that is received in the described writing position according to the buffer memory indication information of this sub-buffer area when daily record data being saved in the sub-buffer area, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
5. memory storage as claimed in claim 3 is characterized in that:
Described buffer queue management device is preserved each sub-buffer area corresponding cache indication information respectively; Described buffer memory indication information comprises write pointer and read pointer; Described write pointer is used for indicating the writing position of corresponding sub-buffer area daily record data; Described read pointer is used for indicating the reading location of corresponding sub-buffer area daily record data;
Described buffer queue management device received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one when daily record data being saved in the sub-buffer area; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
6. memory storage as claimed in claim 5 is characterized in that:
Described buffer queue management device adopts a buffer pointers array to preserve the reading and writing pointer that corresponds respectively to each sub-buffer area.
7. the storage means of a daily record data comprises:
Receive daily record data, the daily record data that is received is saved in the buffer memory;
Ask dissimilar daily record datas in turn;
Extract the daily record data of the institute's request type that did not extract in the described buffer memory, store in the database.
8. method as claimed in claim 7 is characterized in that:
The daily record data that is extracted is stored in the step in the database, is disposable the storing in the database of daily record data of extracting at every turn.
9. method as claimed in claim 7 is characterized in that, also comprises before the step that receives daily record data:
Buffer memory is divided into and a plurality of one to one sub-buffer areas of all types of daily record datas;
The daily record data that is received is saved in the buffer memory is meant: the daily record data that is received is saved in the pairing sub-buffer area of daily record data of the type;
The daily record data that extracts the institute's request type that did not extract in the described buffer memory is meant: in the pairing sub-buffer area of the daily record data of institute's request type, and the daily record data that did not extract.
10. method as claimed in claim 9 is characterized in that, also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information is used for indicating the writing position and the reading location of corresponding sub-buffer area daily record data;
When daily record data being saved in the sub-buffer area, according to the buffer memory indication information of this sub-buffer area the daily record data that is received is saved in the described writing position, and this sub-buffer area corresponding cache indication information of corresponding modify; When from sub-buffer area, extracting daily record data, buffer memory indication information according to this sub-buffer area extracts from described reading location to the daily record data the described writing position, comprise the daily record data in the reading location, do not comprise the daily record data in the writing position, and this sub-buffer area corresponding cache indication information of corresponding modify.
11. method as claimed in claim 9 is characterized in that, also comprises:
Preserve each sub-buffer area corresponding cache indication information respectively; Each described buffer memory indication information comprises write pointer and read pointer respectively, and described write pointer is used for indicating the writing position of corresponding sub-buffer area daily record data, and described read pointer is used for indicating the reading location of corresponding sub-buffer area daily record data;
When daily record data being saved in the sub-buffer area, received daily record data is saved in the write pointer of this sub-buffer area position pointed, and the write pointer of this sub-buffer area adds one; When from sub-buffer area, extracting daily record data, begin to extract daily record data from the read pointer of this sub-buffer area position pointed, stop to extract when extracting the write pointer of this sub-buffer area position pointed always, and after extraction according to the read pointer of corresponding this sub-buffer area of increase of bar number of the daily record data that reads.
12. method as claimed in claim 11 is characterized in that:
The described reading and writing pointer that corresponds respectively to each sub-buffer area is kept in the buffer pointers array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102372031A CN102053923B (en) | 2009-11-05 | 2009-11-05 | Storage method and storage device for logbook data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102372031A CN102053923B (en) | 2009-11-05 | 2009-11-05 | Storage method and storage device for logbook data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102053923A true CN102053923A (en) | 2011-05-11 |
| CN102053923B CN102053923B (en) | 2013-06-12 |
Family
ID=43958268
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102372031A Active CN102053923B (en) | 2009-11-05 | 2009-11-05 | Storage method and storage device for logbook data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102053923B (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368210A (en) * | 2011-10-25 | 2012-03-07 | 曙光信息产业(北京)有限公司 | Pipeline based metadata journaling method for distributed system |
| CN102637142A (en) * | 2012-04-13 | 2012-08-15 | 浪潮(北京)电子信息产业有限公司 | Computer system and method for realizing log management |
| CN103138964A (en) * | 2011-11-25 | 2013-06-05 | 上海晨兴希姆通电子科技有限公司 | Log information recording device and recording method thereof |
| CN103200037A (en) * | 2013-04-11 | 2013-07-10 | 深圳市共进电子股份有限公司 | System log (syslog) storing method |
| CN103488717A (en) * | 2013-09-11 | 2014-01-01 | 北京华胜天成科技股份有限公司 | Lock-free data gathering method and lock-free data gathering device |
| CN104657366A (en) * | 2013-11-18 | 2015-05-27 | 深圳市腾讯计算机系统有限公司 | Method and device for writing mass logs in database and log disaster-tolerant system |
| CN105224444A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | Daily record generation method and device |
| CN106027595A (en) * | 2016-04-25 | 2016-10-12 | 乐视控股（北京）有限公司 | Access log processing method and system for CDN node |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| CN106919515A (en) * | 2015-12-24 | 2017-07-04 | 北京谊安医疗系统股份有限公司 | A kind of daily record linked list data reading/writing method in memory |
| CN108572955A (en) * | 2017-03-07 | 2018-09-25 | 大唐移动通信设备有限公司 | A kind of journal file generation method and device |
| CN109743757A (en) * | 2018-12-29 | 2019-05-10 | 深圳和而泰数据资源与云技术有限公司 | Data processing method, device, wireless module and internet of things equipment |
| CN109992572A (en) * | 2019-03-06 | 2019-07-09 | 华南理工大学 | A kind of method of adaptive equalization log storage request |
| WO2019134341A1 (en) * | 2018-01-05 | 2019-07-11 | 平安科技（深圳）有限公司 | Log text processing method and apparatus, and storage medium |
| CN110716695A (en) * | 2019-09-12 | 2020-01-21 | 北京浪潮数据技术有限公司 | Node log storage method and system, electronic device and storage medium |
| CN111984618A (en) * | 2020-08-20 | 2020-11-24 | 北京人大金仓信息技术股份有限公司 | Log compression method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1677931A (en) * | 2004-04-02 | 2005-10-05 | 鸿富锦精密工业（深圳）有限公司 | Network daily-record data management system and method |
| US20060106891A1 (en) * | 2004-11-18 | 2006-05-18 | International Business Machines (Ibm) Corporation | Managing atomic updates on metadata tracks in a storage system |
| CN1829366A (en) * | 2005-03-02 | 2006-09-06 | 华为技术有限公司 | Wireless network controller log recording system and its method |
| CN101272276A (en) * | 2008-04-22 | 2008-09-24 | 深圳国人通信有限公司 | Log management implementing method |
-
2009
- 2009-11-05 CN CN2009102372031A patent/CN102053923B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1677931A (en) * | 2004-04-02 | 2005-10-05 | 鸿富锦精密工业（深圳）有限公司 | Network daily-record data management system and method |
| US20060106891A1 (en) * | 2004-11-18 | 2006-05-18 | International Business Machines (Ibm) Corporation | Managing atomic updates on metadata tracks in a storage system |
| CN1829366A (en) * | 2005-03-02 | 2006-09-06 | 华为技术有限公司 | Wireless network controller log recording system and its method |
| CN101272276A (en) * | 2008-04-22 | 2008-09-24 | 深圳国人通信有限公司 | Log management implementing method |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368210A (en) * | 2011-10-25 | 2012-03-07 | 曙光信息产业(北京)有限公司 | Pipeline based metadata journaling method for distributed system |
| CN103138964A (en) * | 2011-11-25 | 2013-06-05 | 上海晨兴希姆通电子科技有限公司 | Log information recording device and recording method thereof |
| CN102637142A (en) * | 2012-04-13 | 2012-08-15 | 浪潮(北京)电子信息产业有限公司 | Computer system and method for realizing log management |
| CN103200037A (en) * | 2013-04-11 | 2013-07-10 | 深圳市共进电子股份有限公司 | System log (syslog) storing method |
| CN103488717A (en) * | 2013-09-11 | 2014-01-01 | 北京华胜天成科技股份有限公司 | Lock-free data gathering method and lock-free data gathering device |
| CN103488717B (en) * | 2013-09-11 | 2017-02-22 | 北京华胜天成科技股份有限公司 | Lock-free data gathering method and lock-free data gathering device |
| CN104657366A (en) * | 2013-11-18 | 2015-05-27 | 深圳市腾讯计算机系统有限公司 | Method and device for writing mass logs in database and log disaster-tolerant system |
| CN104657366B (en) * | 2013-11-18 | 2019-05-24 | 深圳市腾讯计算机系统有限公司 | The method, apparatus and log disaster tolerance system of massive logs write-in database |
| CN106033322B (en) * | 2015-03-17 | 2019-11-01 | 北京元心科技有限公司 | A kind of date storage method and device |
| CN106033322A (en) * | 2015-03-17 | 2016-10-19 | 北京元心科技有限公司 | Method and device for data storage |
| CN105224444B (en) * | 2015-10-14 | 2019-02-01 | 深圳市金证科技股份有限公司 | Log generation method and device |
| CN105224444A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | Daily record generation method and device |
| CN106919515A (en) * | 2015-12-24 | 2017-07-04 | 北京谊安医疗系统股份有限公司 | A kind of daily record linked list data reading/writing method in memory |
| CN106919515B (en) * | 2015-12-24 | 2020-09-18 | 北京谊安医疗系统股份有限公司 | Method for reading and writing log linked list data in memory |
| CN106027595A (en) * | 2016-04-25 | 2016-10-12 | 乐视控股（北京）有限公司 | Access log processing method and system for CDN node |
| CN108572955A (en) * | 2017-03-07 | 2018-09-25 | 大唐移动通信设备有限公司 | A kind of journal file generation method and device |
| WO2019134341A1 (en) * | 2018-01-05 | 2019-07-11 | 平安科技（深圳）有限公司 | Log text processing method and apparatus, and storage medium |
| CN109743757A (en) * | 2018-12-29 | 2019-05-10 | 深圳和而泰数据资源与云技术有限公司 | Data processing method, device, wireless module and internet of things equipment |
| CN109743757B (en) * | 2018-12-29 | 2022-04-12 | 深圳和而泰数据资源与云技术有限公司 | Data processing method and device, wireless module and Internet of things equipment |
| CN109992572A (en) * | 2019-03-06 | 2019-07-09 | 华南理工大学 | A kind of method of adaptive equalization log storage request |
| CN110716695A (en) * | 2019-09-12 | 2020-01-21 | 北京浪潮数据技术有限公司 | Node log storage method and system, electronic device and storage medium |
| CN111984618A (en) * | 2020-08-20 | 2020-11-24 | 北京人大金仓信息技术股份有限公司 | Log compression method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102053923B (en) | 2013-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102053923B (en) | Storage method and storage device for logbook data | |
| CN101673192B (en) | Method for time-sequence data processing, device and system therefor | |
| CN102158349A (en) | Log management device and method thereof | |
| CN102663086A (en) | Method for retrieving data block indexes | |
| CN102521419A (en) | Hierarchical storage realization method and system | |
| CN104615684B (en) | A kind of mass data communication concurrent processing method and system | |
| CN101839997A (en) | Seismic data parallel storage system | |
| CN100449504C (en) | Buffer management based on bitmap list | |
| CN105787012A (en) | Method for improving small file processing capability of storage system and storage system | |
| CN106681660B (en) | IO scheduling method and IO scheduling device | |
| CN105159616A (en) | Disk space management method and device | |
| TW200708950A (en) | Memory management method and system | |
| CN104391653A (en) | Data block-based cache design method | |
| CN103985393A (en) | Method and device for parallel management of multi-optical-disc data | |
| CN109241187A (en) | MongoDB log collection and analysis system based on Kafka message queue | |
| CN111752905B (en) | Large file distributed cache system based on object storage | |
| CN104156395A (en) | Data storage system | |
| CN104158863A (en) | Cloud storage mechanism based on transaction-level whole-course high-speed buffer | |
| CN101420233B (en) | Bit interleaver and interleaving method | |
| CN104714898B (en) | A kind of distribution method and device of Cache | |
| CN103123575A (en) | Data write-in method | |
| CN102637148B (en) | DDR SDRAM (double data rate synchronous dynamic random-access memory) based stacked data caching device and method thereof | |
| CN111427920B (en) | Data acquisition method, device, system, computer equipment and storage medium | |
| CN116257191B (en) | Memory controller, memory component, electronic device and command scheduling method | |
| CN103236938A (en) | Method and system for user action collection based on cache memory and asynchronous processing technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |