CN102567395A - Signature server and control method thereof - Google Patents

Signature server and control method thereof Download PDF

Info

Publication number
CN102567395A
CN102567395A CN2010106159263A CN201010615926A CN102567395A CN 102567395 A CN102567395 A CN 102567395A CN 2010106159263 A CN2010106159263 A CN 2010106159263A CN 201010615926 A CN201010615926 A CN 201010615926A CN 102567395 A CN102567395 A CN 102567395A
Authority
CN
China
Prior art keywords
signature
checked
newly added
memory
inquire
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN2010106159263A
Other languages
Chinese (zh)
Inventor
蒋锦鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN2010106159263A priority Critical patent/CN102567395A/en
Publication of CN102567395A publication Critical patent/CN102567395A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a signature server and a control method thereof. The control method includes the following steps: receiving a to-be-inquired signature; inquiring the to-be-inquired signature in a first storer; obtaining a corresponding data block from a second storer if the to-be-inquired signature is not found in the first storer, and inquiring the to-be-inquired signature in the corresponding data block. By means of the mode, performance of the signature server is further improved, and speed of signature inquiring is quickened. Further, by means of compact storage, storage space is saved by 50%, storage space is effectively utilized, and continuous operation of service is guaranteed by combining and reconstructing mechanisms. Furthermore, the signature server can also be used for removing reduplication of web pages or verifying normalization of links.

Description

A kind of signature server and control method thereof
Technical field
The present invention relates to internet arena, particularly a kind of signature server and control method thereof.
Background technology
Along with development of science and technology, Internet technology is fast development thereupon also, and search engine must be widely used in numerous Internet users gradually.Search engine is ageing in order to promote, and must carry out a large amount of calculating in real time.In calculating in real time, run into a problem: judge whether a link belongs to a link set through regular meeting.For example, judge whether a link is included by search engine, judge whether a web page address after the link conversion is present in real internet, or not repetition.The judgment models of this type problem can be classified as judges that a signature is whether in a signature set.
But the data volume of the signature set of search engines store is very big, even can reach several hundred billion magnitudes.The requirement of query performance is also very high, need to support the inquiry of per second 1,000,000 magnitudes just can guarantee the speed of inquiring about, and will support to add in real time signature.In addition, in order to guarantee the ageing of search engine, also need uninterrupted service.Therefore, existing signature server is slightly not enough on the signature inquiry velocity, and the space availability ratio of storage signature is not very high.
Summary of the invention
The invention provides a kind of signature server and control method thereof, can accelerate the inquiry velocity of signing.
Concrete technical scheme is following:
The embodiment of the invention provides a kind of signature server and control method thereof, and this control method comprises: a. receives signature to be checked; B. in first memory, inquire about signature to be checked; C. if in first memory, do not inquire signature to be checked, then obtain the corresponding data piece, and in the corresponding data piece, inquire about signature to be checked from second memory.
According to one preferred embodiment of the present invention, further comprise among the step b: inquiry signature to be checked in the newly added signature dictionary of first memory, if in the newly added signature dictionary, inquire signature to be checked, then return signature and have information.
According to one preferred embodiment of the present invention, step c further comprises: if in the corresponding data piece, inquire signature to be checked, then return signature and have information; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the newly added signature dictionary.
According to one preferred embodiment of the present invention, in first enquiry module,, then in the buffer memory of first memory, inquire about signature to be checked if in the newly added signature dictionary, do not inquire signature to be checked; If in buffer memory, inquire signature to be checked, then return signature and have information.
According to one preferred embodiment of the present invention, step c further comprises: if in the corresponding data piece, inquire signature to be checked, then signature to be checked is write buffer memory, and return signature and have information; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the newly added signature dictionary.
According to one preferred embodiment of the present invention, in first enquiry module,, judge then whether signature server is in merging reconstruction state if in the newly added signature dictionary, do not inquire signature to be checked; If judging to be in merges the reconstruction state, then inquiry signature to be checked in the new newly added signature dictionary of first memory if in new newly added signature dictionary, inquire signature to be checked, then returns signature and has information.
According to one preferred embodiment of the present invention; Step c further comprises: if in the corresponding data piece, inquire signature to be checked; Then return signature and have information,, then signature to be checked is added in the new newly added signature dictionary if in the corresponding data piece, do not inquire signature to be checked.
According to one preferred embodiment of the present invention; In first enquiry module,, then in the buffer memory of first memory, inquire about signature to be checked if in new newly added signature dictionary, do not inquire about signature to be checked; If in buffer memory, inquire signature to be checked, then return signature and have information.
According to one preferred embodiment of the present invention; Step c comprises: if in the corresponding data piece, inquire signature to be checked; Then return signature and have information; And signature to be checked write buffer memory, if in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the new newly added signature dictionary.
According to one preferred embodiment of the present invention, step c is included in the stored position information of inquiry corresponding data piece in the index in the first memory, obtains the corresponding data piece according to stored position information.
According to one preferred embodiment of the present invention; The a plurality of data blocks of storage in the second memory; Comprise a plurality of signatures in each data block, the signature in a plurality of data blocks is pressed the signature contents sequential storage, stores the index relative of stored position information of signature and data block of the boundary of data block in the index.
According to one preferred embodiment of the present invention, in each data block, signature carries out compression memory.
According to one preferred embodiment of the present invention, in each data block,, store first signature itself,, store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the first signature of follow-up signature to follow-up signature to first signature.
According to one preferred embodiment of the present invention, control method further comprises: d. utilizes the newly added signature dictionary to generate the newly added signature file, and merges reconstruction with the backup file of data block, to generate new backup file.
According to one preferred embodiment of the present invention, steps d further comprises: before utilizing newly added signature dictionary generation newly added signature file, generate new newly added signature dictionary.
According to one preferred embodiment of the present invention, control method further comprises: utilize new backup file Update Information piece and index, and utilize new newly added signature dictionary replacement newly added signature dictionary.
According to one preferred embodiment of the present invention, newly added signature file and backup file are stored in the 3rd storer.
According to one preferred embodiment of the present invention, the 3rd storer is a hard disk.
According to one preferred embodiment of the present invention, first memory is an internal memory, and second memory is a solid-state memory.
The present invention further provides a kind of signature server, and this signature server comprises: first memory and second memory; Receive the receiver module of signature to be checked; First enquiry module of inquiry signature to be checked in first memory; If in first memory, do not inquire signature to be checked, then obtain the corresponding data piece, and in the corresponding data piece, inquire about second enquiry module of signature to be checked from second memory.
According to one preferred embodiment of the present invention; Signature server comprises that further information returns module; First enquiry module is inquired about signature to be checked in the newly added signature dictionary of first memory, if in the newly added signature dictionary, inquire signature to be checked, then information is returned module and returned signature and have information.
According to one preferred embodiment of the present invention, in second enquiry module, if in the corresponding data piece, inquire signature to be checked, then information is returned module and is returned signature and have information; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the newly added signature dictionary.
According to one preferred embodiment of the present invention; Signature server further comprises the query module; In first enquiry module, if in the newly added signature dictionary, do not inquire signature to be checked, then the query module is inquired about signature to be checked in the buffer memory of first memory; If in buffer memory, inquire signature to be checked, then information is returned module and is returned signature and have information.
According to one preferred embodiment of the present invention, in second enquiry module, if in the corresponding data piece, inquire signature to be checked, then signature to be checked is write buffer memory, and return signature and have information; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the newly added signature dictionary.
According to one preferred embodiment of the present invention; In first enquiry module, signature server further comprises the 4th enquiry module, in first enquiry module; If in the newly added signature dictionary, do not inquire signature to be checked, judge then whether signature server is in merging reconstruction state; If judging to be in merges the reconstruction state, then the 4th enquiry module inquire about signature to be checked in the new newly added signature dictionary of first memory, if in new newly added signature dictionary, inquire signature to be checked, then information is returned module and returned to sign and have information.
According to one preferred embodiment of the present invention; In second enquiry module, if in the corresponding data piece, inquire signature to be checked, then information is returned module and is returned signature and have information; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the new newly added signature dictionary.
According to one preferred embodiment of the present invention; Signature server further comprises the query module; In the 4th enquiry module, if in new newly added signature dictionary, do not inquire about signature to be checked, then the query module is inquired about signature to be checked in the buffer memory of first memory; If in buffer memory, inquire signature to be checked, then information is returned module and is returned signature and have information.
According to one preferred embodiment of the present invention; In second enquiry module; If in the corresponding data piece, inquire signature to be checked, then information is returned module and is returned signature and have information, and signature to be checked is write buffer memory; If in the corresponding data piece, do not inquire signature to be checked, then signature to be checked is added in the new newly added signature dictionary.
According to one preferred embodiment of the present invention, second enquiry module is included in the stored position information of inquiry corresponding data piece in the index in the first memory, obtains the corresponding data piece according to stored position information.
According to one preferred embodiment of the present invention; The a plurality of data blocks of storage in the second memory; Comprise a plurality of signatures in each data block, the signature in a plurality of data blocks is pressed the signature contents sequential storage, stores the index relative of stored position information of signature and data block of the boundary of data block in the index.
According to one preferred embodiment of the present invention, in each data block, signature carries out compression memory.
According to one preferred embodiment of the present invention, in each data block,, store first signature itself,, store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the first signature of follow-up signature to follow-up signature to first signature.
According to one preferred embodiment of the present invention, signature server further comprises: utilize the newly added signature dictionary to generate the newly added signature file, and merge reconstruction with the backup file of data block, to generate the merging module of new backup file.
According to one preferred embodiment of the present invention, before utilizing newly added signature dictionary generation newly added signature file, generate new newly added signature dictionary.
According to one preferred embodiment of the present invention, signature server further comprises: utilize new backup file Update Information piece and index, and utilize the update module of new newly added signature dictionary replacement newly added signature dictionary.
According to one preferred embodiment of the present invention, newly added signature file and backup file are stored in the 3rd storer.
According to one preferred embodiment of the present invention, the 3rd storer is a hard disk.
According to one preferred embodiment of the present invention, first memory is an internal memory, and second memory is a solid-state memory.
Can find out that by above technical scheme the performance of signature server of the present invention is further promoted, accelerate the signature inquiry velocity.In addition, make storage space save 50%, effectively utilize storage space, and ensured that through merging re-establishment mechanism service runs without interruption through compression memory.And this signature server also can be used for removing duplicate webpages or link normalization checking.
Description of drawings
Fig. 1 is the schematic flow sheet of first embodiment of the control method of a kind of signature server of the present invention.
Fig. 2 is the schematic flow sheet of second embodiment of the control method of a kind of signature server of the present invention.
Fig. 3 is the synoptic diagram of a kind of data block compression method of the present invention.
Fig. 4 is the schematic flow sheet of the 3rd embodiment of the control method of a kind of signature server of the present invention.
Fig. 5 is the schematic flow sheet that a kind of signed data of the present invention merges method for reconstructing.
Fig. 6 is that signature carries out another synoptic diagram that data merge method for reconstructing among Fig. 5.
Fig. 7 is the schematic flow sheet of the 4th embodiment of the control method of a kind of signature server of the present invention.
Fig. 8 is the schematic block diagram of first embodiment of a kind of signature server of the present invention.
Fig. 9 is the schematic block diagram of second embodiment of a kind of signature server of the present invention.
Figure 10 is the schematic block diagram of the 3rd embodiment of a kind of signature server of the present invention.
Embodiment
In order to make the object of the invention, technical scheme and advantage clearer, describe the present invention below in conjunction with accompanying drawing and specific embodiment.
As shown in Figure 1, Fig. 1 is the schematic flow sheet of first embodiment of the control method of a kind of signature server of the present invention.
In step 10, receive signature to be checked.This signature is obtained through various signature algorithms well known in the art by web page address or link, repeats no more at this.Certainly, signature also can be by other similar data-switching.
In step 11, inquiry signature to be checked in first memory.
In step 12, judge whether in first memory, to inquire signature to be checked.If in first memory, inquire signature to be checked, then get into step 13.If in first memory, do not inquire signature to be checked, then get into step 14.
In step 13, return signature and have information.Further, if signature exists, explain that then this web page address or link are existing, can browse.Perhaps,, inquire about this signature of discovery and exist, explain that then webpage repeats, and should carry out removing duplicate webpages if this signature is the signature of up-to-date appearance.
In step 14, obtain the corresponding data piece from second memory, and in the corresponding data piece, inquire about said signature to be checked.
As shown in Figure 2, Fig. 2 is the schematic flow sheet of second embodiment of the control method of a kind of signature server of the present invention.
In step 100, receive signature to be checked.This signature is obtained through various signature algorithms well known in the art by web page address or link, repeats no more at this.Certainly, signature also can be by other similar data-switching.
In step 101, inquiry signature to be checked in the newly added signature dictionary of first memory, with judge should signature whether in the newly added signature dictionary.If in the newly added signature dictionary, inquire signature to be checked, then get into step 102.If in the newly added signature dictionary, do not inquire signature to be checked, then get into step 104.In the present embodiment, the newly added signature dictionary can adopt plurality of data structures to realize, for example Hash table (hash table), RBTree, Bloom filter (bloom filter), B+ tree etc. are repeated no more at this.
In step 102, return signature and have information.Further, if signature exists, explain that then this web page address or link are existing, can browse.Perhaps,, inquire about this signature of discovery and exist, explain that then webpage repeats, and should carry out removing duplicate webpages if this signature is the signature of up-to-date appearance.
In step 104, the stored position information of the corresponding data piece that inquiry signature to be checked is stored in second memory in the index of first memory.This index can pass through plurality of data structures such as Hash table (hash table), RBTree, Bloom filter (bloom filter), B+ tree equally to be realized, repeats no more at this.
In the present embodiment, newly added signature dictionary and index stores are in first memory, and the corresponding data piece then is stored in the second memory.First memory is preferably internal memory, and second memory is preferably solid-state memory, for example flash memory (Flash Memory).The a plurality of data blocks of storage in this second memory, each data block internal memory contains a plurality of signatures, and the signature in these a plurality of data blocks is pressed the signature contents sequential storage.Then store the index relative of stored position information of signature and data block of the boundary of each data block in the index.Therefore,, can judge the corresponding data piece of signature to be checked, and then obtain the stored position information of corresponding data piece through the signature in signature more to be checked and the index.In the present embodiment, stored position information comprises filec descriptor, data block length information of data block identifying information, solid-state memory etc.In a preferred embodiment, if these a plurality of data blocks be continuously and length fixing, then then can only store signature and the stored position information of first data block of the boundary of each data block in the index.At this moment,, judge the sequence number of the corresponding data piece of signature to be checked, calculate the stored position information of corresponding data piece according to the stored position information of this sequence number, data block length and first data block through the signature in signature more to be checked and the index.
In step 105, obtain the corresponding data piece in the second memory according to stored position information, and in the corresponding data piece, inquire about signature to be checked, whether be stored in the corresponding data piece to judge signature to be checked.If in the corresponding data piece, inquire signature to be checked, then get into step 102.If in the corresponding data piece, do not inquire signature to be checked, then get into step 106.
In step 106, return signature and do not have information, and signature to be checked is added in the newly added signature dictionary.
In step 105, because signature is pressed the signature contents sequential storage in data block, therefore available binary chop is inquired about.Binary chop is applicable to the inquiry that data volume is very big, its restrictive condition then is that data must be through ordering, the inquiry that therefore is used for the signature in the data block be again appropriate only.The basic thought of binary chop is: the tentation data sequence is by ascending sort, for the given value of searching, begins comparison from the centre position of data sequence, if the current location value equals the value of searching, and successful inquiring then.If the value of searching less than the current location value, is then inquired about in preceding half section of data sequence.If the value of searching greater than the current location value, then continues inquiry, till finding in the second half section of data sequence.Use the method for binary chop to inquire about, can further practice thrift the used time of inquiry.
In like manner; In step 104; Because the signature in the data block is a sequential storage; Therefore the signature in the index also is a sequential storage, therefore also can use simple binary chop to come to navigate to fast corresponding which data block of signature to be checked, so and then obtains the stored position information of corresponding data piece.
Except simple binary chop, plurality of data structures such as all right Hash table (hash table), RBTree, Bloom filter (bloom filter), B+ tree realize repeating no more the query script of signature in data block at this.
In the prior art, if carry out binary chop on all signed datas in solid-state memory, then will cause read (N is all number of signatures) of the inferior solid-state memory of log (N), this will cause very big influence to the signature server performance.Yet, in the present embodiment, earlier in solid-state memory, seek scope is narrowed down to the data block rank through the internal memory index, read into memory whole data block is disposable again.Doing one query like this will only need once read solid-state memory.
For conserve storage, in each data block, preferably signature is carried out compression memory.
As shown in Figure 3, Fig. 3 is the synoptic diagram of a kind of data block compress mode of the present invention.In the present embodiment, data block comprises signature number, signature 1, signature 2, signature 3 or the like.Wherein, the signature number is represented the number of the interior signature of being stored of notebook data piece.In a preferred embodiment, the data block size is 2KB, and this is because can be known that by the ardware feature of solid-state memory the solid-state memory least unit of reading of data at random is 2KB.That is to say that reading the 2KB data from solid-state memory is suitable with the expense that reads 1 byte data.In the present embodiment, because the data block size is merely 2KB, therefore only need 16bit to store the signature number.
In the compression process of signature, to the first signature (for example, signature 1) of each data block, directly store signature itself, for example utilize 8 bytes (64bit) to store signature itself.To follow-up signature (for example, signature 2, signature 3), store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the first signature of follow-up signature.In the present embodiment, to follow-up signature, store high 32 high 32 differences with respect to first signature of follow-up signature, that directly stores follow-up signature again hangs down 32.
After the compression of signature, comprise: difference data length, high 32 potential difference values and low 32 in the storage data of each follow-up signature.In the present embodiment, utilize 2bit to represent 4 difference data length:
00: high 32 potential difference value length are 0bit, and this moment, high 32 potential difference values were 0;
01: high 32 potential difference value length are 2bit, and this moment, high 32 potential difference values can be 1,2,3 or 4;
10: high 32 potential difference value length are 8bit, this moment high 32 potential difference values can be 5,6... or 260;
11: high 32 potential difference value length are 32bit, and this moment, high 32 potential difference values can be 261.......
Use this compress mode, when each data block size is 2KB, approximately can deposit 500 signatures, can practice thrift general 50% storage space.
In decompression procedure, the data that read follow-up corresponding length (0bit, 2bit, 8bit or 32bit) according to difference data length are as high 32 potential difference values, and with high 32 computings that add deduct of first signature, to obtain the high 32 of follow-up signature.Further, read follow-up 32 bit data as low 32, and with high 32 that calculate the follow-up signature that obtains before make up, and then obtain this follow-up signature.
Find that the inquiry of signature has more intense temporal locality usually in actual the use.That is to say, at short notice, inquire about same signature possibly times without number.Though the reading performance at random of solid-state memory is fine, to differ from several magnitude compared with the reading speed of internal memory.In order further to promote the performance of signature server, the present invention further provides a kind of caching mechanism (cache), and the signature of those solid-state memories that hit further improves inquiry velocity through buffer memory.
As shown in Figure 4, Fig. 4 is the schematic flow sheet of the 3rd embodiment of the control method of a kind of signature server of the present invention.
In step 300, receive signature to be checked.This signature is obtained through various signature algorithms well known in the art by web page address or link, repeats no more at this.Certainly, signature also can be by other similar data-switching.
In step 301, inquiry signature to be checked in the newly added signature dictionary of first memory, with judge should signature whether in the newly added signature dictionary.If in the newly added signature dictionary, inquire signature to be checked, then get into step 302.If in the newly added signature dictionary, do not inquire signature to be checked, then get into step 303.In the present embodiment, the newly added signature dictionary can adopt plurality of data structures to realize, for example Hash table (hash table), RBTree, Bloom filter (bloomfilter), B+ tree etc. are repeated no more at this.
In step 302, return signature and have information.Further, if signature exists, explain that then this web page address or link are existing, can browse.Perhaps,, inquire about this signature of discovery and exist, explain that then webpage repeats, and should carry out removing duplicate webpages if this signature is the signature of up-to-date appearance.
In step 303, inquiry signature to be checked in the buffer memory of first memory is to judge that signature to be checked is whether in buffer memory.If in buffer memory, inquire signature to be checked, then get into step 302.If do not inquire signature to be checked, then get into step 304 at buffer memory.
In step 304, the stored position information of the corresponding data piece that inquiry signature to be checked is stored in second memory in the index of first memory.In the present embodiment, newly added signature dictionary and index stores are in first memory, and the corresponding data piece then is stored in the second memory.First memory is preferably internal memory, and second memory is preferably solid-state memory.The a plurality of data blocks of storage in this second memory, each data block internal memory contains a plurality of signatures, and the signature in these a plurality of data blocks is pressed the signature contents sequential storage.Then store the index relative of stored position information of signature and data block of the boundary of each data block in the index.Therefore,, can judge the corresponding data piece of signature to be checked, and then obtain the stored position information of corresponding data piece through the signature in signature more to be checked and the index.In the present embodiment, stored position information comprises filec descriptor, data block length information of data block identifying information, solid-state memory etc.In a preferred embodiment, if these a plurality of data blocks be continuously and length fixing, the stored position information of the signature of the boundary of each data block and first data block only then in the index then.At this moment,, judge the sequence number of the corresponding data piece of signature to be checked, calculate the stored position information of corresponding data piece according to the stored position information of this sequence number, data block length and first data block through the signature in signature more to be checked and the index.
In step 305, obtain the corresponding data piece according to stored position information, and in the corresponding data piece, inquire about signature to be checked, whether be stored in the corresponding data piece to judge signature to be checked.If in the corresponding data piece, inquire signature to be checked, then get into step 302 and get into step 307.If in the corresponding data piece, do not inquire signature to be checked, then get into step 306.
In step 306, return signature and do not have information, and signature to be checked is added in the newly added signature dictionary.
In step 307, signature to be checked is write in the buffer memory.
In a preferred embodiment, be buffered in the internal memory and realize, the outer chain Hash table mode of concrete employing, and on the outer chain of Hash table, realized LRU (Least-Recently-Used) function, promptly when buffer memory is full, preferentially eliminate the signature of not looking at most.
The search algorithm of buffer memory is following: at first use signature to be checked as key assignments (key) to Hash table bucket progress row modulo operation, find corresponding bucket.Travel through the outer chained list of this barrel correspondence then, inquire about this signature to be checked.If do not find, poll-final then.If find, then be written to this node the head of outer chained list, poll-final.
Buffer memory to write algorithm following: at first use signature to be checked as key assignments (key) to Hash table bucket progress row modulo operation, find corresponding bucket.Whether the outer chained list of judging this barrel correspondence surpasses the length that limits.If the length that surpass to limit, then delete the node of the afterbody of this outer chained list.Signature to be checked is written to the head of outer chained list.
Owing to writing fashionable outer chained list head that signature to be checked is inserted into, then can the node motion that inquire be arrived outer chained list head during inquiry, can make the signature of frequent use come outer chained list front end like this, and the signature that is of little use come outer chained list tail end.When the length of outer chained list reaches restriction, will delete the signature that does not often use through above-mentioned LRU function.
The endless increase of signature in the newly added signature dictionary need regularly dump to signature in the newly added signature dictionary on the solid-state memory.
As shown in Figure 5, Fig. 5 is the schematic flow sheet that a kind of signed data of the present invention merges method for reconstructing.
In step 400, utilize the newly added signature dictionary to generate the newly added signature file.
In step 401, create new newly added signature dictionary.And, further signature server is labeled as merging reconstruction state.At this moment, if the signature to be checked that does not inquire is arranged, then add in the new newly added signature dictionary.
In step 402, the backup file of newly added signature file and data block is merged reconstruction, to generate new backup file.
In step 403, utilize new backup file to upgrade data block and the index in the internal memory in the solid-state memory.
In step 404, utilize new newly added signature dictionary replacement newly added signature dictionary.
In the present embodiment, newly added signature file and backup file are stored in the 3rd storer, and above-mentioned steps 400 and 402 is accomplished on the 3rd storer.The 3rd storer is preferably hard disk.
To combine Fig. 6 to specify signed data below and merge the detailed process of rebuilding.
Of Fig. 6, Fig. 6 is another synoptic diagram that signed data merges process of reconstruction among Fig. 5.Wherein, newly added signature dictionary 500 and index 501 are stored in the internal memory, and data block 506 is stored in the solid-state memory.Store 4 files in the hard disk, comprising: temporary file 502, newly added signature file 503, backup file 504 and new backup file 505.Temporary file 502 is used for the signature that blotter newly added signature dictionary 500 increases newly.When carrying out signed data merging reconstruction, temporary file 502 is converted into newly added signature file 503, and in newly added signature file 503, signature is sorted by signature contents.At this moment, set up new temporary file 502, with the newly-increased signature of further record.The newly added signature file 503 that ordering is good merges reconstruction with backup file 504, generates new backup file 505.Merge reconstruction algorithm and can adopt the binary merge mode.Further, according to new backup file 505, piece 506 and index 501 Update Information.Specifically; The signature sequential storage that ordering in the new backup file 505 is good is in a plurality of data blocks 506 of solid-state memory, and the index relative of the stored position information of the signature of the boundary of each data block 506 after record upgrades again in index 501 and each data block 506.Then, new backup file 505 transfers backup file 504 to, overrides original backup file 504, rebuilds for the signed data merging of next time and prepares.
In order to merge the inquiry of uninterrupted follow-up signature in the process of reconstruction and to write at signed data; In another embodiment; Merge in the process of reconstruction at signed data; Is to merge the reconstruction state by the service module of signature server with the status indication of signature server, and is normal state at other times with the status indication of signature server.Merging under the reconstruction state, signature server can regenerate a new newly added signature dictionary in internal memory, and the signature that does not inquire is added in the new newly added signature dictionary.After merging the reconstruction completion, new newly added signature dictionary is replaced original newly added signature dictionary, become existing newly added signature dictionary.Therefore, effectively avoided the endless increase of signature in the newly added signature dictionary.
As shown in Figure 7, Fig. 7 is the schematic flow sheet of the 4th embodiment of the control method of a kind of signature server of the present invention.
In step 600, receive signature to be checked.
In step 601, inquiry signature to be checked in the newly added signature dictionary is to judge that signature to be checked is whether in the newly added signature dictionary.If signature to be checked in the newly added signature dictionary, then gets into step 602, if signature to be checked not in the newly added signature dictionary, then gets into step 603.
In step 602, return signature and have information.Further, if signature exists, explain that then this web page address or link are existing, can browse.Perhaps,, inquire about this signature of discovery and exist, explain that then webpage repeats, and should carry out removing duplicate webpages if this signature is the signature of up-to-date appearance.
In step 603, judge whether signature server is in merging reconstruction state.Be not in merging reconstruction state if judge signature server, then get into step 604.Be in merging reconstruction state if judge signature server, then get into step 607.
In step 604, the stored position information of the corresponding data piece of inquiry signature to be checked in index.
In step 605, obtain the corresponding data piece according to stored position information, whether inquiry signature to be checked is stored in the corresponding data piece to judge signature to be checked in the corresponding data piece.If signature to be checked is stored in the corresponding data piece, then get into step 602.If signature to be checked is not stored in the corresponding data piece, then get into step 606.
In step 606, return signature and do not have information, and signature to be checked is added in the newly added signature dictionary.
In step 607, inquiry signature to be checked in new newly added signature dictionary is to judge that signature to be checked is whether in new newly added signature dictionary.If in new newly added signature dictionary, inquire signature to be checked, then get into step 602.If in new newly added signature dictionary, do not inquire signature to be checked, then get into step 608.
In step 608, the stored position information of the corresponding data piece of inquiry signature to be checked in index.
In step 609, obtain the corresponding data piece according to stored position information, whether inquiry signature to be checked is stored in the corresponding data piece to judge signature to be checked in the corresponding data piece.Newly added signature dictionary, new newly added signature dictionary and index stores are in first memory, and the corresponding data piece is stored in the second memory.The a plurality of data blocks of storage comprise a plurality of signatures in each data block, and the signature in these a plurality of data blocks are pressed the signature contents sequential storage in the second memory.Store the index relative of stored position information of signature and data block of the boundary of data block in the index.First memory is preferably internal memory, and second memory is preferably solid-state memory.If in the corresponding data piece, inquire signature to be checked, then get into step 602.If in the corresponding data piece, do not inquire signature to be checked, then get into step 610.
In step 610, return signature and do not have information, and signature to be checked is added in the new newly added signature dictionary.
The concrete implementation procedure of above-mentioned steps can repeat no more at this referring to first, second and the 3rd embodiment.Certainly, those skilled in the art can expect in the 4th embodiment adding the caching mechanism identical with the 3rd embodiment fully, with before carrying out index, in buffer memory, inquire about signature to be checked earlier, and judge that signature to be checked is whether in buffer memory.
Added the determining step that data is merged the reconstruction state among the 4th embodiment; And when data merge reconstruction; Increase a new newly added signature dictionary, avoid the follow-up signature that writes is write the chaotic of the data that cause in the newly added signature dictionary that merges and need empty or loses.And after data merge the reconstruction completion, new newly added signature dictionary will replace original newly added signature dictionary, become existing newly added signature dictionary.
As shown in Figure 8, Fig. 8 is the schematic block diagram of first embodiment of a kind of signature server of the present invention.In the present embodiment, signature server comprises: receiver module 701, first enquiry module 702, first memory 703, information are returned module 704, index module 705, second enquiry module 706, second memory 707, are merged module 709, update module 710 and the 3rd storer 708.
First memory 703 is used to store newly added signature dictionary and index, and is preferably internal memory.Wherein, can store a plurality of signatures in the newly added signature dictionary.707 of second memories are used to store a plurality of data blocks, and are preferably solid-state memory.Comprise a plurality of signatures in each data block.Signature sequential storage in the second memory 707.Then store the index relative of stored position information of signature and data block of the boundary of data block in the index.
In each data block, signature can carry out compression memory.Concrete compression process can be: to first signature, store first signature itself, to follow-up signature, store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the first signature of follow-up signature.
In actual use, receiver module 701 is used to receive signature to be checked.First enquiry module 702 is used in newly added signature dictionary query signature.When information is returned module 704 and in the newly added signature dictionary, inquired signature to be checked, returning signature and having information.Further, if signature exists, explain that then this web page address or link are existing, can browse.Perhaps,, inquire about this signature of discovery and exist, explain that then webpage repeats, and should carry out removing duplicate webpages if this signature is the signature of up-to-date appearance.
When 705 of index module do not inquire signature to be checked in the newly added signature dictionary, the stored position information of the corresponding data piece of inquiry signature to be checked in index.
Second enquiry module 706 obtains the corresponding data piece according to the stored position information that index module 705 is inquired, and in the corresponding data piece, inquires about signature to be checked.If in the corresponding data piece, inquire signature to be checked, then information is returned module 704 and is returned signature and have information.When in the corresponding data piece, not inquiring signature to be checked, second enquiry module 706 adds signature to be checked in the newly added signature dictionary to, and, return module 704 by information and return signature and do not have information.
Merge module 709 and utilize the newly added signature dictionary to generate the newly added signature file, and merge reconstruction, to generate new backup file with the backup file of data block.
710 of update module are utilized new backup file Update Information piece and index.
The 3rd storer 708 is used to store newly added signature file and backup file.And the 3rd storer 708 is preferably hard disk.
Search except signing or carry out the removing duplicate webpages, also can use this signature server to link the normalization checking.For example, through rule the link appearance being carried out normalization, possibly be invalid link but the link after the conversion is arranged, this moment can through signature server judge link after the conversion whether effective.
As shown in Figure 9, Fig. 9 is the schematic block diagram of second embodiment of a kind of signature server of the present invention.In the present embodiment, signature server comprises: receiver module 801, first enquiry module 802, first memory 803, information are returned module 804, index module 805, second enquiry module 806, second memory 807, are merged module 809, update module 810 and the 3rd storer 808.Above-mentioned module is identical with functional module 801-810 among Fig. 7, repeats no more at this.
The difference of present embodiment and the described embodiment of Fig. 7 is that the signature server of present embodiment further comprises query module 812.If first enquiry module 802 does not inquire signature to be checked in the newly added signature dictionary, then query module 812 is inquired about signature to be checked earlier in the buffer memory of first memory 803.If in buffer memory, inquire signature to be checked, then information is returned module 804 and is returned signature and have information, if in buffer memory, do not inquire signature to be checked, in index, is inquired about the stored position information of the corresponding data piece of signature to be checked again by index module 805.If inquire signature to be checked in the corresponding data piece, then second enquiry module 806 writes signature to be checked in the buffer memory of first memory 803.
Shown in figure 10, Figure 10 is the schematic block diagram of the 3rd embodiment of a kind of signature server of the present invention.In the present embodiment, receiver module 901, first enquiry module 902, first memory 903, information are returned module 904, index module 905, second enquiry module 906, second memory 907, are merged module 909, update module 910 and the 3rd storer 908.Above-mentioned module is identical with functional module 701-710 among Fig. 7, repeats no more at this.
The difference of present embodiment and the described embodiment of Fig. 7 is that the signature server of present embodiment further comprises judge module 912 and the 4th enquiry module 913.
The main effect of judge module 912 is to judge whether signature server is in merging reconstruction state.
When judging that signature server is not in merging reconstruction state, in index, inquire about the stored position information of the corresponding data piece of signature to be checked by index module 905.Second enquiry module 906 obtains the corresponding data piece according to the stored position information that index module 905 is inquired, and in the corresponding data piece, inquires about signature to be checked.If in the corresponding data piece, inquire signature to be checked, then return module 904 and return signature and have information by information.If do not inquire signature to be checked in the corresponding data piece, then second enquiry module 906 adds signature to be checked in the newly added signature dictionary to.
When judging that signature server is in merging reconstruction state, inquiry signature to be checked in 913 new newly added signature dictionaries in first memory 903 of the 4th enquiry module.If in new newly added signature dictionary, inquire signature to be checked, then information is returned module 904 and is returned signature and have information.If in new newly added signature dictionary, do not inquire signature to be checked, then index module 905 is inquired about the stored position information of the corresponding data piece of signature to be checked in index.Second enquiry module 906 obtains the corresponding data piece according to stored position information, and in the corresponding data piece, inquires about signature to be checked.If in the corresponding data piece, inquire signature to be checked, then information is returned module 904 and is returned signature and have information, if in the corresponding data piece, do not inquire signature to be checked, then second enquiry module 906 adds signature to be checked in the new newly added signature dictionary to.
In the present embodiment; Can use for reference the caching mechanism among Fig. 9 equally; Between judge module 912 and index module 905, between the 4th enquiry module 913 and the index module 905, insert the query module that is connected with first memory 903; Be used for buffer memory inquiry signature to be checked, repeat no more at this at first memory 903.
Can find out by above technical scheme; The performance of signature server of the present invention is further promoted, and accelerates the signature inquiry velocity, makes storage space save 50% through compression memory; Effectively utilize storage space, and ensured that through merging re-establishment mechanism service runs without interruption.And this signature server can also be used for removing duplicate webpages or link normalization checking.
The above is merely preferred embodiment of the present invention, and is in order to restriction the present invention, not all within spirit of the present invention and principle, any modification of being made, is equal to replacement, improvement etc., all should be included within the scope that the present invention protects.

Claims (38)

1. the control method of a signature server is characterized in that, said control method comprises:
A. receive signature to be checked;
B. in first memory, inquire about said signature to be checked;
C. if in said first memory, do not inquire said signature to be checked, then obtain the corresponding data piece, and in said corresponding data piece, inquire about said signature to be checked from second memory.
2. the control method of signature server according to claim 1; It is characterized in that; Further comprise among the said step b: the said signature to be checked of inquiry in the newly added signature dictionary of said first memory; If in said newly added signature dictionary, inquire said signature to be checked, then return signature and have information.
3. the control method of signature server according to claim 2 is characterized in that, said step c further comprises: if in said corresponding data piece, inquire said signature to be checked, then return signature and have information; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said newly added signature dictionary.
4. the control method of signature server according to claim 2 is characterized in that, if in said newly added signature dictionary, do not inquire said signature to be checked, then in the buffer memory of said first memory, inquires about said signature to be checked; If in said buffer memory, inquire said signature to be checked, then return signature and have information.
5. the control method of signature server according to claim 4; It is characterized in that; Said step c further comprises: if in said corresponding data piece, inquire said signature to be checked, then said signature to be checked is write said buffer memory, and return to sign and have information; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said newly added signature dictionary.
6. the control method of signature server according to claim 2 is characterized in that, if in said newly added signature dictionary, do not inquire said signature to be checked, judges then whether said signature server is in merging reconstruction state; If judging to be in merges the reconstruction state, then the said signature to be checked of inquiry in the new newly added signature dictionary of said first memory if in said new newly added signature dictionary, inquire said signature to be checked, then returns signature and has information.
7. the control method of signature server according to claim 6; It is characterized in that; Said step c further comprises: if in said corresponding data piece, inquire said signature to be checked; Then return signature and have information,, then said signature to be checked is added in the said new newly added signature dictionary if in said corresponding data piece, do not inquire said signature to be checked.
8. the control method of signature server according to claim 6; It is characterized in that; If in said new newly added signature dictionary, do not inquire said signature to be checked; Then the said signature to be checked of inquiry in the buffer memory of said first memory if in said buffer memory, inquire said signature to be checked, then returns signature and has information.
9. the control method of signature server according to claim 8 is characterized in that, said step c comprises:
If in said corresponding data piece, inquire said signature to be checked; Then return signature and have information; And said signature to be checked write said buffer memory; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said new newly added signature dictionary.
10. the control method of signature server according to claim 1; It is characterized in that; Said step c is included in the stored position information of the said corresponding data piece of inquiry in the index in the said first memory, obtains said corresponding data piece according to said stored position information.
11. the control method of signature server according to claim 10; It is characterized in that; The a plurality of data blocks of storage in the said second memory; Comprise a plurality of signatures in each said data block, the signature in said a plurality of data blocks is pressed the signature contents sequential storage, stores the index relative of stored position information of signature and said data block of the boundary of said data block in the said index.
12. the control method of signature server according to claim 11 is characterized in that, in each said data block, said signature carries out compression memory.
13. the control method of signature server according to claim 12; It is characterized in that; In each said data block,, store first signature itself to first signature; To follow-up signature, store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the said first signature of follow-up signature.
14. the control method of signature server according to claim 11 is characterized in that, said control method further comprises:
D. utilize said newly added signature dictionary to generate the newly added signature file, and merge reconstruction, to generate new backup file with the backup file of said data block.
15. the control method of signature server according to claim 14 is characterized in that, said steps d further comprises: before utilizing said newly added signature dictionary generation newly added signature file, generate new newly added signature dictionary.
16. the control method of signature server according to claim 15 is characterized in that, said control method further comprises:
Utilize said new backup file to upgrade said data block and said index, and utilize said new newly added signature dictionary to replace said newly added signature dictionary.
17. the control method of signature server according to claim 14 is characterized in that, said newly added signature file and said backup file are stored in the 3rd storer.
18. the control method of signature server according to claim 17 is characterized in that, said the 3rd storer is a hard disk.
19. the control method of signature server according to claim 1 is characterized in that, said first memory is an internal memory, and said second memory is a solid-state memory.
20. a signature server is characterized in that comprising:
First memory and second memory;
Receiver module receives signature to be checked;
First enquiry module, the said signature to be checked of inquiry in said first memory;
Second enquiry module if in said first memory, do not inquire said signature to be checked, then obtains the corresponding data piece from said second memory, and in said corresponding data piece, inquires about said signature to be checked.
21. signature server according to claim 20; It is characterized in that; Said signature server comprises that further information returns module; Said first enquiry module is inquired about said signature to be checked in the newly added signature dictionary of said first memory, if in said newly added signature dictionary, inquire said signature to be checked, then said information is returned module and returned signature and have information.
22. signature server according to claim 21 is characterized in that, in said second enquiry module, if in said corresponding data piece, inquire said signature to be checked, then said information is returned module and is returned signature and have information; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said newly added signature dictionary.
23. signature server according to claim 21; It is characterized in that; Said signature server further comprises the query module; In said first enquiry module, if in said newly added signature dictionary, do not inquire said signature to be checked, then said query module is inquired about said signature to be checked in the buffer memory of said first memory; If in said buffer memory, inquire said signature to be checked, then said information is returned module and is returned signature and have information.
24. signature server according to claim 23 is characterized in that, in said second enquiry module, if in said corresponding data piece, inquire said signature to be checked, then said signature to be checked is write said buffer memory, and returns to sign and have information; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said newly added signature dictionary.
25. signature server according to claim 21; It is characterized in that; Said signature server further comprises the 4th enquiry module; In said first enquiry module,, judge then whether said signature server is in merging reconstruction state if in said newly added signature dictionary, do not inquire said signature to be checked; If judge to be in and merge the reconstruction state; Then said the 4th enquiry module is inquired about said signature to be checked in the new newly added signature dictionary of said first memory; If in said new newly added signature dictionary, inquire said signature to be checked, then said information is returned module and is returned signature and have information.
26. signature server according to claim 25; It is characterized in that; In said second enquiry module, if in said corresponding data piece, inquire said signature to be checked, then said information is returned module and is returned signature and have information; If in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said new newly added signature dictionary.
27. signature server according to claim 25; It is characterized in that said signature server further comprises the query module, in said the 4th enquiry module; If in said new newly added signature dictionary, do not inquire said signature to be checked; Then said query module is inquired about said signature to be checked in the buffer memory of said first memory, if in said buffer memory, inquire said signature to be checked, then said information is returned module and returned signature and have information.
28. signature server according to claim 27; It is characterized in that, in said second enquiry module, if in said corresponding data piece, inquire said signature to be checked; Then said information is returned module and is returned signature and have information; And said signature to be checked write said buffer memory, if in said corresponding data piece, do not inquire said signature to be checked, then said signature to be checked is added in the said new newly added signature dictionary.
29. signature server according to claim 20; It is characterized in that; Said second enquiry module is included in the stored position information of the said corresponding data piece of inquiry in the index in the said first memory, obtains said corresponding data piece according to said stored position information.
30. signature server according to claim 29; It is characterized in that; The a plurality of data blocks of storage in the said second memory; Comprise a plurality of signatures in each said data block, the signature in said a plurality of data blocks is pressed the signature contents sequential storage, stores the index relative of stored position information of signature and said data block of the boundary of said data block in the said index.
31. signature server according to claim 30 is characterized in that, in each said data block, said signature carries out compression memory.
32. signature server according to claim 31; It is characterized in that; In each said data block,, store first signature itself to first signature; To follow-up signature, store difference and the back pre-determined bit of follow-up signature of preceding pre-determined bit of preceding pre-determined bit and the said first signature of follow-up signature.
33. signature server according to claim 30 is characterized in that, said signature server further comprises:
Merge module, said merging module utilizes said newly added signature dictionary to generate the newly added signature file, and merges reconstruction with the backup file of said data block, to generate new backup file.
34. signature server according to claim 33 is characterized in that, before utilizing said newly added signature dictionary generation newly added signature file, generates new newly added signature dictionary.
35. signature server according to claim 34 is characterized in that, said signature server further comprises:
Update module, said update module utilize said new backup file to upgrade said data block and said index, and utilize said new newly added signature dictionary to replace said newly added signature dictionary.
36. signature server according to claim 33 is characterized in that, said newly added signature file and said backup file are stored in the 3rd storer.
37. signature server according to claim 36 is characterized in that, said the 3rd storer is a hard disk.
38. signature server according to claim 20 is characterized in that, said first memory is an internal memory, and said second memory is a solid-state memory.
CN2010106159263A 2010-12-30 2010-12-30 Signature server and control method thereof Pending CN102567395A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010106159263A CN102567395A (en) 2010-12-30 2010-12-30 Signature server and control method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010106159263A CN102567395A (en) 2010-12-30 2010-12-30 Signature server and control method thereof

Publications (1)

Publication Number Publication Date
CN102567395A true CN102567395A (en) 2012-07-11

Family

ID=46412830

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010106159263A Pending CN102567395A (en) 2010-12-30 2010-12-30 Signature server and control method thereof

Country Status (1)

Country Link
CN (1) CN102567395A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823873A (en) * 2014-02-27 2014-05-28 北京奇虎科技有限公司 Reading/writing method, device and system of browser setting item
CN110856015A (en) * 2014-04-07 2020-02-28 尼尔森(美国)有限公司 Method and apparatus for rating media
CN113158002A (en) * 2021-04-28 2021-07-23 北京达佳互联信息技术有限公司 Searching method, searching device, electronic equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1409836A (en) * 1999-12-17 2003-04-09 阿克蒂夫卡德公司 Computer system for application by accreditation access
CN1859293A (en) * 2005-12-30 2006-11-08 华为技术有限公司 Device and method for realizing message repeating

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1409836A (en) * 1999-12-17 2003-04-09 阿克蒂夫卡德公司 Computer system for application by accreditation access
CN1859293A (en) * 2005-12-30 2006-11-08 华为技术有限公司 Device and method for realizing message repeating

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823873A (en) * 2014-02-27 2014-05-28 北京奇虎科技有限公司 Reading/writing method, device and system of browser setting item
CN103823873B (en) * 2014-02-27 2017-05-24 北京奇虎科技有限公司 Reading/writing method, device and system of browser setting item
CN110856015A (en) * 2014-04-07 2020-02-28 尼尔森(美国)有限公司 Method and apparatus for rating media
CN110856015B (en) * 2014-04-07 2022-01-28 尼尔森(美国)有限公司 Method and apparatus for rating media
US11533535B2 (en) 2014-04-07 2022-12-20 The Nielsen Company (Us), Llc Signature retrieval and matching for media monitoring
CN113158002A (en) * 2021-04-28 2021-07-23 北京达佳互联信息技术有限公司 Searching method, searching device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
KR102007070B1 (en) Reference block aggregating into a reference set for deduplication in memory management
Lu et al. Wisckey: Separating keys from values in ssd-conscious storage
US11010300B2 (en) Optimized record lookups
CN103098035B (en) Storage system
US9292584B1 (en) Efficient data communication based on lossless reduction of data by deriving data from prime data elements resident in a content-associative sieve
CN102591947B (en) For the index that the quick of data deduplication and low RAM take
US10248556B2 (en) Forward-only paged data storage management where virtual cursor moves in only one direction from header of a session to data field of the session
US8738857B1 (en) System and method for improving cache performance
CN107947799B (en) Data compression method and device
CN105224237A (en) A kind of date storage method and device
CN110741637B (en) Method for simplifying video data, computer readable storage medium and electronic device
KR20200122994A (en) Key Value Append
CN108121813B (en) Data management method, device, system, storage medium and electronic equipment
CN102810089A (en) Short link system based on content and implementation method thereof
CN104731886A (en) Processing method and system for mass small files
US20170123689A1 (en) Pipelined Reference Set Construction and Use in Memory Management
US20170123677A1 (en) Integration of Reference Sets with Segment Flash Management
CN108475508B (en) Simplification of audio data and data stored in block processing storage system
US20200319797A1 (en) System and method for file processing from a block device
Changtong An improved HDFS for small file
CN109992469B (en) Method and device for merging logs
CN111831691B (en) Data reading and writing method and device, electronic equipment and storage medium
Tulkinbekov et al. CaseDB: Lightweight key-value store for edge computing environment
CN102567395A (en) Signature server and control method thereof
CN110633261A (en) Picture storage method, picture query method and device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C12 Rejection of a patent application after its publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20120711