CN103491124A - Method for processing multimedia message data and distributed cache system - Google Patents
Method for processing multimedia message data and distributed cache system Download PDFInfo
- Publication number
- CN103491124A CN103491124A CN201210195876.7A CN201210195876A CN103491124A CN 103491124 A CN103491124 A CN 103491124A CN 201210195876 A CN201210195876 A CN 201210195876A CN 103491124 A CN103491124 A CN 103491124A
- Authority
- CN
- China
- Prior art keywords
- multimedia message
- message data
- keynode
- data
- key value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for processing multimedia message data and a distributed cache system. The method is applied to a network structure which includes a multimedia message system and the distributed cache system. The method includes the step that the multimedia message data sent from the multimedia message system are stored by the distributed cache system through an IFS storage mode. The distributed cache system comprises a receiving module, a processing module and a storage module; the receiving module is used for receiving the multimedia message data sent from the multimedia message system; the processing module is used for writing in the multimedia message data received by the receiving module to the storage module through the IFS storage mode; the storage module is used for storing the multimedia message data written in to the storage module by the processing module. According to the method for processing the multimedia message data and the distributed cache system, the IFS storage mode, with high writing and reading efficiency, of the distributed cache system is used, the storage efficiency of the distributed cache system is equivalent to that of NAS equipment, but the cost of the distributed cache system is lower than that of the NAS equipment.
Description
Technical field
The present invention relates to the Distributed Storage technical field, relate in particular to a kind of method that multimedia message data is processed and distributed cache system.
Background technology
Cloud computing (Cloud Computing) is that the product that the traditional calculations machine technology such as (Utility Computing) network storage (Network Storage Technologies), virtual (Virtualization), load balancing (Load Balance) and network technical development merge is calculated in grid computing (Grid Computing), Distributed Calculation (Distributed Computing), parallel computation (Parallel Computing), effectiveness.It is intended to, and by network, a plurality of costs, relatively low computational entity is integrated into a system with powerful calculating ability.Distributed caching is a field in the cloud computing category, its effect is to provide the ability of distributed storage service and the high-speed read-write access of mass data, there is very strong extensibility, and very low for the server hardware requirement, can greatly reduce hardware cost.
In existing MMS system, what the storage of bottom was used is traditional NAS (Network-Attached Storage, network attached storage) equipment, but traditional NAS equipment is faced with the problem of hardware cost and licence costliness.
Summary of the invention
Technical problem to be solved by this invention is to provide a kind of method that multimedia message data is processed and distributed cache system, to overcome the too high defect of hardware use cost in existing MMS system.
For addressing the above problem, the invention provides a kind of method that multimedia message data is processed, in the network configuration that is applied to comprise MMS system and distributed cache system, described method comprises:
The multimedia message data that described distributed cache system adopts full index scanning (IFS) storage mode to send described MMS system is stored.
Further, described method specifically comprises:
Receive that described MMS system sends carry multimedia message data to be written and keyword (key) value corresponding to described multimedia message data write data command the time, the storage address information that described distributed cache system is first searched corresponding Keynode according to the cryptographic Hash of described key value in Hash table; Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in described distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file;
If do not find, described multimedia message data to be written is stored in the data file of current active, and correspondingly increase a Keynode in internal memory, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table.
Further, described method also comprises:
If described distributed cache system finds the storage address information of corresponding Keynode in Hash table according to the cryptographic Hash of described key value, according to the storage address information of the Keynode searched, find corresponding Keynode, when the key value comprised in judging this Keynode key value corresponding from described multimedia message data to be written is different, stored position information according to the multimedia message data comprised in this Keynode in data file, use described multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly the information in this Keynode.
Further,
When the key value in the Keynode until newly-increased surpasses pre-configured key value maximum number of byte, described by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table, specifically comprises:
First to this key value after md5 processes, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with the described multimedia message data through the md5 processing writes in described Hash table.
Further,
When the key value in the Keynode until newly-increased surpasses pre-configured key value maximum number of byte byte, described to judge the key value that comprises in this Keynode key value corresponding from described multimedia message data to be written different, specifically comprise:
Judge the key value corresponding through the multimedia message data described to be written of md5 processing different from the key value comprised in the Keynode found.
Further, described method also comprises:
When receiving the deletion data command, key value inquiry packet in internal memory that described distributed cache system is corresponding according to the data to be deleted of carrying in described deletion data command contains the Keynode of this key value; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode is labeled as deletion by this multimedia message data, and this Keynode is put in the chained list of idle Keynode in this corresponding data file.
Further,
Also carry information effective time of multimedia message data to be written in the write data order;
Described distributed cache system is in the data file that described multimedia message data to be written is stored into to current active the time, also information effective time of described multimedia message data to be written together stored;
The prewired clearance time information that starts that is equipped with in described distributed cache system;
Described method also comprises: when described beginning clearance time arrives, described distributed cache system starts to carry out data preparation, and each data file is carried out to poll;
For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion;
For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
Correspondingly, the present invention also provides a kind of distributed cache system, comprising:
Receiver module, the multimedia message data of sending for receiving MMS system;
Processing module, be written to memory module for the multimedia message data that adopts full index scanning (IFS) storage mode that described receiver module is received;
Described memory module, be written to the multimedia message data of this module for preserving described processing module.
Further,
Described processing module for receive at described receiver module that described MMS system sends carry multimedia message data to be written and keyword (key) value corresponding to described multimedia message data write data command the time, the storage address information of searching corresponding Keynode according to the cryptographic Hash of described key value in Hash table; Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in described distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file;
Described processing module is not also for when finding the storage address information of corresponding Keynode, described multimedia message data to be written is stored in the data file of the current active in described memory module, and correspondingly increase a Keynode in internal memory, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table.
Further,
Described processing module also for the cryptographic Hash according to described key value when Hash table finds the storage address information of corresponding Keynode, find corresponding Keynode according to the storage address information of the Keynode searched, when the key value comprised in judging this Keynode key value corresponding from described multimedia message data to be written is different, stored position information according to the multimedia message data comprised in this Keynode in data file, use described multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly the information in this Keynode.
Further,
Described processing module is when judging key value in newly-increased Keynode and surpass pre-configured key value maximum number of byte, described by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table, specifically comprises:
Described processing module for first to this key value after md5 processes, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with the described multimedia message data of passing through the md5 processing writes in described Hash table.
Further,
Described processing module is also for judging when the key of newly-increased Keynode value surpasses pre-configured key value maximum number of byte, described to judge the key value that comprises in this Keynode key value corresponding from described multimedia message data to be written different, specifically comprise:
Described processing module is different for the key value of judging key value corresponding to multimedia message data described to be written processed through md5 and comprising from the Keynode found.
Further,
Described receiver module is also for receiving the deletion data command;
Described processing module is also for when described receiver module is received described deletion data command, and according to the data to be deleted of carrying in described deletion data command, corresponding key value inquiry packet in internal memory contains the Keynode of this key value; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode is labeled as deletion by this multimedia message data, and this Keynode is put in the chained list of idle Keynode in this corresponding data file.
Further,
Also carry information effective time of multimedia message data to be written in the write data order; The prewired clearance time information that starts that is equipped with also in described memory module;
Described processing module is also for when storing described multimedia message data to be written the data file of current active into, and information effective time of described multimedia message data to be written is together stored; Described processing module also, for when described beginning clearance time arrives, starts to carry out data preparation, and each data file is carried out to poll; For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion; For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
The present invention has used the IFS memory module that the distributed cache system read-write efficiency is higher, and storage efficiency is suitable with NAS equipment, but cost is cheaper.
The accompanying drawing explanation
Fig. 1 carries out the network element structure chart of the system of multimedia message data access based on distributed cache system in the embodiment of the present invention;
Fig. 2 is the structural representation of distributed cache system in the embodiment of the present invention.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, hereinafter in connection with accompanying drawing, embodiments of the invention are elaborated.It should be noted that, in the situation that do not conflict, the embodiment in the application and the feature in embodiment be combination in any mutually.
Go up because distributed caching can be deployed in cheap PC Server (server), and coordinating some characteristics (as data arrange, be about to stale data and regularly cleared up) of self, can well support the application of MMS system.
In view of existing MMS system is used high hardware and the maintenance cost in NAS device side Lingao, the present embodiment has proposed a kind of distributed caching that uses and has replaced the method for traditional NAS equipment as the data storage server of MMS system, the related data of MMS system is stored in the distributed cache system be deployed on cheap PC Server, characteristic in conjunction with distributed caching self, as: IFS (Index Full Scan, full index scanning) persistent storage pattern, regularly arrange the function of stale data, the use cost of MMS system is reduced greatly, and memory property is suitable with NAS equipment before.
Traditional its index of persistent storage scheme all is based upon in disk file, search index, renewal, reading out data all can produce the random access disk, response once application reads or writes request, random IO (the Input/Output of several times disk will occur in the persistent storage system, I/O) access, therefore be easy to produce the IO bottleneck.The technical characteristics of IFS storage scheme is to make the random IO access of disk become order IO access, and reduces as far as possible disk access, thereby can greatly improve the efficiency of persistent storage.
In the present embodiment, carry out the system configuration of multimedia message data access based on distributed cache system, as shown in Figure 1, comprising: MMS system, client and distributed cache system.
Wherein, distributed cache system is mounted and is arranged on a plurality of data storage servers, make it to become a cluster, externally can regard as is a memory device, be equivalent to a plurality of data storage servers and jointly formed distributed cache system, data are safer, and system memory size also facilitates dynamic expansion; By client-side interface is provided in native system, the related data of MMS system can be deposited in distributed cache system.
In said system, the method for multimedia message data being processed by distributed cache system comprises:
One, write request, update request and the removal request received is converted to the sequential write operation, improved the efficiency of writing data, access efficiency and NAS equipment are substantially suitable.Specifically comprise:
A, write data;
In distributed cache system, adopt the IFS storage mode to store multimedia message data; When writing multimedia message data, directly this multimedia message data is write to the data file of current active.Wherein, at any time, movable data file only has one.
When receiving key (keyword) value corresponding to multimedia message data to be written and this multimedia message data, the storage address information that distributed cache system first can be searched corresponding Keynode according to hash (Hash) value of this key value in HASH TABLE (Hash table); Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file.
If do not find, above-mentioned multimedia message data to be written is stored in the data file of current active, and correspondingly increase a Keynode in internal memory, by the storage address information of this Keynode, the corresponding relation of the key value corresponding with this multimedia message data writes in HASH TABLE;
If find, according to the storage address information of the Keynode that searches, find corresponding Keynode, whether the key value relatively comprised in this Keynode key value corresponding with multimedia message data to be written be identical, if identical, do not upgrade operation; If different, the stored position information in data file according to the multimedia message data comprised in this Keynode, use above-mentioned multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly information in this Keynode (any one or combination in any as in the length of the length information of key value, key, this multimedia message data, need be determined on a case-by-case basis).
During due to employing IFS memory module, the length of the key value of storing in General Requirements Keynode can not surpass pre-configured key value maximum number of byte (have now and usually be configured to 70 bytes).Therefore, when the key value in the Keynode until newly-increased surpasses 70 bytes, can first to this key value, after processing, be preserved again md5.Correspondingly, when the key value comprised in corresponding key value and the Keynode found in the multimedia message data by be written compares, the length of the key value that multimedia message data as to be written as this is corresponding has surpassed 70 bytes, after first this key value being carried out to the md5 processing, then compares.
B, deletion data;
When receiving the deletion data command, key value inquiry packet in internal memory that distributed cache system is corresponding according to the data to be deleted of carrying in this deletion data command contains the Keynode of this key value; As do not find, return to deletion and unsuccessfully point out; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode, this multimedia message data is labeled as to deletion in this corresponding data file, and this Keynode is put into to freelisthead (chained list of idle Keynode, reuse Keynode for recovery).Check in memory cache simultaneously whether this multimedia message data is arranged, if any deleting in the lump.
Two, the regular arrangement of stale data, regularly batch Delete Expired data, reclaim the disk space that invalid data takies.Specifically comprise:
Steps A: the user can open the function that the stale data of distributed cache system is regularly cleared up by client, and the time period of setting cleaning, this time period comprises: start clearance time and finish clearance time, that is to say, cleaning operation just carries out within this time period, exceed this time period, do not carry out any cleaning operation;
Step B: due to when in distributed cache system, writing multimedia message data, can be by the effective time of this multimedia message data of data writing file together.Therefore, when starting the clearance time arrival, distributed cache system starts to carry out data preparation, and each data file is carried out to poll;
For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion;
For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
From above-mentioned explanation, can find out, said method becomes the random IO access of disk into order IO access, and reduce as far as possible disk access, use the index technology of the storage based on internal memory simultaneously, thereby can greatly improve the efficiency of persistent storage, under equal number of disks, access efficiency and NAS equipment are substantially suitable.According to the effective time of every data self, every day, the multimedia message data by expired was regularly cleared up.The invention solves MMS system and use the expensive problem of traditional NAS memory device.After using distributed caching as memory device, hardware can be become by expensive NAS PC Server cheaply.And distributed caching IFS stores data and the characteristic of regularly clearing up stale data, also meet very much the demand of MMS system.
Correspondingly, in this example, a kind of distributed cache system as shown in Figure 2, comprising:
Receiver module, the multimedia message data of sending for receiving MMS system;
Processing module, be written to memory module for the multimedia message data that adopts full index scanning (IFS) storage mode that described receiver module is received;
Described memory module, be written to the multimedia message data of this module for preserving described processing module.
Preferably,
Described processing module for receive at described receiver module that described MMS system sends carry multimedia message data to be written and keyword (key) value corresponding to described multimedia message data write data command the time, the storage address information of searching corresponding Keynode according to the cryptographic Hash of described key value in Hash table; Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in described distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file;
Described processing module is not also for when finding the storage address information of corresponding Keynode, described multimedia message data to be written is stored in the data file of the current active in described memory module, and correspondingly increase a Keynode in internal memory, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table.
Preferably,
Described processing module also for the cryptographic Hash according to described key value when Hash table finds the storage address information of corresponding Keynode, find corresponding Keynode according to the storage address information of the Keynode searched, when the key value comprised in judging this Keynode key value corresponding from described multimedia message data to be written is different, stored position information according to the multimedia message data comprised in this Keynode in data file, use described multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly the information in this Keynode.
Preferably,
Described processing module is when judging key value in newly-increased Keynode and surpass pre-configured key value maximum number of byte, described by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table, specifically comprises:
Described processing module for first to this key value after md5 processes, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with the described multimedia message data of passing through the md5 processing writes in described Hash table.
Preferably,
Described processing module is also for judging when the key of newly-increased Keynode value surpasses pre-configured key value maximum number of byte, described to judge the key value that comprises in this Keynode key value corresponding from described multimedia message data to be written different, specifically comprise:
Described processing module is different for the key value of judging key value corresponding to multimedia message data described to be written processed through md5 and comprising from the Keynode found.
Preferably,
Described receiver module is also for receiving the deletion data command;
Described processing module is also for when described receiver module is received described deletion data command, and according to the data to be deleted of carrying in described deletion data command, corresponding key value inquiry packet in internal memory contains the Keynode of this key value; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode is labeled as deletion by this multimedia message data, and this Keynode is put in the chained list of idle Keynode in this corresponding data file.
Preferably,
Also carry information effective time of multimedia message data to be written in the write data order; The prewired clearance time information that starts that is equipped with also in described memory module;
Described processing module is also for when storing described multimedia message data to be written the data file of current active into, and information effective time of described multimedia message data to be written is together stored; Described processing module also, for when described beginning clearance time arrives, starts to carry out data preparation, and each data file is carried out to poll; For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion; For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
One of ordinary skill in the art will appreciate that all or part of step in said method can come the instruction related hardware to complete by program, described program can be stored in computer-readable recording medium, as read-only memory, disk or CD etc.Alternatively, all or part of step of above-described embodiment also can realize with one or more integrated circuits.Correspondingly, each the module/unit in above-described embodiment can adopt the form of hardware to realize, also can adopt the form of software function module to realize.The present invention is not restricted to the combination of the hardware and software of any particular form.
The foregoing is only the preferred embodiments of the present invention, be not intended to limit protection scope of the present invention.According to summary of the invention of the present invention; also other various embodiments can be arranged; in the situation that do not deviate from spirit of the present invention and essence thereof; those of ordinary skill in the art are when making according to the present invention various corresponding changes and distortion; within the spirit and principles in the present invention all; any modification of doing, be equal to replacement, improvement etc., within protection scope of the present invention all should be included in.
Claims (14)
1. the method that multimedia message data is processed, in the network configuration that is applied to comprise MMS system and distributed cache system, described method comprises:
The multimedia message data that described distributed cache system adopts full index scanning (IFS) storage mode to send described MMS system is stored.
2. the method for claim 1, is characterized in that, specifically comprises:
Receive that described MMS system sends carry multimedia message data to be written and keyword (key) value corresponding to described multimedia message data write data command the time, the storage address information that described distributed cache system is first searched corresponding Keynode according to the cryptographic Hash of described key value in Hash table; Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in described distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file;
If do not find, described multimedia message data to be written is stored in the data file of current active, and correspondingly increase a Keynode in internal memory, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table.
3. method as claimed in claim 2, is characterized in that, also comprises:
If described distributed cache system finds the storage address information of corresponding Keynode in Hash table according to the cryptographic Hash of described key value, according to the storage address information of the Keynode searched, find corresponding Keynode, when the key value comprised in judging this Keynode key value corresponding from described multimedia message data to be written is different, stored position information according to the multimedia message data comprised in this Keynode in data file, use described multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly the information in this Keynode.
4. method as claimed in claim 2 is characterized in that:
When the key value in the Keynode until newly-increased surpasses pre-configured key value maximum number of byte, described by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table, specifically comprises:
First to this key value after md5 processes, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with the described multimedia message data through the md5 processing writes in described Hash table.
5. method as claimed in claim 3 is characterized in that:
When the key value in the Keynode until newly-increased surpasses pre-configured key value maximum number of byte byte, described to judge the key value that comprises in this Keynode key value corresponding from described multimedia message data to be written different, specifically comprise:
Judge the key value corresponding through the multimedia message data described to be written of md5 processing different from the key value comprised in the Keynode found.
6. method as claimed in claim 2, is characterized in that, also comprises:
When receiving the deletion data command, key value inquiry packet in internal memory that described distributed cache system is corresponding according to the data to be deleted of carrying in described deletion data command contains the Keynode of this key value; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode is labeled as deletion by this multimedia message data, and this Keynode is put in the chained list of idle Keynode in this corresponding data file.
7. method as described as claim 2 or 6 is characterized in that:
Also carry information effective time of multimedia message data to be written in the write data order;
Described distributed cache system is in the data file that described multimedia message data to be written is stored into to current active the time, also information effective time of described multimedia message data to be written together stored;
The prewired clearance time information that starts that is equipped with in described distributed cache system;
Described method also comprises: when described beginning clearance time arrives, described distributed cache system starts to carry out data preparation, and each data file is carried out to poll;
For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion;
For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
8. a distributed cache system comprises:
Receiver module, the multimedia message data of sending for receiving MMS system;
Processing module, be written to memory module for the multimedia message data that adopts full index scanning (IFS) storage mode that described receiver module is received;
Described memory module, be written to the multimedia message data of this module for preserving described processing module.
9. distributed cache system as claimed in claim 8 is characterized in that:
Described processing module for receive at described receiver module that described MMS system sends carry multimedia message data to be written and keyword (key) value corresponding to described multimedia message data write data command the time, the storage address information of searching corresponding Keynode according to the cryptographic Hash of described key value in Hash table; Wherein, each Keynode is the key value that comprises a multimedia message data of preserving in described distributed cache system, the length information of key, the length of this multimedia message data and the record of the stored position information of this multimedia message data in data file;
Described processing module is not also for when finding the storage address information of corresponding Keynode, described multimedia message data to be written is stored in the data file of the current active in described memory module, and correspondingly increase a Keynode in internal memory, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table.
10. distributed cache system as claimed in claim 9 is characterized in that:
Described processing module also for the cryptographic Hash according to described key value when Hash table finds the storage address information of corresponding Keynode, find corresponding Keynode according to the storage address information of the Keynode searched, when the key value comprised in judging this Keynode key value corresponding from described multimedia message data to be written is different, stored position information according to the multimedia message data comprised in this Keynode in data file, use described multimedia message data to be written to be upgraded the data of this memory location storage, and upgrade accordingly the information in this Keynode.
11. distributed cache system as claimed in claim 9 is characterized in that:
Described processing module is when judging key value in newly-increased Keynode and surpass pre-configured key value maximum number of byte, described by the storage address information of described Keynode, the corresponding relation of the key value corresponding with described multimedia message data writes in described Hash table, specifically comprises:
Described processing module for first to this key value after md5 processes, by the storage address information of described Keynode, the corresponding relation of the key value corresponding with the described multimedia message data of passing through the md5 processing writes in described Hash table.
12. distributed cache system as claimed in claim 10 is characterized in that:
Described processing module is also for judging when the key of newly-increased Keynode value surpasses pre-configured key value maximum number of byte, described to judge the key value that comprises in this Keynode key value corresponding from described multimedia message data to be written different, specifically comprise:
Described processing module is different for the key value of judging key value corresponding to multimedia message data described to be written processed through md5 and comprising from the Keynode found.
13. distributed cache system as claimed in claim 9 is characterized in that:
Described receiver module is also for receiving the deletion data command;
Described processing module is also for when described receiver module is received described deletion data command, and according to the data to be deleted of carrying in described deletion data command, corresponding key value inquiry packet in internal memory contains the Keynode of this key value; If find, the stored position information in data file according to the multimedia message data comprised in this Keynode is labeled as deletion by this multimedia message data, and this Keynode is put in the chained list of idle Keynode in this corresponding data file.
14. distributed cache system as described as claim 9 or 13 is characterized in that:
Also carry information effective time of multimedia message data to be written in the write data order; The prewired clearance time information that starts that is equipped with also in described memory module;
Described processing module is also for when storing described multimedia message data to be written the data file of current active into, and information effective time of described multimedia message data to be written is together stored; Described processing module also, for when described beginning clearance time arrives, starts to carry out data preparation, and each data file is carried out to poll; For each multimedia message data of preserving in each data file, when the effective time of judging this multimedia message data is out of date and be not marked as deletion, this multimedia message data is labeled as to deletion; For each data file, after all multimedia message data in judging whole data file all have been marked as deletion, whole data file is deleted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210195876.7A CN103491124B (en) | 2012-06-14 | 2012-06-14 | A kind of method that multimedia message data is handled and distributed cache system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210195876.7A CN103491124B (en) | 2012-06-14 | 2012-06-14 | A kind of method that multimedia message data is handled and distributed cache system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103491124A true CN103491124A (en) | 2014-01-01 |
| CN103491124B CN103491124B (en) | 2018-08-14 |
Family
ID=49831081
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210195876.7A Active CN103491124B (en) | 2012-06-14 | 2012-06-14 | A kind of method that multimedia message data is handled and distributed cache system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103491124B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104113830A (en) * | 2014-07-29 | 2014-10-22 | 广东能龙教育股份有限公司 | Large-scale short message resend-preventing method based on message digest 5 algorithm (MD5) and distributed cache |
| CN105653198A (en) * | 2014-11-13 | 2016-06-08 | 杭州迪普科技有限公司 | Data processing method and device |
| WO2018145227A1 (en) * | 2017-02-13 | 2018-08-16 | Pivotal Software, Inc. | Managing data records in object-based cloud storage systems |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101163265A (en) * | 2007-11-20 | 2008-04-16 | 中兴通讯股份有限公司 | Distributed database based on multimedia message log inquiring method and system |
| US20100268902A1 (en) * | 2009-04-21 | 2010-10-21 | Google Inc. | Asynchronous distributed object uploading for replicated content addressable storage clusters |
| CN101997911A (en) * | 2010-10-21 | 2011-03-30 | 中兴通讯股份有限公司 | Data migration method and system |
| CN102317938A (en) * | 2008-12-22 | 2012-01-11 | 谷歌公司 | Asynchronous distributed de-duplication for replicated content addressable storage clusters |
| CN102387169A (en) * | 2010-08-26 | 2012-03-21 | 阿里巴巴集团控股有限公司 | Delete method, system and delete server for distributed cache objects |
-
2012
- 2012-06-14 CN CN201210195876.7A patent/CN103491124B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101163265A (en) * | 2007-11-20 | 2008-04-16 | 中兴通讯股份有限公司 | Distributed database based on multimedia message log inquiring method and system |
| CN102317938A (en) * | 2008-12-22 | 2012-01-11 | 谷歌公司 | Asynchronous distributed de-duplication for replicated content addressable storage clusters |
| US20100268902A1 (en) * | 2009-04-21 | 2010-10-21 | Google Inc. | Asynchronous distributed object uploading for replicated content addressable storage clusters |
| CN102387169A (en) * | 2010-08-26 | 2012-03-21 | 阿里巴巴集团控股有限公司 | Delete method, system and delete server for distributed cache objects |
| CN101997911A (en) * | 2010-10-21 | 2011-03-30 | 中兴通讯股份有限公司 | Data migration method and system |
Non-Patent Citations (1)
| Title |
|---|
| 董惟: "基于分布式结构的数据业务管理系统", 《中国优秀硕士学位论文全文数据库（信息科技辑）》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104113830A (en) * | 2014-07-29 | 2014-10-22 | 广东能龙教育股份有限公司 | Large-scale short message resend-preventing method based on message digest 5 algorithm (MD5) and distributed cache |
| CN105653198A (en) * | 2014-11-13 | 2016-06-08 | 杭州迪普科技有限公司 | Data processing method and device |
| WO2018145227A1 (en) * | 2017-02-13 | 2018-08-16 | Pivotal Software, Inc. | Managing data records in object-based cloud storage systems |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103491124B (en) | 2018-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8799601B1 (en) | Techniques for managing deduplication based on recently written extents | |
| CN102694828B (en) | A kind of method of distributed cache system data access and device | |
| CN107179878B (en) | Data storage method and device based on application optimization | |
| US10585611B2 (en) | Inline deduplication | |
| CN105549905A (en) | Method for multiple virtual machines to access distributed object storage system | |
| CN105183839A (en) | Hadoop-based storage optimizing method for small file hierachical indexing | |
| CN102467572B (en) | Data block inquiring method for supporting data de-duplication program | |
| CN105094695B (en) | A kind of storage method and system | |
| CN105027069A (en) | Deduplication of volume regions | |
| CN102662992A (en) | Method and device for storing and accessing massive small files | |
| CN103501319A (en) | Low-delay distributed storage system for small files | |
| CN103034684A (en) | Optimizing method for storing virtual machine mirror images based on CAS (content addressable storage) | |
| CN103246616A (en) | Global shared cache replacement method for realizing long-short cycle access frequency | |
| US20130297969A1 (en) | File management method and apparatus for hybrid storage system | |
| CN110908589B (en) | Data file processing method, device, system and storage medium | |
| WO2013107029A1 (en) | Data processing method, device and system based on block storage | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| CN113626431A (en) | LSM tree-based key value separation storage method and system for delaying garbage recovery | |
| WO2016041127A1 (en) | Data duplication method and storage array | |
| CN109522283A (en) | A kind of data de-duplication method and system | |
| CN108089825B (en) | Storage system based on distributed cluster | |
| CN105159845A (en) | Memory reading method | |
| CN103970875A (en) | Parallel repeated data deleting method | |
| CN104184812A (en) | Multi-point data transmission method based on private cloud | |
| US10877848B2 (en) | Processing I/O operations in parallel while maintaining read/write consistency using range and priority queues in a data protection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20180517 Address after: 210012 No. 68, Bauhinia Road, Ningnan street, Yuhuatai District, Nanjing, Jiangsu Applicant after: Nanjing Zhongxing Software Co., Ltd. Address before: 518057 Nanshan District high tech Industrial Park, Shenzhen, Guangdong, Ministry of justice, Zhongxing Road, South China road. Applicant before: ZTE Corporation |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20191107 Address after: 518057 Nanshan District science and Technology Industrial Park, Guangdong high tech Industrial Park, ZTE building Patentee after: ZTE Communications Co., Ltd. Address before: 210012 Nanjing, Yuhuatai District, South Street, Bauhinia Road, No. 68 Patentee before: Nanjing Zhongxing Software Co., Ltd. |