CN105630903B - Method and device for rapidly storing mass data - Google Patents
Method and device for rapidly storing mass data Download PDFInfo
- Publication number
- CN105630903B CN105630903B CN201510962553.XA CN201510962553A CN105630903B CN 105630903 B CN105630903 B CN 105630903B CN 201510962553 A CN201510962553 A CN 201510962553A CN 105630903 B CN105630903 B CN 105630903B
- Authority
- CN
- China
- Prior art keywords
- data
- unstructured
- storing
- file
- structured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
Abstract
The invention provides a method and a device for rapidly storing mass data, wherein the method comprises the following steps: aiming at a data packet acquired in real time in a network, storing structured data in the data packet as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory; according to a set first time interval, regularly checking the storage directory, migrating the structure files in the storage directory to a structured database, and migrating the compressed files in the storage directory to an unstructured database; storing the structure file in the structured database into a database server; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; and storing the storage address of each compressed file in the unstructured server into a database server.
Description
Technical Field
The invention relates to the field of internet technologies, in particular to a method and a device for rapidly storing mass data.
Background
With the development of the information society and the large-scale popularization of the internet, the instantaneous data generated by the internet is more and more. On one hand, from the rapid development trend of the internet, the requirement on data storage is greater and greater, and the requirement on storage speed is faster and faster; on the other hand, a higher demand is placed on rapid retrieval of data. The existing traditional data storage mode and retrieval mode are low in efficiency and cannot meet the requirements for quickly storing and retrieving the instantaneous mass data of the internet. Currently, one solution to this problem is structured storage, i.e. database storage, which has the disadvantage of inefficient storage and querying when the amount of data is massive and contains unstructured data such as pictures, videos, etc. Another solution is unstructured Storage, namely SAN (Storage Area Network, Network Area Storage) or NAS (Network attached Storage); the disadvantages of SAN storage are: the cost is high, the method is not suitable for a large-scale storage system, the data sharing performance is poor, and multi-user file sharing cannot be supported; the disadvantages of NAS storage are: the network bandwidth is shared, the concurrency performance is poor, the performance is further reduced along with the expansion of the system, the file server needs to be traversed no matter simple list data or unstructured data such as videos and pictures are inquired, and the inquiry efficiency is low.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method and a device for rapidly storing mass data, improving the storage efficiency of the mass data, reducing the maintenance cost of the mass data and improving the retrieval speed of the mass data.
The invention adopts the technical scheme that the method for rapidly storing the mass data comprises the following steps:
step 1: aiming at a data packet acquired in real time in a network, storing structured data in the data packet as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
step 2: according to a set first time interval, regularly checking the storage directory, migrating the structure files in the storage directory to a structured database, and migrating the compressed files in the storage directory to an unstructured database;
and step 3: storing the structure file in the structured database into a database server; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; and storing the storage address of each compressed file in the unstructured server into a database server.
Further, the step 1 includes:
step 11: analyzing a data packet acquired in real time in a network, and acquiring a protocol type and a protocol parameter of the data packet by analyzing a packet header of the data packet;
step 12: decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
step 13: and storing the structured data in the data packet as a structure file, storing the unstructured data in the data packet as a compressed file, and respectively storing the structure file and the compressed file into specified storage directories.
Further, the step 3 specifically includes:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
Further, the method for rapidly storing mass data further includes:
according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server;
acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses;
and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
The invention also provides a device for rapidly storing mass data, which comprises:
a data processing module: the data processing system is used for storing structured data in a data packet acquired in real time in a network as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
a file migration module: the system comprises a storage directory, a structured database and a compressed file, wherein the storage directory is used for periodically checking the storage directory according to a set first time interval, transferring the structured file in the storage directory to the structured database, and transferring the compressed file in the storage directory to the unstructured database;
a fast storage module: the database server is used for storing the structure files in the structured database; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; and storing the storage address of each compressed file in the unstructured server into a database server.
Further, the data processing module includes:
an acquisition module: the device comprises a data packet acquisition module, a data packet processing module and a data processing module, wherein the data packet acquisition module is used for acquiring a data packet from a network;
a decomposition module: the data processing device is used for decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
a storage module: the data processing system is used for storing the structured data in the data package as a structure file, storing the unstructured data in the data package as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory.
Further, the fast storage module is specifically configured to:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
Further, the apparatus for rapidly storing mass data further includes:
a data retrieval module: according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server; acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses; and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
By adopting the technical scheme, the invention at least has the following advantages:
the method and the device for rapidly storing the mass data adopt a mode of combining the structured data storage with the unstructured data storage to rapidly store the mass data. In the invention, the unstructured data is stored as a compressed file, so that the problem of large occupation of data storage space can be solved. The invention stores the structured data into a database server and stores the unstructured data into a plurality of unstructured servers. The use of a plurality of unstructured servers can improve the processing speed of data storage, and the operation performance of the unstructured servers cannot be affected due to the fact that the amount of data needing to be stored is too large.
Drawings
FIG. 1 is a flowchart illustrating a mass data fast storage method according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating a mass data fast storage method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a mass data fast storage device according to a third embodiment of the present invention.
Detailed Description
To further explain the technical means and effects of the present invention adopted to achieve the intended purpose, the present invention will be described in detail with reference to the accompanying drawings and preferred embodiments.
A first embodiment of the present invention provides a method for rapidly storing mass data, as shown in fig. 1, including the following specific steps:
step S101: aiming at a data packet acquired in real time in a network, storing structured data in the data packet as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
specifically, the step S101 includes:
a1: analyzing a data packet acquired in real time in a network, and acquiring a protocol type and a protocol parameter of the data packet by analyzing a packet header of the data packet;
a2: decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
a3: and storing the structured data in the data packet as a structure file, storing the unstructured data in the data packet as a compressed file, and respectively storing the structure file and the compressed file into specified storage directories.
Step S102: and periodically checking the storage directory according to a set first time interval, migrating the structure files in the storage directory to a structured database, and migrating the compressed files in the storage directory to an unstructured database.
Step S103: storing the structure file in the structured database into a database server; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; storing the storage address of each compressed file in the unstructured server into a database server;
specifically, the step S103 specifically includes:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
Further, the method for rapidly storing mass data further includes:
according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server;
acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses;
and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
A second embodiment of the present invention provides a method for rapidly storing mass data, as shown in fig. 2, including the following specific steps:
step S201: aiming at a data packet acquired in real time in a network, storing structured data in the data packet as a BCP file, storing unstructured data in the data packet associated with the structured data as a ZIP file, and respectively storing the BCP file and the ZIP file into specified storage directories;
specifically, the step S201 includes:
b1: acquiring a data packet in real time in a network, wherein the data packet comprises a plurality of protocol types;
b2: reading protocol configuration information in a system memory, wherein the protocol configuration information includes processing parameters of various protocol types, such as: file size, file attachment, date saved, etc.; the system loads a storage configuration file, wherein the storage configuration file comprises storage position information of structured data and unstructured data;
b3: analyzing the data packet, and obtaining the protocol type and protocol parameters of the data packet by analyzing the packet head of the data packet;
b4: decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
b5: and storing the structured data in the data packet into a BCP file, storing the unstructured data in the data packet into a ZIP file, and respectively storing the BCP file and the ZIP file into specified storage directories according to the storage position information in the storage configuration file.
Step S202: monitoring the storage directory through a daemon thread, migrating a BCP (binary-coded process) file in the storage directory into a structured database, and migrating a ZIP (binary-coded process) file in the storage directory into an unstructured database;
specifically, the step S202 includes:
c1: reading a migration configuration file in a system memory;
c2: starting a daemon process, reading the migration configuration file again every 8 minutes, and restarting a timed migration task according to the migration configuration file;
c3: executing a timed migration task, and migrating the BCP file to a structured database when the BCP file is stored in the storage directory; and when the ZIP file is stored in the storage directory, migrating the ZIP file to an unstructured database.
Step S203: storing the BCP file in the structured database into a database server; respectively and correspondingly storing ZIP files in the unstructured database into different unstructured servers; and storing the storage address of each ZIP file in the unstructured server into a database server.
Specifically, the step S203 includes:
acquiring a BCP file from a structured database in a multithreading mode every 8 minutes, decomposing the BCP file into a text file and a statistical file, and storing structured data in the text file and structured data in the statistical file into a data table in a database server;
obtaining a ZIP file from an unstructured database in a multithreading mode every 8 minutes, respectively and correspondingly storing the ZIP file into unstructured servers with different protocol types according to the protocol type of the ZIP file, and storing unstructured data in the ZIP file into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
Step S204: according to the retrieval requirements of users, retrieving the structured data stored in the database server and the unstructured data stored in each unstructured server, and displaying the retrieved structured data and the unstructured data associated with the structured data;
specifically, the step S204 includes:
d1: according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server;
d2: acquiring the storage addresses of the structured data and the unstructured data in the data table meeting the retrieval requirement;
d3: storing the storage address of the unstructured data into a system memory;
d4: returning the storage addresses of the structured data and the unstructured data to a page in a table form;
d5: reading the storage address of the unstructured data in the system memory;
d6: and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server and returning the unstructured data to the page.
A third embodiment of the present invention, which is a device for fast storing mass data and corresponds to the first embodiment, as shown in fig. 3, includes the following components:
(1) the data processing module 301: the data processing system is used for storing structured data in a data packet acquired in real time in a network as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
specifically, the data processing module 301 includes:
an acquisition module: the device comprises a data packet acquisition module, a data packet processing module and a data processing module, wherein the data packet acquisition module is used for acquiring a data packet from a network;
a decomposition module: the data processing device is used for decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
a storage module: the data processing system is used for storing the structured data in the data package as a structure file, storing the unstructured data in the data package as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory.
(2) The file migration module 302: the system comprises a storage directory, a structured database and a compressed file, wherein the storage directory is used for periodically checking the storage directory according to a set first time interval, migrating the structured file in the storage directory to the structured database, and migrating the compressed file in the storage directory to the unstructured database.
(3) The fast storage module 303: the database server is used for storing the structure files in the structured database; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; storing the storage address of each compressed file in the unstructured server into a database server;
specifically, the fast storage module 303 is specifically configured to:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
Further, the apparatus for rapidly storing mass data further includes:
a data retrieval module: the data table searching method comprises the steps of finding a data table meeting a retrieval requirement in a database server according to the retrieval requirement of a user; acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses; and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
The method and the device for rapidly storing the mass data adopt a mode of combining the structured data storage with the unstructured data storage to rapidly store the mass data. In the invention, the unstructured data is stored as a compressed file, so that the problem of large occupation of data storage space can be solved. The invention finally stores the structured data and the unstructured data into a database server and a plurality of unstructured servers. Storing basic information of data and a storage address of unstructured data in a database server; the unstructured server stores various unstructured large-capacity data such as pictures and videos. The use of a plurality of unstructured servers can improve the processing speed of data storage, and the operation performance of the unstructured servers cannot be affected due to the fact that the amount of data needing to be stored is too large.
While the invention has been described in connection with specific embodiments thereof, it is to be understood that it is intended by the appended drawings and description that the invention may be embodied in other specific forms without departing from the spirit or scope of the invention.
Claims (6)
1. A method for rapidly storing mass data is characterized by comprising the following steps:
step 1: aiming at a data packet acquired in real time in a network, storing structured data in the data packet as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
step 2: according to a set first time interval, regularly checking the storage directory, migrating the structure files in the storage directory to a structured database, and migrating the compressed files in the storage directory to an unstructured database;
and step 3: storing the structure file in the structured database into a database server; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; storing the storage address of each compressed file in the unstructured server into a database server;
the step 3 specifically includes:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
2. The method for rapidly storing mass data according to claim 1, wherein the step 1 comprises:
step 11: analyzing a data packet acquired in real time in a network, and acquiring a protocol type and a protocol parameter of the data packet by analyzing a packet header of the data packet;
step 12: decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
step 13: and storing the structured data in the data packet as a structure file, storing the unstructured data in the data packet as a compressed file, and respectively storing the structure file and the compressed file into specified storage directories.
3. The method for rapidly storing mass data according to claim 1, wherein the method for rapidly storing mass data further comprises:
according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server;
acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses;
and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
4. An apparatus for fast storage of mass data, comprising:
a data processing module: the data processing system is used for storing structured data in a data packet acquired in real time in a network as a structure file, storing unstructured data in the data packet associated with the structured data as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory;
a file migration module: the system comprises a storage directory, a structured database and a compressed file, wherein the storage directory is used for periodically checking the storage directory according to a set first time interval, transferring the structured file in the storage directory to the structured database, and transferring the compressed file in the storage directory to the unstructured database;
a fast storage module: the database server is used for storing the structure files in the structured database; respectively and correspondingly storing the compressed files in the unstructured database into different unstructured servers; storing the storage address of each compressed file in the unstructured server into a database server;
the fast storage module is specifically configured to:
regularly acquiring a structure file from a structured database in a multithreading mode according to a set second time interval, decomposing the structure file into a structure text file and a structure statistical file, and storing the structured data in the structure text file and the structured data in the structure statistical file into a data table in a database server;
acquiring compressed files from an unstructured database regularly in a multithreading mode according to a set third time interval, respectively and correspondingly storing the compressed files into unstructured servers with different protocol types according to the protocol types of the compressed files, and storing unstructured data in the compressed files into a data table in the unstructured server; then storing the storage address of the unstructured data in the unstructured server into a data table in a database server;
and according to the incidence relation between the structured data and the unstructured data, associating the data table in which the structured data are stored with the data table in which the storage addresses of the unstructured data are stored in a database server.
5. The apparatus for rapidly storing mass data according to claim 4, wherein the data processing module comprises:
an acquisition module: the device comprises a data packet acquisition module, a data packet processing module and a data processing module, wherein the data packet acquisition module is used for acquiring a data packet from a network;
a decomposition module: the data processing device is used for decomposing data in the data packet into structured data and unstructured data according to the protocol type and the protocol parameters of the data packet;
a storage module: the data processing system is used for storing the structured data in the data package as a structure file, storing the unstructured data in the data package as a compressed file, and respectively storing the structure file and the compressed file into a specified storage directory.
6. The apparatus for rapidly storing mass data according to claim 4, wherein said apparatus for rapidly storing mass data further comprises:
a data retrieval module: according to the retrieval requirement of a user, finding a data table meeting the retrieval requirement in a database server; acquiring storage addresses of structured data and unstructured data in a data table meeting the retrieval requirement, and returning and displaying the storage addresses; and according to the storage address of the unstructured data, acquiring the unstructured data from the corresponding unstructured server, and returning and displaying the unstructured data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510962553.XA CN105630903B (en) | 2015-12-21 | 2015-12-21 | Method and device for rapidly storing mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510962553.XA CN105630903B (en) | 2015-12-21 | 2015-12-21 | Method and device for rapidly storing mass data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105630903A CN105630903A (en) | 2016-06-01 |
| CN105630903B true CN105630903B (en) | 2020-02-21 |
Family
ID=56045836
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510962553.XA Active CN105630903B (en) | 2015-12-21 | 2015-12-21 | Method and device for rapidly storing mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105630903B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7804610B2 (en) * | 2005-06-30 | 2010-09-28 | Konica Minolta Business Technologies, Inc. | Image forming apparatus, image forming method and computer readable media |
| CN102521138A (en) * | 2011-11-28 | 2012-06-27 | 成都市华为赛门铁克科技有限公司 | Method and device of data migration trigger |
| CN103491187A (en) * | 2013-09-30 | 2014-01-01 | 华南理工大学 | Big data unified analyzing and processing method based on cloud computing |
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103678603A (en) * | 2013-12-13 | 2014-03-26 | 江苏物联网研究发展中心 | Multi-source heterogeneous data efficient converging and storing frame system |
| CN104102652A (en) * | 2013-04-08 | 2014-10-15 | 国家电网公司 | Unstructured data storage system and method |
| CN104765840A (en) * | 2015-04-16 | 2015-07-08 | 成都睿峰科技有限公司 | Big data distributed storage method and device |
| CN105045820A (en) * | 2015-06-25 | 2015-11-11 | 浙江立元通信技术股份有限公司 | Method for processing video image information of mass data and database system |
-
2015
- 2015-12-21 CN CN201510962553.XA patent/CN105630903B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7804610B2 (en) * | 2005-06-30 | 2010-09-28 | Konica Minolta Business Technologies, Inc. | Image forming apparatus, image forming method and computer readable media |
| CN102521138A (en) * | 2011-11-28 | 2012-06-27 | 成都市华为赛门铁克科技有限公司 | Method and device of data migration trigger |
| CN104102652A (en) * | 2013-04-08 | 2014-10-15 | 国家电网公司 | Unstructured data storage system and method |
| CN103491187A (en) * | 2013-09-30 | 2014-01-01 | 华南理工大学 | Big data unified analyzing and processing method based on cloud computing |
| CN103544322A (en) * | 2013-11-08 | 2014-01-29 | 北京邮电大学 | Hotspot metadata management method based on server cluster |
| CN103678603A (en) * | 2013-12-13 | 2014-03-26 | 江苏物联网研究发展中心 | Multi-source heterogeneous data efficient converging and storing frame system |
| CN104765840A (en) * | 2015-04-16 | 2015-07-08 | 成都睿峰科技有限公司 | Big data distributed storage method and device |
| CN105045820A (en) * | 2015-06-25 | 2015-11-11 | 浙江立元通信技术股份有限公司 | Method for processing video image information of mass data and database system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105630903A (en) | 2016-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5661104B2 (en) | Method and system for search using search engine indexing and index | |
| CN103473267B (en) | Data store query method and system | |
| CN106980699B (en) | Data processing platform and system | |
| CN104125163B (en) | Data processing method and device and terminal | |
| CN103631623A (en) | Method and device for allocating application software in trunking system | |
| CN109033360B (en) | Data query method, device, server and storage medium | |
| CN104301442A (en) | Method for achieving client of access object storage cluster based on fuse | |
| CN104239353B (en) | WEB classification control and log audit method | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN105072196A (en) | Distributed data package storage, recall method and system | |
| CN104834650A (en) | Method and system for generating effective query tasks | |
| CN105447146A (en) | Massive data collecting and exchanging system and method | |
| CN102779138A (en) | Hard disk access method of real time data | |
| CN103716384A (en) | Method and device for realizing cloud storage data synchronization in cross-data-center manner | |
| CN111858730A (en) | Data importing and exporting device, method, equipment and medium of graph database | |
| CN105306581A (en) | Router and cloud disk based data synchronization method | |
| CN105554181B (en) | A kind of DNS log compression method and apparatus | |
| CN110989935A (en) | Data processing and transmitting method and device of flash system | |
| CN104881475A (en) | Method and system for randomly sampling big data | |
| US11210212B2 (en) | Conflict resolution and garbage collection in distributed databases | |
| CN104778252A (en) | Index storage method and index storage device | |
| WO2017000592A1 (en) | Data processing method, apparatus and system | |
| CN104270272B (en) | A kind of electric energy quality monitoring data management scheme based on mobile Agent | |
| CN103092955B (en) | Checkpointed method, Apparatus and system | |
| CN105630903B (en) | Method and device for rapidly storing mass data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |